2024-03-26T08:06:31.759 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-26T08:06:31.759 DEBUG:teuthology.run:Teuthology command: teuthology --name mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi --archive /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622859 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/iozone}} --verbose --owner mchangir -- /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622859/orig.config.yaml 2024-03-26T08:06:31.808 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T08:06:32.238 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622859 branch: wip-mchangir-testing-main-20240318.032620 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/iozone}} email: mchangir@redhat.com first_in_suite: false job_id: '7622859' kernel: flavor: default kdb: true sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-mchangir-testing-main-20240318.032620 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 1 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: dry_run: true install: ceph: flavor: default sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-testing-main-20240318.032620 sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 owner: mchangir priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3185 sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 sleep_before_teardown: 0 subset: 155/256 suite: fs suite_branch: wip-mchangir-testing-main-20240318.032620 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi043.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMH1jU9U0sU6ZzsJUSlAx35h4aq/E8tu/suOdSwcUCjbL9GoT5tueabtH+nGQtab6zRcT5my8TA1K63WPLCVWjo= smithi052.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPDmoGQ8x1m+DjjCuVi05h5AL0D32ZkYNW7vpMtjqsfAkKVxWnwHq4rVn+0LsMX/EWuhyE38tYEc0w8fR6weaFk= smithi183.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHnsG7FtOLb0DVBiNZN4/949cs/40B5XkLLnMRNH0GgTSj6HsaOofln4VgLEUFw2+lTtox1LiudHiKLGWb2qHGQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - ceph-fuse: null - check-counter: {} - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-26_07:59:35 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-26T08:06:32.239 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa; will attempt to use it 2024-03-26T08:06:32.239 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks 2024-03-26T08:06:32.240 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-26T08:06:32.241 INFO:teuthology.task.internal:Checking packages... 2024-03-26T08:06:32.264 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3be9a5c9bc793e11e2800a8c0c696e8b46742033' 2024-03-26T08:06:32.265 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-26T08:06:32.265 INFO:teuthology.packaging:ref: None 2024-03-26T08:06:32.265 INFO:teuthology.packaging:tag: None 2024-03-26T08:06:32.265 INFO:teuthology.packaging:branch: wip-mchangir-testing-main-20240318.032620 2024-03-26T08:06:32.265 INFO:teuthology.packaging:sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T08:06:32.265 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-mchangir-testing-main-20240318.032620 2024-03-26T08:06:32.410 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2145.g3be9a5c9 2024-03-26T08:06:32.411 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-26T08:06:32.419 INFO:teuthology.task.internal:no buildpackages task found 2024-03-26T08:06:32.419 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-26T08:06:32.438 INFO:teuthology.task.internal:Saving configuration 2024-03-26T08:06:32.456 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-26T08:06:32.464 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T08:06:32.489 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622859', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 08:01:41.334318', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMH1jU9U0sU6ZzsJUSlAx35h4aq/E8tu/suOdSwcUCjbL9GoT5tueabtH+nGQtab6zRcT5my8TA1K63WPLCVWjo='} 2024-03-26T08:06:32.546 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi052.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622859', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 08:01:41.335277', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPDmoGQ8x1m+DjjCuVi05h5AL0D32ZkYNW7vpMtjqsfAkKVxWnwHq4rVn+0LsMX/EWuhyE38tYEc0w8fR6weaFk='} 2024-03-26T08:06:32.567 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622859', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 08:01:41.332967', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHnsG7FtOLb0DVBiNZN4/949cs/40B5XkLLnMRNH0GgTSj6HsaOofln4VgLEUFw2+lTtox1LiudHiKLGWb2qHGQ='} 2024-03-26T08:06:32.567 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-26T08:06:32.574 INFO:teuthology.task.internal:roles: ubuntu@smithi043.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-26T08:06:32.575 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-26T08:06:32.575 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-26T08:06:32.575 INFO:teuthology.run_tasks:Running task console_log... 2024-03-26T08:06:32.714 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5cdb6d50d0>, signals=[15]) 2024-03-26T08:06:32.714 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-26T08:06:32.720 INFO:teuthology.task.internal:Opening connections... 2024-03-26T08:06:32.720 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi043.front.sepia.ceph.com 2024-03-26T08:06:32.722 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T08:06:32.833 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi052.front.sepia.ceph.com 2024-03-26T08:06:32.834 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T08:06:32.939 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi183.front.sepia.ceph.com 2024-03-26T08:06:32.940 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T08:06:33.042 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-26T08:06:33.049 DEBUG:teuthology.orchestra.run.smithi043:> uname -m 2024-03-26T08:06:33.068 INFO:teuthology.orchestra.run.smithi043.stdout:x86_64 2024-03-26T08:06:33.068 DEBUG:teuthology.orchestra.run.smithi043:> cat /etc/os-release 2024-03-26T08:06:33.126 INFO:teuthology.orchestra.run.smithi043.stdout:NAME="CentOS Stream" 2024-03-26T08:06:33.126 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION="9" 2024-03-26T08:06:33.126 INFO:teuthology.orchestra.run.smithi043.stdout:ID="centos" 2024-03-26T08:06:33.126 INFO:teuthology.orchestra.run.smithi043.stdout:ID_LIKE="rhel fedora" 2024-03-26T08:06:33.126 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_ID="9" 2024-03-26T08:06:33.126 INFO:teuthology.orchestra.run.smithi043.stdout:PLATFORM_ID="platform:el9" 2024-03-26T08:06:33.126 INFO:teuthology.orchestra.run.smithi043.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-26T08:06:33.127 INFO:teuthology.orchestra.run.smithi043.stdout:ANSI_COLOR="0;31" 2024-03-26T08:06:33.127 INFO:teuthology.orchestra.run.smithi043.stdout:LOGO="fedora-logo-icon" 2024-03-26T08:06:33.127 INFO:teuthology.orchestra.run.smithi043.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-26T08:06:33.127 INFO:teuthology.orchestra.run.smithi043.stdout:HOME_URL="https://centos.org/" 2024-03-26T08:06:33.127 INFO:teuthology.orchestra.run.smithi043.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T08:06:33.127 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-26T08:06:33.127 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T08:06:33.128 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server 2024-03-26T08:06:33.155 DEBUG:teuthology.orchestra.run.smithi052:> uname -m 2024-03-26T08:06:33.174 INFO:teuthology.orchestra.run.smithi052.stdout:x86_64 2024-03-26T08:06:33.174 DEBUG:teuthology.orchestra.run.smithi052:> cat /etc/os-release 2024-03-26T08:06:33.232 INFO:teuthology.orchestra.run.smithi052.stdout:NAME="CentOS Stream" 2024-03-26T08:06:33.232 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION="9" 2024-03-26T08:06:33.232 INFO:teuthology.orchestra.run.smithi052.stdout:ID="centos" 2024-03-26T08:06:33.232 INFO:teuthology.orchestra.run.smithi052.stdout:ID_LIKE="rhel fedora" 2024-03-26T08:06:33.232 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION_ID="9" 2024-03-26T08:06:33.232 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM_ID="platform:el9" 2024-03-26T08:06:33.232 INFO:teuthology.orchestra.run.smithi052.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-26T08:06:33.232 INFO:teuthology.orchestra.run.smithi052.stdout:ANSI_COLOR="0;31" 2024-03-26T08:06:33.232 INFO:teuthology.orchestra.run.smithi052.stdout:LOGO="fedora-logo-icon" 2024-03-26T08:06:33.232 INFO:teuthology.orchestra.run.smithi052.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-26T08:06:33.232 INFO:teuthology.orchestra.run.smithi052.stdout:HOME_URL="https://centos.org/" 2024-03-26T08:06:33.233 INFO:teuthology.orchestra.run.smithi052.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T08:06:33.233 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-26T08:06:33.233 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T08:06:33.233 INFO:teuthology.lock.ops:Updating smithi052.front.sepia.ceph.com on lock server 2024-03-26T08:06:33.255 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2024-03-26T08:06:33.273 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2024-03-26T08:06:33.273 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2024-03-26T08:06:33.328 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream" 2024-03-26T08:06:33.329 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="9" 2024-03-26T08:06:33.329 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos" 2024-03-26T08:06:33.329 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora" 2024-03-26T08:06:33.329 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="9" 2024-03-26T08:06:33.329 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el9" 2024-03-26T08:06:33.329 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-26T08:06:33.329 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2024-03-26T08:06:33.329 INFO:teuthology.orchestra.run.smithi183.stdout:LOGO="fedora-logo-icon" 2024-03-26T08:06:33.329 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-26T08:06:33.329 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/" 2024-03-26T08:06:33.329 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T08:06:33.329 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-26T08:06:33.330 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T08:06:33.330 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2024-03-26T08:06:33.346 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-26T08:06:33.357 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-26T08:06:33.379 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-26T08:06:33.379 DEBUG:teuthology.orchestra.run.smithi043:> test '!' -e /home/ubuntu/cephtest 2024-03-26T08:06:33.382 DEBUG:teuthology.orchestra.run.smithi052:> test '!' -e /home/ubuntu/cephtest 2024-03-26T08:06:33.384 DEBUG:teuthology.orchestra.run.smithi183:> test '!' -e /home/ubuntu/cephtest 2024-03-26T08:06:33.400 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-26T08:06:33.412 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-26T08:06:33.413 DEBUG:teuthology.orchestra.run.smithi043:> test -z $(ls -A /var/lib/ceph) 2024-03-26T08:06:33.441 DEBUG:teuthology.orchestra.run.smithi052:> test -z $(ls -A /var/lib/ceph) 2024-03-26T08:06:33.444 DEBUG:teuthology.orchestra.run.smithi183:> test -z $(ls -A /var/lib/ceph) 2024-03-26T08:06:33.470 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-26T08:06:33.551 INFO:teuthology.run_tasks:Running task kernel... 2024-03-26T08:06:33.565 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'} 2024-03-26T08:06:33.565 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}, 'mon.c': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}}, timeout 300 2024-03-26T08:06:33.566 DEBUG:teuthology.orchestra.run.smithi043:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-26T08:06:33.566 DEBUG:teuthology.orchestra.run.smithi052:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-26T08:06:33.567 DEBUG:teuthology.orchestra.run.smithi183:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-26T08:06:33.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T08:06:33.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T08:06:33.585 DEBUG:teuthology.task.kernel:sha1 for mon.c is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-26T08:06:33.585 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-26T08:06:33.585 DEBUG:teuthology.orchestra.run.smithi183:> uname -r 2024-03-26T08:06:33.585 DEBUG:teuthology.task.kernel:sha1 for mon.b is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-26T08:06:33.585 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-26T08:06:33.585 DEBUG:teuthology.orchestra.run.smithi052:> uname -r 2024-03-26T08:06:33.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T08:06:33.587 DEBUG:teuthology.task.kernel:sha1 for mon.a is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-26T08:06:33.587 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-26T08:06:33.587 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-03-26T08:06:33.639 INFO:teuthology.orchestra.run.smithi183.stdout:5.14.0-361.el9.x86_64 2024-03-26T08:06:33.640 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-26T08:06:33.640 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-26T08:06:33.640 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-26T08:06:33.644 INFO:teuthology.orchestra.run.smithi043.stdout:5.14.0-361.el9.x86_64 2024-03-26T08:06:33.645 INFO:teuthology.orchestra.run.smithi052.stdout:5.14.0-361.el9.x86_64 2024-03-26T08:06:33.645 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-26T08:06:33.645 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-26T08:06:33.645 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-26T08:06:33.646 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-26T08:06:33.646 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-26T08:06:33.646 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-26T08:06:33.792 DEBUG:teuthology.orchestra.run.smithi052:> sudo yum upgrade -y linux-firmware 2024-03-26T08:06:33.794 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum upgrade -y linux-firmware 2024-03-26T08:06:33.797 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum upgrade -y linux-firmware 2024-03-26T08:06:36.235 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS-9 - AppStream 7.4 MB/s | 6.7 MB 00:00 2024-03-26T08:06:36.389 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS-9 - AppStream 6.7 MB/s | 6.7 MB 00:00 2024-03-26T08:06:36.536 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS-9 - AppStream 6.2 MB/s | 6.7 MB 00:01 2024-03-26T08:06:37.847 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS-9 - BaseOS 6.6 MB/s | 2.0 MB 00:00 2024-03-26T08:06:37.868 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS-9 - BaseOS 3.7 MB/s | 2.0 MB 00:00 2024-03-26T08:06:37.961 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS-9 - BaseOS 7.2 MB/s | 2.0 MB 00:00 2024-03-26T08:06:38.794 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.0 MB 00:00 2024-03-26T08:06:39.063 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 9.3 MB/s | 8.0 MB 00:00 2024-03-26T08:06:39.126 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - BaseOS 8.7 MB/s | 8.0 MB 00:00 2024-03-26T08:06:40.736 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-03-26T08:06:44.055 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 4.9 MB/s | 19 MB 00:03 2024-03-26T08:06:45.003 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - CRB 9.4 MB/s | 6.1 MB 00:00 2024-03-26T08:06:45.809 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - AppStream 3.3 MB/s | 19 MB 00:05 2024-03-26T08:06:46.722 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-03-26T08:06:47.905 INFO:teuthology.orchestra.run.smithi052.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-03-26T08:06:48.089 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.1 MB 00:00 2024-03-26T08:06:49.832 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 16 kB 00:00 2024-03-26T08:06:51.785 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.1 MB 00:02 2024-03-26T08:06:52.981 INFO:teuthology.orchestra.run.smithi052.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-03-26T08:06:53.486 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 16 kB 00:00 2024-03-26T08:06:53.541 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 5.8 MB/s | 21 MB 00:03 2024-03-26T08:06:55.323 INFO:teuthology.orchestra.run.smithi052.stdout:Dependencies resolved. 2024-03-26T08:06:55.330 INFO:teuthology.orchestra.run.smithi052.stdout:================================================================================ 2024-03-26T08:06:55.330 INFO:teuthology.orchestra.run.smithi052.stdout: Package Arch Version Repository Size 2024-03-26T08:06:55.330 INFO:teuthology.orchestra.run.smithi052.stdout:================================================================================ 2024-03-26T08:06:55.330 INFO:teuthology.orchestra.run.smithi052.stdout:Upgrading: 2024-03-26T08:06:55.331 INFO:teuthology.orchestra.run.smithi052.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-26T08:06:55.331 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-26T08:06:55.331 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction Summary 2024-03-26T08:06:55.331 INFO:teuthology.orchestra.run.smithi052.stdout:================================================================================ 2024-03-26T08:06:55.331 INFO:teuthology.orchestra.run.smithi052.stdout:Upgrade 1 Package 2024-03-26T08:06:55.331 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-26T08:06:55.331 INFO:teuthology.orchestra.run.smithi052.stdout:Total download size: 375 M 2024-03-26T08:06:55.331 INFO:teuthology.orchestra.run.smithi052.stdout:Downloading Packages: 2024-03-26T08:06:55.370 INFO:teuthology.orchestra.run.smithi183.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-03-26T08:06:58.849 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-26T08:07:00.461 INFO:teuthology.orchestra.run.smithi183.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-03-26T08:07:01.225 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-26T08:07:01.234 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-26T08:07:01.235 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-03-26T08:07:01.235 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-26T08:07:01.235 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrading: 2024-03-26T08:07:01.235 INFO:teuthology.orchestra.run.smithi043.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-26T08:07:01.235 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:07:01.235 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-03-26T08:07:01.235 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-26T08:07:01.235 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrade 1 Package 2024-03-26T08:07:01.235 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:07:01.235 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 375 M 2024-03-26T08:07:01.235 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-03-26T08:07:02.704 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-03-26T08:07:02.714 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-03-26T08:07:02.714 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2024-03-26T08:07:02.714 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-03-26T08:07:02.714 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrading: 2024-03-26T08:07:02.714 INFO:teuthology.orchestra.run.smithi183.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-26T08:07:02.714 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-26T08:07:02.714 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-03-26T08:07:02.714 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-03-26T08:07:02.714 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrade 1 Package 2024-03-26T08:07:02.714 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-26T08:07:02.715 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 375 M 2024-03-26T08:07:02.715 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-03-26T08:07:07.398 INFO:teuthology.orchestra.run.smithi052.stdout:linux-firmware-20240219-143.el9.noarch.rpm 31 MB/s | 375 MB 00:12 2024-03-26T08:07:07.399 INFO:teuthology.orchestra.run.smithi052.stdout:-------------------------------------------------------------------------------- 2024-03-26T08:07:07.400 INFO:teuthology.orchestra.run.smithi052.stdout:Total 31 MB/s | 375 MB 00:12 2024-03-26T08:07:07.400 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction check 2024-03-26T08:07:07.483 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction check succeeded. 2024-03-26T08:07:07.483 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction test 2024-03-26T08:07:09.966 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction test succeeded. 2024-03-26T08:07:09.966 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction 2024-03-26T08:07:13.416 INFO:teuthology.orchestra.run.smithi043.stdout:linux-firmware-20240219-143.el9.noarch.rpm 31 MB/s | 375 MB 00:12 2024-03-26T08:07:13.417 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-03-26T08:07:13.417 INFO:teuthology.orchestra.run.smithi043.stdout:Total 31 MB/s | 375 MB 00:12 2024-03-26T08:07:13.418 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-03-26T08:07:13.511 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-03-26T08:07:13.511 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-03-26T08:07:15.002 INFO:teuthology.orchestra.run.smithi183.stdout:linux-firmware-20240219-143.el9.noarch.rpm 31 MB/s | 375 MB 00:12 2024-03-26T08:07:15.003 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-03-26T08:07:15.004 INFO:teuthology.orchestra.run.smithi183.stdout:Total 31 MB/s | 375 MB 00:12 2024-03-26T08:07:15.004 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-03-26T08:07:15.094 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-03-26T08:07:15.094 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-03-26T08:07:16.036 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-03-26T08:07:16.036 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-03-26T08:07:16.554 INFO:teuthology.orchestra.run.smithi052.stdout: Preparing : 1/1 2024-03-26T08:07:16.787 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-26T08:07:16.885 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-26T08:07:17.503 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-03-26T08:07:17.504 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-03-26T08:07:18.057 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-26T08:07:18.057 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-26T08:07:18.427 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-26T08:07:18.427 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-26T08:07:18.427 INFO:teuthology.orchestra.run.smithi052.stdout:Upgraded: 2024-03-26T08:07:18.427 INFO:teuthology.orchestra.run.smithi052.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-26T08:07:18.427 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-26T08:07:18.427 INFO:teuthology.orchestra.run.smithi052.stdout:Complete! 2024-03-26T08:07:18.743 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.b... 2024-03-26T08:07:18.744 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-26T08:07:18.897 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ 2024-03-26T08:07:18.897 DEBUG:teuthology.orchestra.run.smithi052:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-26T08:07:18.900 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi052.front.sepia.ceph.com to complete... 2024-03-26T08:07:22.668 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-03-26T08:07:22.930 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-26T08:07:23.027 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-26T08:07:23.654 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-03-26T08:07:23.877 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-26T08:07:23.951 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-26T08:07:24.226 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-26T08:07:24.226 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-26T08:07:24.615 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-26T08:07:24.616 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:07:24.616 INFO:teuthology.orchestra.run.smithi043.stdout:Upgraded: 2024-03-26T08:07:24.616 INFO:teuthology.orchestra.run.smithi043.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-26T08:07:24.616 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:07:24.616 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-26T08:07:24.920 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.a... 2024-03-26T08:07:24.920 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-26T08:07:25.067 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ 2024-03-26T08:07:25.067 DEBUG:teuthology.orchestra.run.smithi043:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-26T08:07:25.070 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi043.front.sepia.ceph.com to complete... 2024-03-26T08:07:25.150 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-26T08:07:25.150 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-26T08:07:25.557 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-26T08:07:25.558 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-26T08:07:25.558 INFO:teuthology.orchestra.run.smithi183.stdout:Upgraded: 2024-03-26T08:07:25.558 INFO:teuthology.orchestra.run.smithi183.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-26T08:07:25.558 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-26T08:07:25.558 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-03-26T08:07:25.844 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.c... 2024-03-26T08:07:25.845 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-26T08:07:25.988 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ 2024-03-26T08:07:25.988 DEBUG:teuthology.orchestra.run.smithi183:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-26T08:07:25.990 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi183.front.sepia.ceph.com to complete... 2024-03-26T08:07:33.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26 08:07:33 URL:https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [336317861/336317861] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-26T08:07:33.005 INFO:teuthology.orchestra.run.smithi052.stderr:FINISHED --2024-03-26 08:07:33-- 2024-03-26T08:07:33.005 INFO:teuthology.orchestra.run.smithi052.stderr:Total wall clock time: 14s 2024-03-26T08:07:33.005 INFO:teuthology.orchestra.run.smithi052.stderr:Downloaded: 1 files, 321M in 14s (23.2 MB/s) 2024-03-26T08:07:33.008 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.b... 2024-03-26T08:07:33.008 DEBUG:teuthology.orchestra.run.smithi052:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-26T08:07:35.044 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying... ######################################## 2024-03-26T08:07:35.108 INFO:teuthology.orchestra.run.smithi052.stdout:Preparing... ######################################## 2024-03-26T08:07:35.192 INFO:teuthology.orchestra.run.smithi052.stdout:Updating / installing... 2024-03-26T08:07:43.014 INFO:teuthology.orchestra.run.smithi052.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-03-26T08:07:50.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26 08:07:50 URL:https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [336317861/336317861] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-26T08:07:50.248 INFO:teuthology.orchestra.run.smithi043.stderr:FINISHED --2024-03-26 08:07:50-- 2024-03-26T08:07:50.248 INFO:teuthology.orchestra.run.smithi043.stderr:Total wall clock time: 25s 2024-03-26T08:07:50.248 INFO:teuthology.orchestra.run.smithi043.stderr:Downloaded: 1 files, 321M in 25s (13.0 MB/s) 2024-03-26T08:07:50.251 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.a... 2024-03-26T08:07:50.251 DEBUG:teuthology.orchestra.run.smithi043:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-26T08:07:52.288 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying... ######################################## 2024-03-26T08:07:52.355 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing... ######################################## 2024-03-26T08:07:52.439 INFO:teuthology.orchestra.run.smithi043.stdout:Updating / installing... 2024-03-26T08:07:52.723 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26 08:07:52 URL:https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [336317861/336317861] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-26T08:07:52.723 INFO:teuthology.orchestra.run.smithi183.stderr:FINISHED --2024-03-26 08:07:52-- 2024-03-26T08:07:52.724 INFO:teuthology.orchestra.run.smithi183.stderr:Total wall clock time: 27s 2024-03-26T08:07:52.724 INFO:teuthology.orchestra.run.smithi183.stderr:Downloaded: 1 files, 321M in 26s (12.2 MB/s) 2024-03-26T08:07:52.726 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.c... 2024-03-26T08:07:52.727 DEBUG:teuthology.orchestra.run.smithi183:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-26T08:07:54.659 INFO:teuthology.orchestra.run.smithi183.stdout:Verifying... ######################################## 2024-03-26T08:07:54.721 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing... ######################################## 2024-03-26T08:07:54.800 INFO:teuthology.orchestra.run.smithi183.stdout:Updating / installing... 2024-03-26T08:07:58.651 INFO:teuthology.orchestra.run.smithi052.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-26T08:08:00.291 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-03-26T08:08:02.343 INFO:teuthology.orchestra.run.smithi183.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-03-26T08:08:16.113 INFO:teuthology.orchestra.run.smithi043.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-26T08:08:17.291 INFO:teuthology.orchestra.run.smithi183.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-26T08:08:21.724 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi052.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-26T08:08:21.725 DEBUG:teuthology.orchestra.run.smithi052:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-26T08:08:21.799 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-03-26T08:08:21.799 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-03-26T08:08:21.799 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-03-26T08:08:21.799 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-03-26T08:08:21.799 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-03-26T08:08:21.799 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-03-26T08:08:21.799 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-03-26T08:08:21.799 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-26T08:08:21.800 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-26T08:08:21.800 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-26T08:08:21.800 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-26T08:08:21.800 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-26T08:08:21.800 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-26T08:08:21.800 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-26T08:08:21.800 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-26T08:08:21.800 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-26T08:08:21.800 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-26T08:08:21.800 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-26T08:08:21.801 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-26T08:08:21.801 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-26T08:08:21.801 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-26T08:08:21.801 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-26T08:08:21.801 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-26T08:08:21.801 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-26T08:08:21.801 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-26T08:08:21.801 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-26T08:08:21.801 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-26T08:08:21.801 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-03-26T08:08:21.801 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-03-26T08:08:21.801 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-03-26T08:08:21.801 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-26T08:08:21.801 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-03-26T08:08:21.802 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-26T08:08:21.802 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-26T08:08:21.802 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-26T08:08:21.802 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-03-26T08:08:21.802 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-26T08:08:21.802 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-26T08:08:21.802 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-26T08:08:21.802 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-03-26T08:08:21.802 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-26T08:08:21.802 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-26T08:08:21.802 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-26T08:08:21.802 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-26T08:08:21.803 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-26T08:08:21.803 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-26T08:08:21.803 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-26T08:08:21.803 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-26T08:08:21.803 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-26T08:08:21.803 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-26T08:08:21.803 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-26T08:08:21.803 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-26T08:08:21.803 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-26T08:08:21.803 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-26T08:08:21.803 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-26T08:08:21.803 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-26T08:08:21.803 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-26T08:08:21.804 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-26T08:08:21.804 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-26T08:08:21.804 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-26T08:08:21.804 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-26T08:08:21.804 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-26T08:08:21.804 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-26T08:08:21.804 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-26T08:08:21.804 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-26T08:08:21.804 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-26T08:08:21.804 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-26T08:08:21.804 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-26T08:08:21.804 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-26T08:08:21.805 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-26T08:08:21.805 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-26T08:08:21.805 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-26T08:08:21.805 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-26T08:08:21.805 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-26T08:08:21.805 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-26T08:08:21.805 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-26T08:08:21.805 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-26T08:08:21.805 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-26T08:08:21.805 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-26T08:08:21.805 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-26T08:08:21.805 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-26T08:08:21.805 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-26T08:08:21.805 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-26T08:08:21.806 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-26T08:08:21.806 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-26T08:08:21.806 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-03-26T08:08:21.806 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-03-26T08:08:21.806 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-26T08:08:21.806 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-26T08:08:21.806 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-03-26T08:08:21.806 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-26T08:08:21.806 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-26T08:08:21.806 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-26T08:08:21.806 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-26T08:08:21.806 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-26T08:08:21.806 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-03-26T08:08:21.807 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-26T08:08:21.807 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-26T08:08:21.807 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-03-26T08:08:21.807 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-26T08:08:21.807 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-26T08:08:21.807 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-26T08:08:21.807 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-26T08:08:21.807 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-26T08:08:21.807 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-26T08:08:21.807 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-26T08:08:21.807 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-26T08:08:21.807 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-26T08:08:21.808 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-26T08:08:21.808 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-26T08:08:21.808 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-26T08:08:21.808 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-26T08:08:21.808 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-26T08:08:21.808 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-26T08:08:21.808 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-26T08:08:21.808 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-26T08:08:21.808 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-26T08:08:21.809 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-26T08:08:21.809 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-26T08:08:21.809 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-26T08:08:21.809 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-26T08:08:21.809 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-26T08:08:21.809 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-26T08:08:21.809 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-26T08:08:21.809 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-26T08:08:21.810 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-26T08:08:21.810 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-26T08:08:21.810 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-26T08:08:21.810 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-26T08:08:21.810 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-26T08:08:21.810 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-26T08:08:21.810 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-26T08:08:21.810 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-26T08:08:21.811 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-26T08:08:21.811 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-03-26T08:08:21.811 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-26T08:08:21.811 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-26T08:08:21.811 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-26T08:08:21.811 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-26T08:08:21.811 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-26T08:08:21.811 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-26T08:08:21.811 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-26T08:08:21.812 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-26T08:08:21.812 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-26T08:08:21.812 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-26T08:08:21.812 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-26T08:08:21.812 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-26T08:08:21.812 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-26T08:08:21.812 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-26T08:08:21.812 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-26T08:08:21.813 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-26T08:08:21.813 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-26T08:08:21.813 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-03-26T08:08:21.813 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-26T08:08:21.813 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-03-26T08:08:21.813 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-03-26T08:08:21.813 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-26T08:08:21.813 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-26T08:08:21.814 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-03-26T08:08:21.814 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-26T08:08:21.814 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-03-26T08:08:21.814 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-03-26T08:08:21.814 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-26T08:08:21.814 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-03-26T08:08:21.814 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-03-26T08:08:21.814 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-26T08:08:21.815 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-26T08:08:21.815 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-03-26T08:08:21.815 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-03-26T08:08:21.815 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-26T08:08:21.815 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-26T08:08:21.815 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-03-26T08:08:21.815 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-26T08:08:21.815 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-26T08:08:21.816 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-26T08:08:21.816 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-03-26T08:08:21.816 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-03-26T08:08:21.816 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-26T08:08:21.816 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-03-26T08:08:21.816 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-03-26T08:08:21.816 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-26T08:08:21.816 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-03-26T08:08:21.816 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-26T08:08:21.817 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-03-26T08:08:21.817 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-26T08:08:21.817 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-26T08:08:21.817 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-26T08:08:21.817 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-26T08:08:21.817 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-26T08:08:21.817 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-03-26T08:08:21.817 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-26T08:08:21.818 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-26T08:08:21.818 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-26T08:08:21.818 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-26T08:08:21.818 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-26T08:08:21.818 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-26T08:08:21.818 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-26T08:08:21.818 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-03-26T08:08:21.818 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-26T08:08:21.819 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-26T08:08:21.819 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-26T08:08:21.819 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-03-26T08:08:21.819 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-03-26T08:08:21.819 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-26T08:08:21.819 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-26T08:08:21.819 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-26T08:08:21.819 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-26T08:08:21.820 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-26T08:08:21.820 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-03-26T08:08:21.820 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-03-26T08:08:21.820 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-03-26T08:08:21.820 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-26T08:08:21.820 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-26T08:08:21.820 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-26T08:08:21.820 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-26T08:08:21.820 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-26T08:08:21.821 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-26T08:08:21.821 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-03-26T08:08:21.821 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-03-26T08:08:21.821 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-26T08:08:21.821 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-26T08:08:21.821 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-03-26T08:08:21.821 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-03-26T08:08:21.821 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-03-26T08:08:21.822 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-26T08:08:21.822 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-26T08:08:21.822 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-26T08:08:21.822 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-26T08:08:21.822 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-26T08:08:21.822 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-03-26T08:08:21.822 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-26T08:08:21.822 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-03-26T08:08:21.822 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-26T08:08:21.823 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-03-26T08:08:21.823 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-26T08:08:21.823 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-03-26T08:08:21.823 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-03-26T08:08:21.823 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-26T08:08:21.823 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-03-26T08:08:21.823 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-26T08:08:21.823 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-03-26T08:08:21.823 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-26T08:08:21.823 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-26T08:08:21.823 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-26T08:08:21.823 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-26T08:08:21.824 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-26T08:08:21.824 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-26T08:08:21.824 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-26T08:08:21.824 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-26T08:08:21.824 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-26T08:08:21.824 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-26T08:08:21.824 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-26T08:08:21.824 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-26T08:08:21.824 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-26T08:08:21.824 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-03-26T08:08:21.824 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-03-26T08:08:21.824 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-26T08:08:21.825 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-03-26T08:08:21.825 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-26T08:08:21.825 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-26T08:08:21.825 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-03-26T08:08:21.825 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-26T08:08:21.825 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-26T08:08:21.825 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-26T08:08:21.825 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-26T08:08:21.825 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-26T08:08:21.825 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-26T08:08:21.825 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-26T08:08:21.825 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-26T08:08:21.826 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-26T08:08:21.826 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-26T08:08:21.826 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-26T08:08:21.826 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-26T08:08:21.826 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-26T08:08:21.826 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-26T08:08:21.826 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-26T08:08:21.826 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-26T08:08:21.826 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-26T08:08:21.826 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-26T08:08:21.827 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-26T08:08:21.827 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-26T08:08:21.827 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-26T08:08:21.827 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-26T08:08:21.827 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-26T08:08:21.827 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-26T08:08:21.827 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-26T08:08:21.827 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-26T08:08:21.827 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-26T08:08:21.827 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-26T08:08:21.828 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-26T08:08:21.828 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-26T08:08:21.828 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-26T08:08:21.828 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-26T08:08:21.828 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-26T08:08:21.828 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-26T08:08:21.828 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-26T08:08:21.828 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-26T08:08:21.829 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-26T08:08:21.829 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-26T08:08:21.829 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-26T08:08:21.829 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-26T08:08:21.829 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-26T08:08:21.829 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-26T08:08:21.829 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-26T08:08:21.829 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-26T08:08:21.829 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-26T08:08:21.829 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-26T08:08:21.829 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-26T08:08:21.830 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-26T08:08:21.830 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-26T08:08:21.830 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-26T08:08:21.830 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-26T08:08:21.830 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-26T08:08:21.830 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-26T08:08:21.830 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-26T08:08:21.830 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-26T08:08:21.830 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-26T08:08:21.831 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-26T08:08:21.831 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-26T08:08:21.831 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-26T08:08:21.831 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-26T08:08:21.831 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-26T08:08:21.831 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-26T08:08:21.831 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-26T08:08:21.831 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-26T08:08:21.831 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-03-26T08:08:21.831 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-26T08:08:21.832 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-26T08:08:21.832 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-26T08:08:21.832 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-03-26T08:08:21.832 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-26T08:08:21.832 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-26T08:08:21.832 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-26T08:08:21.832 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-26T08:08:21.832 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-26T08:08:21.832 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-26T08:08:21.832 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-26T08:08:21.832 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-26T08:08:21.833 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-26T08:08:21.833 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-26T08:08:21.833 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-26T08:08:21.833 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-26T08:08:21.833 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-26T08:08:21.833 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-26T08:08:21.833 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-26T08:08:21.833 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-26T08:08:21.833 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-26T08:08:21.833 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-26T08:08:21.833 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-26T08:08:21.833 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-26T08:08:21.834 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-26T08:08:21.834 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-26T08:08:21.834 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-26T08:08:21.834 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-26T08:08:21.834 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-26T08:08:21.834 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-26T08:08:21.834 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-26T08:08:21.834 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-26T08:08:21.834 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-26T08:08:21.834 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-26T08:08:21.834 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-26T08:08:21.834 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-26T08:08:21.834 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-26T08:08:21.835 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-26T08:08:21.835 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-26T08:08:21.835 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-26T08:08:21.835 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-26T08:08:21.835 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-26T08:08:21.835 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-26T08:08:21.835 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-26T08:08:21.835 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-26T08:08:21.835 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-26T08:08:21.835 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-26T08:08:21.835 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-26T08:08:21.835 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-26T08:08:21.835 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-26T08:08:21.835 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-26T08:08:21.836 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-26T08:08:21.836 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-26T08:08:21.836 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-26T08:08:21.836 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-26T08:08:21.836 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-26T08:08:21.836 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-26T08:08:21.836 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-26T08:08:21.836 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-26T08:08:21.836 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-26T08:08:21.836 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-26T08:08:21.836 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-26T08:08:21.836 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-26T08:08:21.836 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-26T08:08:21.836 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-26T08:08:21.836 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-26T08:08:21.837 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-26T08:08:21.837 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-26T08:08:21.837 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-26T08:08:21.837 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-26T08:08:21.837 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-26T08:08:21.837 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-26T08:08:21.837 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-26T08:08:21.837 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-26T08:08:21.837 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-26T08:08:21.837 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-26T08:08:21.837 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-26T08:08:21.837 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-26T08:08:21.837 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-26T08:08:21.837 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-26T08:08:21.837 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-26T08:08:21.838 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-26T08:08:21.838 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-26T08:08:21.838 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-26T08:08:21.838 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-26T08:08:21.838 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-26T08:08:21.838 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-26T08:08:21.838 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-26T08:08:21.838 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-26T08:08:21.838 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-26T08:08:21.838 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-26T08:08:21.838 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-26T08:08:21.838 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-03-26T08:08:21.838 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-26T08:08:21.839 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-26T08:08:21.839 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-26T08:08:21.839 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-26T08:08:21.839 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-26T08:08:21.839 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-26T08:08:21.839 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-26T08:08:21.839 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-26T08:08:21.839 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-26T08:08:21.839 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-26T08:08:21.839 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-26T08:08:21.839 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-26T08:08:21.839 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-26T08:08:21.840 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-26T08:08:21.840 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-26T08:08:21.840 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-26T08:08:21.840 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-26T08:08:21.840 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-26T08:08:21.840 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-26T08:08:21.840 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-26T08:08:21.840 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-26T08:08:21.840 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-26T08:08:21.840 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-26T08:08:21.840 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-26T08:08:21.840 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-26T08:08:21.840 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-26T08:08:21.841 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-26T08:08:21.841 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-26T08:08:21.841 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-26T08:08:21.841 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-26T08:08:21.841 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-26T08:08:21.841 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-26T08:08:21.841 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-26T08:08:21.841 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-26T08:08:21.841 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-26T08:08:21.841 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-26T08:08:21.841 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-26T08:08:21.841 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-26T08:08:21.841 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-26T08:08:21.841 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-26T08:08:21.841 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-26T08:08:21.842 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-03-26T08:08:21.842 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-03-26T08:08:21.842 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-26T08:08:21.842 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-26T08:08:21.842 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-03-26T08:08:21.842 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-26T08:08:21.842 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-26T08:08:21.842 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-26T08:08:21.842 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-26T08:08:21.842 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-26T08:08:21.842 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-26T08:08:21.842 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-26T08:08:21.842 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-26T08:08:21.842 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-26T08:08:21.843 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-26T08:08:21.843 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-26T08:08:21.843 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-26T08:08:21.843 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-26T08:08:21.843 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-26T08:08:21.843 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-26T08:08:21.843 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-26T08:08:21.843 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-26T08:08:21.843 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-26T08:08:21.843 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-26T08:08:21.843 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-26T08:08:21.843 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-26T08:08:21.843 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-26T08:08:21.843 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-26T08:08:21.843 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-03-26T08:08:21.843 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-26T08:08:21.844 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-26T08:08:21.844 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-26T08:08:21.844 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-26T08:08:21.844 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-03-26T08:08:21.844 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-03-26T08:08:21.844 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-26T08:08:21.844 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-26T08:08:21.844 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-26T08:08:21.844 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-26T08:08:21.844 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-26T08:08:21.844 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-26T08:08:21.844 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-26T08:08:21.844 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-03-26T08:08:21.844 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-03-26T08:08:21.845 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-26T08:08:21.845 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-03-26T08:08:21.845 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-26T08:08:21.845 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-03-26T08:08:21.845 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-26T08:08:21.845 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-03-26T08:08:21.845 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-26T08:08:21.845 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-03-26T08:08:21.845 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-03-26T08:08:21.845 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-26T08:08:21.845 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-03-26T08:08:21.845 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-26T08:08:21.845 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-03-26T08:08:21.845 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-26T08:08:21.846 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-03-26T08:08:21.846 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-26T08:08:21.846 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-03-26T08:08:21.846 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-26T08:08:21.846 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-03-26T08:08:21.846 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-03-26T08:08:21.846 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-26T08:08:21.846 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-26T08:08:21.846 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-26T08:08:21.846 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-26T08:08:21.846 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-03-26T08:08:21.846 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-26T08:08:21.846 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-03-26T08:08:21.846 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-26T08:08:21.846 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-26T08:08:21.847 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-26T08:08:21.847 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-26T08:08:21.847 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-26T08:08:21.847 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-26T08:08:21.847 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-26T08:08:21.847 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-26T08:08:21.847 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-03-26T08:08:21.847 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-26T08:08:21.847 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-03-26T08:08:21.847 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-26T08:08:21.847 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-26T08:08:21.847 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-26T08:08:21.847 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-26T08:08:21.847 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-26T08:08:21.847 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-03-26T08:08:21.848 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-26T08:08:21.848 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-26T08:08:21.848 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-26T08:08:21.848 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-26T08:08:21.848 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-03-26T08:08:21.848 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-26T08:08:21.848 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-26T08:08:21.848 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-26T08:08:21.848 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-26T08:08:21.848 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-26T08:08:21.848 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-26T08:08:21.848 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-26T08:08:21.848 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-26T08:08:21.848 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-26T08:08:21.848 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-03-26T08:08:21.849 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-03-26T08:08:21.849 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-26T08:08:21.849 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-26T08:08:21.849 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-26T08:08:21.849 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-26T08:08:21.849 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-26T08:08:21.849 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-26T08:08:21.849 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-26T08:08:21.849 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-26T08:08:21.849 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-26T08:08:21.849 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-26T08:08:21.849 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-26T08:08:21.849 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-26T08:08:21.849 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-26T08:08:21.850 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-26T08:08:21.850 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-26T08:08:21.850 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-26T08:08:21.850 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-03-26T08:08:21.850 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-26T08:08:21.850 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-26T08:08:21.850 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-26T08:08:21.850 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-26T08:08:21.850 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-26T08:08:21.850 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-03-26T08:08:21.850 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-03-26T08:08:21.850 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-26T08:08:21.851 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-26T08:08:21.851 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-26T08:08:21.851 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-26T08:08:21.851 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-26T08:08:21.851 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-26T08:08:21.851 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-26T08:08:21.851 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-03-26T08:08:21.851 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-26T08:08:21.851 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-26T08:08:21.851 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-03-26T08:08:21.851 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-26T08:08:21.852 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-26T08:08:21.852 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-26T08:08:21.852 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-26T08:08:21.852 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-03-26T08:08:21.852 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-03-26T08:08:21.852 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-26T08:08:21.852 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-26T08:08:21.852 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-26T08:08:21.852 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-26T08:08:21.852 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-03-26T08:08:21.852 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-26T08:08:21.852 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-03-26T08:08:21.853 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-26T08:08:21.853 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-26T08:08:21.853 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-26T08:08:21.853 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-26T08:08:21.853 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-26T08:08:21.853 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-26T08:08:21.853 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-26T08:08:21.853 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-03-26T08:08:21.853 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-26T08:08:21.853 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-26T08:08:21.853 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-03-26T08:08:21.853 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-26T08:08:21.853 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-26T08:08:21.854 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-26T08:08:21.854 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-03-26T08:08:21.854 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-26T08:08:21.854 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-03-26T08:08:21.854 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-26T08:08:21.854 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-26T08:08:21.854 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-26T08:08:21.854 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-26T08:08:21.854 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-26T08:08:21.854 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-26T08:08:21.854 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-26T08:08:21.854 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-03-26T08:08:21.854 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-26T08:08:21.855 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-03-26T08:08:21.855 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-03-26T08:08:21.855 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-26T08:08:21.855 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-03-26T08:08:21.855 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-03-26T08:08:21.855 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-26T08:08:21.855 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-26T08:08:21.855 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-26T08:08:21.855 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-03-26T08:08:21.855 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-26T08:08:21.855 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-26T08:08:21.855 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-26T08:08:21.855 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-03-26T08:08:21.855 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-03-26T08:08:21.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-26T08:08:21.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-26T08:08:21.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-26T08:08:21.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-26T08:08:21.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-03-26T08:08:21.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-26T08:08:21.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-26T08:08:21.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-26T08:08:21.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-26T08:08:21.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-03-26T08:08:21.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-26T08:08:21.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-03-26T08:08:21.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-26T08:08:21.856 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-03-26T08:08:21.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-26T08:08:21.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-03-26T08:08:21.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-26T08:08:21.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-03-26T08:08:21.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-26T08:08:21.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-26T08:08:21.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-26T08:08:21.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-03-26T08:08:21.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-03-26T08:08:21.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-26T08:08:21.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-26T08:08:21.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-26T08:08:21.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-26T08:08:21.857 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-26T08:08:21.858 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-03-26T08:08:21.858 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-26T08:08:21.858 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-26T08:08:21.858 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-03-26T08:08:21.858 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-26T08:08:21.858 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-26T08:08:21.858 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-26T08:08:21.858 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-26T08:08:21.858 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-26T08:08:21.858 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-03-26T08:08:21.858 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-26T08:08:21.858 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-26T08:08:21.858 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-03-26T08:08:21.858 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-03-26T08:08:21.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-26T08:08:21.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-03-26T08:08:21.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-26T08:08:21.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-26T08:08:21.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-26T08:08:21.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-26T08:08:21.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-26T08:08:21.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-26T08:08:21.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-26T08:08:21.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-26T08:08:21.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-26T08:08:21.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-26T08:08:21.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-03-26T08:08:21.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-03-26T08:08:21.859 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-26T08:08:21.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-03-26T08:08:21.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-03-26T08:08:21.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-26T08:08:21.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-26T08:08:21.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-26T08:08:21.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-26T08:08:21.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-26T08:08:21.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-26T08:08:21.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-03-26T08:08:21.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-26T08:08:21.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-26T08:08:21.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-26T08:08:21.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-03-26T08:08:21.860 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-03-26T08:08:21.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-26T08:08:21.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-26T08:08:21.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-26T08:08:21.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-03-26T08:08:21.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-26T08:08:21.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-03-26T08:08:21.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-26T08:08:21.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-03-26T08:08:21.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-26T08:08:21.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-03-26T08:08:21.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-26T08:08:21.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-03-26T08:08:21.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-26T08:08:21.861 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-03-26T08:08:21.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-26T08:08:21.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-03-26T08:08:21.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-26T08:08:21.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-03-26T08:08:21.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-26T08:08:21.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-26T08:08:21.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-26T08:08:21.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-26T08:08:21.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-26T08:08:21.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-26T08:08:21.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-03-26T08:08:21.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-26T08:08:21.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-26T08:08:21.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-26T08:08:21.862 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-03-26T08:08:21.863 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-26T08:08:21.863 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-26T08:08:21.863 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-26T08:08:21.863 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-26T08:08:21.863 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-26T08:08:21.863 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-26T08:08:21.863 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-03-26T08:08:21.863 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-26T08:08:21.863 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-03-26T08:08:21.863 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-26T08:08:21.863 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-26T08:08:21.863 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-03-26T08:08:21.863 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-26T08:08:21.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-26T08:08:21.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-03-26T08:08:21.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-26T08:08:21.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-03-26T08:08:21.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-26T08:08:21.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-03-26T08:08:21.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-26T08:08:21.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-26T08:08:21.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-03-26T08:08:21.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-26T08:08:21.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-26T08:08:21.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-26T08:08:21.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-26T08:08:21.864 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-26T08:08:21.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-03-26T08:08:21.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-26T08:08:21.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-03-26T08:08:21.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-26T08:08:21.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-26T08:08:21.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-26T08:08:21.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-26T08:08:21.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-03-26T08:08:21.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-26T08:08:21.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-03-26T08:08:21.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-26T08:08:21.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-03-26T08:08:21.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-26T08:08:21.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-03-26T08:08:21.865 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-26T08:08:21.866 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-26T08:08:21.866 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-03-26T08:08:21.866 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-26T08:08:21.866 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-26T08:08:21.866 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-26T08:08:21.866 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-26T08:08:21.866 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-03-26T08:08:21.866 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-26T08:08:21.866 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-26T08:08:21.866 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-26T08:08:21.866 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-03-26T08:08:21.866 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-26T08:08:21.866 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-26T08:08:21.866 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-26T08:08:21.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-26T08:08:21.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-03-26T08:08:21.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-26T08:08:21.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-03-26T08:08:21.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-26T08:08:21.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-03-26T08:08:21.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-26T08:08:21.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-26T08:08:21.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-03-26T08:08:21.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-26T08:08:21.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-26T08:08:21.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-03-26T08:08:21.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-26T08:08:21.867 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-26T08:08:21.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-03-26T08:08:21.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-26T08:08:21.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-03-26T08:08:21.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-26T08:08:21.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-26T08:08:21.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-03-26T08:08:21.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-26T08:08:21.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-26T08:08:21.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-26T08:08:21.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-03-26T08:08:21.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-26T08:08:21.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-26T08:08:21.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-26T08:08:21.868 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-26T08:08:21.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-26T08:08:21.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-03-26T08:08:21.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-26T08:08:21.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-26T08:08:21.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-03-26T08:08:21.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-26T08:08:21.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-26T08:08:21.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-03-26T08:08:21.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-26T08:08:21.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-26T08:08:21.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-26T08:08:21.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-26T08:08:21.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-26T08:08:21.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-26T08:08:21.869 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-26T08:08:21.870 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-26T08:08:21.870 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-03-26T08:08:21.870 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-26T08:08:21.870 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-26T08:08:21.870 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-26T08:08:21.870 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-26T08:08:21.870 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-26T08:08:21.870 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-26T08:08:21.870 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-26T08:08:21.870 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-26T08:08:21.870 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-26T08:08:21.870 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-26T08:08:21.870 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-26T08:08:21.870 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-26T08:08:21.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-03-26T08:08:21.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-26T08:08:21.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-26T08:08:21.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-26T08:08:21.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-26T08:08:21.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-26T08:08:21.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-26T08:08:21.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-26T08:08:21.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-26T08:08:21.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-26T08:08:21.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-03-26T08:08:21.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-03-26T08:08:21.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-26T08:08:21.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-26T08:08:21.871 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-26T08:08:21.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-03-26T08:08:21.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-26T08:08:21.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-26T08:08:21.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-26T08:08:21.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-26T08:08:21.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-26T08:08:21.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-26T08:08:21.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-03-26T08:08:21.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-26T08:08:21.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-03-26T08:08:21.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-03-26T08:08:21.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-26T08:08:21.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-03-26T08:08:21.872 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-26T08:08:21.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-03-26T08:08:21.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-26T08:08:21.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-03-26T08:08:21.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-03-26T08:08:21.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-26T08:08:21.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-03-26T08:08:21.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-03-26T08:08:21.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-26T08:08:21.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-26T08:08:21.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-03-26T08:08:21.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-26T08:08:21.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-26T08:08:21.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-26T08:08:21.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-26T08:08:21.873 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-03-26T08:08:21.874 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-03-26T08:08:21.874 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-26T08:08:21.874 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-03-26T08:08:21.874 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-26T08:08:21.874 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-26T08:08:21.874 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-03-26T08:08:21.874 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-26T08:08:21.874 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-26T08:08:21.874 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-26T08:08:21.874 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-03-26T08:08:21.874 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-03-26T08:08:21.874 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-26T08:08:21.874 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-03-26T08:08:21.874 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-03-26T08:08:21.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-26T08:08:21.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-26T08:08:21.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-26T08:08:21.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-26T08:08:21.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-26T08:08:21.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-26T08:08:21.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-26T08:08:21.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-26T08:08:21.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-26T08:08:21.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-26T08:08:21.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-26T08:08:21.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-26T08:08:21.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-03-26T08:08:21.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-26T08:08:21.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-26T08:08:21.875 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-26T08:08:21.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-26T08:08:21.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-26T08:08:21.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-26T08:08:21.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-26T08:08:21.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-26T08:08:21.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-26T08:08:21.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-26T08:08:21.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-26T08:08:21.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-26T08:08:21.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-03-26T08:08:21.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-26T08:08:21.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-26T08:08:21.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-03-26T08:08:21.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-03-26T08:08:21.876 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-26T08:08:21.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-26T08:08:21.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-26T08:08:21.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-26T08:08:21.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-03-26T08:08:21.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-26T08:08:21.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-26T08:08:21.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-03-26T08:08:21.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-26T08:08:21.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-26T08:08:21.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-26T08:08:21.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-26T08:08:21.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-26T08:08:21.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-26T08:08:21.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-26T08:08:21.877 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-26T08:08:21.878 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-26T08:08:21.878 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-26T08:08:21.878 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-26T08:08:21.878 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-26T08:08:21.878 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-26T08:08:21.878 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-26T08:08:21.878 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-26T08:08:21.878 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-26T08:08:21.878 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-26T08:08:21.878 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-26T08:08:21.878 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-26T08:08:21.878 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-26T08:08:21.878 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-26T08:08:21.878 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-26T08:08:21.879 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-26T08:08:21.879 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-26T08:08:21.879 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-26T08:08:21.879 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-26T08:08:21.879 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-26T08:08:21.879 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-26T08:08:21.879 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-26T08:08:21.879 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-26T08:08:21.879 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-26T08:08:21.879 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-03-26T08:08:21.879 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-26T08:08:21.879 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-26T08:08:21.879 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-26T08:08:21.879 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-26T08:08:21.879 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-26T08:08:21.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-03-26T08:08:21.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-26T08:08:21.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-26T08:08:21.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-03-26T08:08:21.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-26T08:08:21.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-26T08:08:21.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-26T08:08:21.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-03-26T08:08:21.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-26T08:08:21.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-26T08:08:21.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-03-26T08:08:21.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-26T08:08:21.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-03-26T08:08:21.880 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-26T08:08:21.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-03-26T08:08:21.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-26T08:08:21.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-03-26T08:08:21.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-26T08:08:21.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-26T08:08:21.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-03-26T08:08:21.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-26T08:08:21.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-03-26T08:08:21.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-26T08:08:21.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-26T08:08:21.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-26T08:08:21.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-26T08:08:21.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-26T08:08:21.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-26T08:08:21.881 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-03-26T08:08:21.882 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-26T08:08:21.882 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-26T08:08:21.882 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-26T08:08:21.882 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-26T08:08:21.882 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-26T08:08:21.882 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-03-26T08:08:21.882 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-26T08:08:21.882 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-26T08:08:21.882 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-03-26T08:08:21.882 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-26T08:08:21.882 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-26T08:08:21.882 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-03-26T08:08:21.882 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-26T08:08:21.883 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-26T08:08:21.883 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-26T08:08:21.883 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-26T08:08:21.883 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-26T08:08:21.883 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-26T08:08:21.883 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-26T08:08:21.883 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-03-26T08:08:21.883 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-26T08:08:21.883 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-26T08:08:21.883 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-26T08:08:21.883 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-03-26T08:08:21.883 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-26T08:08:21.883 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-26T08:08:21.883 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-26T08:08:21.883 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-03-26T08:08:21.884 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-26T08:08:21.884 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-03-26T08:08:21.884 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-26T08:08:21.884 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-03-26T08:08:21.884 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-26T08:08:21.884 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-26T08:08:21.884 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-26T08:08:21.884 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-26T08:08:21.884 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-03-26T08:08:21.884 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-26T08:08:21.884 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-03-26T08:08:21.884 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-26T08:08:21.884 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-26T08:08:21.884 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-03-26T08:08:21.885 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-26T08:08:21.885 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-26T08:08:21.885 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-26T08:08:21.885 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-03-26T08:08:21.885 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-26T08:08:21.885 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-26T08:08:21.885 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-03-26T08:08:21.885 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-26T08:08:21.885 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-26T08:08:21.885 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-26T08:08:21.885 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-26T08:08:21.885 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-26T08:08:21.885 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-26T08:08:21.885 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-26T08:08:21.886 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-26T08:08:21.886 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-26T08:08:21.886 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-26T08:08:21.886 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-26T08:08:21.886 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-03-26T08:08:21.886 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-26T08:08:21.886 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-26T08:08:21.886 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-26T08:08:21.886 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-26T08:08:21.886 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-03-26T08:08:21.886 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-26T08:08:21.886 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-03-26T08:08:21.887 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-03-26T08:08:21.887 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-26T08:08:21.887 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-03-26T08:08:21.887 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-26T08:08:21.887 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-26T08:08:21.887 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-26T08:08:21.887 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-26T08:08:21.887 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-26T08:08:21.887 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-26T08:08:21.887 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-03-26T08:08:21.887 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-26T08:08:21.888 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-03-26T08:08:21.888 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-03-26T08:08:21.888 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-03-26T08:08:21.888 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-26T08:08:21.888 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-26T08:08:21.888 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-26T08:08:21.888 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-26T08:08:21.888 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-26T08:08:21.888 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-26T08:08:21.888 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-26T08:08:21.888 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-26T08:08:21.888 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-26T08:08:21.888 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-26T08:08:21.889 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-26T08:08:21.889 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-26T08:08:21.889 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-26T08:08:21.889 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-26T08:08:21.889 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-26T08:08:21.889 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-26T08:08:21.889 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-26T08:08:21.889 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-26T08:08:21.889 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-03-26T08:08:21.889 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-03-26T08:08:21.889 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-26T08:08:21.889 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-26T08:08:21.889 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-26T08:08:21.889 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-26T08:08:21.889 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-03-26T08:08:21.890 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-03-26T08:08:21.890 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-26T08:08:21.890 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-26T08:08:21.890 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-03-26T08:08:21.890 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-26T08:08:21.890 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-03-26T08:08:21.890 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-03-26T08:08:21.890 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-26T08:08:21.890 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-03-26T08:08:21.890 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-26T08:08:21.890 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-03-26T08:08:21.890 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-26T08:08:21.890 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-26T08:08:21.890 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-26T08:08:21.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-26T08:08:21.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-26T08:08:21.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-03-26T08:08:21.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-03-26T08:08:21.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-26T08:08:21.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-26T08:08:21.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-26T08:08:21.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-26T08:08:21.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-03-26T08:08:21.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-26T08:08:21.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-03-26T08:08:21.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-26T08:08:21.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-26T08:08:21.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-03-26T08:08:21.891 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-26T08:08:21.892 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-26T08:08:21.892 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-26T08:08:21.892 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-26T08:08:21.892 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-26T08:08:21.892 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-26T08:08:21.892 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-26T08:08:21.892 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-26T08:08:21.892 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-26T08:08:21.892 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-26T08:08:21.892 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-03-26T08:08:21.892 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-26T08:08:21.892 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-26T08:08:21.892 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-26T08:08:21.892 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-26T08:08:21.893 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-26T08:08:21.893 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-26T08:08:21.893 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-26T08:08:21.893 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-03-26T08:08:21.893 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-26T08:08:21.893 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-03-26T08:08:21.893 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-26T08:08:21.893 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-26T08:08:21.893 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-26T08:08:21.893 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-26T08:08:21.893 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-26T08:08:21.893 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-26T08:08:21.894 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-26T08:08:21.894 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-26T08:08:21.894 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-26T08:08:21.894 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-26T08:08:21.894 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-26T08:08:21.894 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-26T08:08:21.894 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-26T08:08:21.894 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-26T08:08:21.894 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-26T08:08:21.894 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-26T08:08:21.894 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-26T08:08:21.894 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-26T08:08:21.895 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-26T08:08:21.895 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-26T08:08:21.895 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-26T08:08:21.895 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-26T08:08:21.895 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-26T08:08:21.895 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-26T08:08:21.895 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-26T08:08:21.895 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-26T08:08:21.895 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-26T08:08:21.895 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-26T08:08:21.895 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-26T08:08:21.896 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-26T08:08:21.896 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-26T08:08:21.896 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-26T08:08:21.896 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-26T08:08:21.896 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-26T08:08:21.896 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-26T08:08:21.896 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-26T08:08:21.896 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-26T08:08:21.896 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-26T08:08:21.896 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-26T08:08:21.896 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-26T08:08:21.897 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-26T08:08:21.897 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-26T08:08:21.897 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-26T08:08:21.897 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-26T08:08:21.897 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-26T08:08:21.897 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-03-26T08:08:21.897 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-26T08:08:21.897 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-26T08:08:21.897 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-26T08:08:21.897 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-26T08:08:21.897 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-26T08:08:21.898 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-26T08:08:21.898 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-26T08:08:21.898 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-26T08:08:21.898 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-26T08:08:21.898 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-26T08:08:21.898 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-26T08:08:21.898 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-26T08:08:21.898 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-26T08:08:21.898 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-26T08:08:21.898 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-26T08:08:21.899 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-03-26T08:08:21.899 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-26T08:08:21.899 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-26T08:08:21.899 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-26T08:08:21.899 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-03-26T08:08:21.899 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-03-26T08:08:21.899 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-26T08:08:21.899 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-26T08:08:21.899 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-26T08:08:21.899 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-26T08:08:21.899 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-03-26T08:08:21.899 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-26T08:08:21.900 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-26T08:08:21.900 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-26T08:08:21.900 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-26T08:08:21.900 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-03-26T08:08:21.900 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-03-26T08:08:21.900 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-26T08:08:21.900 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-26T08:08:21.900 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-26T08:08:21.900 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-03-26T08:08:21.900 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-03-26T08:08:21.900 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-03-26T08:08:21.900 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-03-26T08:08:21.901 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-03-26T08:08:21.901 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-26T08:08:21.901 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-26T08:08:21.901 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-03-26T08:08:21.901 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-03-26T08:08:21.901 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-03-26T08:08:21.901 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-26T08:08:21.901 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-26T08:08:21.901 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-03-26T08:08:21.901 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-26T08:08:21.901 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-26T08:08:21.901 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-26T08:08:21.902 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-26T08:08:21.902 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-26T08:08:21.902 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-26T08:08:21.902 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-26T08:08:21.902 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-26T08:08:21.902 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-26T08:08:21.902 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-26T08:08:21.902 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-26T08:08:21.902 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-26T08:08:21.902 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-03-26T08:08:21.902 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-26T08:08:21.902 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-26T08:08:21.902 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-26T08:08:21.903 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-26T08:08:21.903 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-26T08:08:21.903 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-26T08:08:21.903 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-26T08:08:21.903 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-26T08:08:21.903 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-26T08:08:21.903 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-26T08:08:21.903 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-26T08:08:21.903 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-26T08:08:21.903 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-26T08:08:21.903 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-26T08:08:21.903 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-26T08:08:21.903 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-26T08:08:21.903 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-26T08:08:21.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-26T08:08:21.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-26T08:08:21.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-26T08:08:21.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-26T08:08:21.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-26T08:08:21.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-26T08:08:21.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-26T08:08:21.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-26T08:08:21.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-26T08:08:21.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-03-26T08:08:21.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-26T08:08:21.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-26T08:08:21.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-26T08:08:21.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-26T08:08:21.904 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-03-26T08:08:21.905 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-26T08:08:21.905 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-26T08:08:21.905 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-03-26T08:08:21.905 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-26T08:08:21.905 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-03-26T08:08:21.905 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-03-26T08:08:21.905 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-26T08:08:21.905 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-03-26T08:08:21.905 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-26T08:08:21.905 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-03-26T08:08:21.905 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-26T08:08:21.905 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-26T08:08:21.905 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-03-26T08:08:21.905 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-26T08:08:21.906 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-03-26T08:08:21.906 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-26T08:08:21.906 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-03-26T08:08:21.906 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-26T08:08:21.906 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-03-26T08:08:21.906 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-26T08:08:21.906 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-03-26T08:08:21.906 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-26T08:08:21.906 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-03-26T08:08:21.906 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-26T08:08:21.906 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-03-26T08:08:21.906 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-26T08:08:21.906 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-03-26T08:08:21.906 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-26T08:08:21.907 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-03-26T08:08:21.907 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-03-26T08:08:21.907 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-26T08:08:21.907 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-03-26T08:08:21.907 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-03-26T08:08:21.907 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-26T08:08:21.907 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-03-26T08:08:21.907 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-03-26T08:08:21.907 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-26T08:08:21.907 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-03-26T08:08:21.907 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-26T08:08:21.908 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-03-26T08:08:21.908 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-26T08:08:21.908 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-03-26T08:08:21.908 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-26T08:08:21.908 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-03-26T08:08:21.908 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-26T08:08:21.908 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-03-26T08:08:21.908 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-26T08:08:21.908 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-03-26T08:08:21.908 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-26T08:08:21.908 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-03-26T08:08:21.909 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-26T08:08:21.909 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-03-26T08:08:21.909 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-03-26T08:08:21.909 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-26T08:08:21.909 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-03-26T08:08:21.909 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-26T08:08:21.909 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-03-26T08:08:21.909 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-26T08:08:21.909 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-26T08:08:21.909 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-26T08:08:21.909 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-26T08:08:21.909 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-03-26T08:08:21.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-26T08:08:21.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-26T08:08:21.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-03-26T08:08:21.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-26T08:08:21.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-03-26T08:08:21.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-26T08:08:21.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-03-26T08:08:21.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-26T08:08:21.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-26T08:08:21.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-26T08:08:21.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-26T08:08:21.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-26T08:08:21.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-26T08:08:21.910 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-26T08:08:21.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-26T08:08:21.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-26T08:08:21.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-26T08:08:21.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-26T08:08:21.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-26T08:08:21.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-26T08:08:21.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-26T08:08:21.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-26T08:08:21.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-26T08:08:21.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-26T08:08:21.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-26T08:08:21.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-26T08:08:21.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-26T08:08:21.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-26T08:08:21.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-26T08:08:21.911 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-26T08:08:21.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-26T08:08:21.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-26T08:08:21.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-26T08:08:21.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-26T08:08:21.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-26T08:08:21.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-26T08:08:21.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-26T08:08:21.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-26T08:08:21.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-26T08:08:21.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-26T08:08:21.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-26T08:08:21.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-26T08:08:21.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-26T08:08:21.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-26T08:08:21.912 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-26T08:08:21.913 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-26T08:08:21.913 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-26T08:08:21.913 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-26T08:08:21.913 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-26T08:08:21.913 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-26T08:08:21.913 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-26T08:08:21.913 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-26T08:08:21.913 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-26T08:08:21.913 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-26T08:08:21.913 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-26T08:08:21.913 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-26T08:08:21.913 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-26T08:08:21.914 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-03-26T08:08:21.914 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-03-26T08:08:21.914 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-26T08:08:21.914 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-03-26T08:08:21.914 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-26T08:08:21.914 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-03-26T08:08:21.914 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-26T08:08:21.914 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-26T08:08:21.914 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-26T08:08:21.914 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-26T08:08:21.914 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-26T08:08:21.914 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-03-26T08:08:21.915 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-26T08:08:21.915 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-03-26T08:08:21.915 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-26T08:08:21.915 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-03-26T08:08:21.915 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-26T08:08:21.915 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-03-26T08:08:21.915 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-26T08:08:21.915 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-03-26T08:08:21.915 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-03-26T08:08:21.915 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-26T08:08:21.916 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-03-26T08:08:21.916 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-26T08:08:21.916 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-26T08:08:21.916 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-03-26T08:08:21.916 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-26T08:08:21.916 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-03-26T08:08:21.916 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-26T08:08:21.916 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-03-26T08:08:21.916 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-26T08:08:21.916 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-03-26T08:08:21.916 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-26T08:08:21.916 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-03-26T08:08:21.917 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-26T08:08:21.917 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-03-26T08:08:21.917 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-03-26T08:08:21.917 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-03-26T08:08:21.917 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-26T08:08:21.917 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-03-26T08:08:21.917 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-26T08:08:21.917 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-03-26T08:08:21.917 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-03-26T08:08:21.918 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-26T08:08:21.918 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-26T08:08:21.918 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-03-26T08:08:21.918 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-26T08:08:21.918 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-26T08:08:21.918 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-26T08:08:21.918 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-26T08:08:21.918 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-03-26T08:08:21.918 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-26T08:08:21.918 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-26T08:08:21.918 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-03-26T08:08:21.919 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-26T08:08:21.919 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-03-26T08:08:21.919 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-26T08:08:21.919 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-03-26T08:08:21.919 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-26T08:08:21.919 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-26T08:08:21.919 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-26T08:08:21.919 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-26T08:08:21.919 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-03-26T08:08:21.919 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-03-26T08:08:21.920 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-03-26T08:08:21.920 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-26T08:08:21.920 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-03-26T08:08:21.920 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-03-26T08:08:21.920 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-26T08:08:21.920 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-26T08:08:21.920 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-26T08:08:21.920 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-26T08:08:21.920 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-26T08:08:21.920 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-26T08:08:21.920 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-26T08:08:21.921 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-26T08:08:21.921 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-03-26T08:08:21.921 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-26T08:08:21.921 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-26T08:08:21.921 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-26T08:08:21.921 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-26T08:08:21.921 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-26T08:08:21.921 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-03-26T08:08:21.921 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-26T08:08:21.921 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-03-26T08:08:21.922 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-26T08:08:21.922 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-26T08:08:21.922 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-26T08:08:21.922 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-26T08:08:21.922 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-03-26T08:08:21.922 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-26T08:08:21.922 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-03-26T08:08:21.922 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-26T08:08:21.922 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-26T08:08:21.923 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-03-26T08:08:21.923 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-26T08:08:21.923 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-26T08:08:21.923 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-26T08:08:21.923 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-26T08:08:21.923 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-26T08:08:21.923 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-26T08:08:21.923 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-26T08:08:21.924 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-26T08:08:21.924 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-26T08:08:21.924 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-26T08:08:21.924 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-26T08:08:21.924 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-26T08:08:21.924 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-26T08:08:21.924 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-26T08:08:21.924 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-26T08:08:21.925 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-26T08:08:21.925 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-26T08:08:21.925 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-26T08:08:21.925 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-26T08:08:21.925 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-26T08:08:21.925 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-26T08:08:21.925 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-26T08:08:21.925 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-26T08:08:21.925 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-26T08:08:21.926 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-03-26T08:08:21.926 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-26T08:08:21.926 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-03-26T08:08:21.926 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-03-26T08:08:21.926 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-26T08:08:21.926 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-03-26T08:08:21.926 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-26T08:08:21.926 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-26T08:08:21.926 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-26T08:08:21.927 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-26T08:08:21.927 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-03-26T08:08:21.927 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-03-26T08:08:21.927 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-03-26T08:08:21.927 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-26T08:08:21.927 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-26T08:08:21.927 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-26T08:08:21.927 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-26T08:08:21.927 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-26T08:08:21.927 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-03-26T08:08:21.928 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-03-26T08:08:21.928 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-26T08:08:21.928 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-26T08:08:21.928 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-26T08:08:21.928 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-03-26T08:08:21.928 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-26T08:08:21.928 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-26T08:08:21.928 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-26T08:08:21.928 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-26T08:08:21.928 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-26T08:08:21.929 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-26T08:08:21.929 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-26T08:08:21.929 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-26T08:08:21.929 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-26T08:08:21.929 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-26T08:08:21.929 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-03-26T08:08:21.929 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-26T08:08:21.929 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-26T08:08:21.929 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-26T08:08:21.929 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-26T08:08:21.929 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-26T08:08:21.930 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-26T08:08:21.930 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-26T08:08:21.930 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-26T08:08:21.930 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-26T08:08:21.930 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-26T08:08:21.930 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-26T08:08:21.930 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-26T08:08:21.930 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-26T08:08:21.930 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-26T08:08:21.931 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-26T08:08:21.931 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-26T08:08:21.931 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-26T08:08:21.931 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-26T08:08:21.931 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-26T08:08:21.931 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-26T08:08:21.931 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-26T08:08:21.931 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-26T08:08:21.931 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-26T08:08:21.931 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-26T08:08:21.931 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-26T08:08:21.932 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-26T08:08:21.932 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-26T08:08:21.932 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-26T08:08:21.932 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-26T08:08:21.932 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-26T08:08:21.932 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-26T08:08:21.932 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-26T08:08:21.932 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-26T08:08:21.932 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-26T08:08:21.932 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-26T08:08:21.932 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-26T08:08:21.933 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-26T08:08:21.933 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-26T08:08:21.933 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-03-26T08:08:21.933 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-26T08:08:21.933 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-26T08:08:21.933 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-26T08:08:21.933 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-26T08:08:21.933 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-26T08:08:21.933 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-26T08:08:21.933 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-26T08:08:21.934 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-03-26T08:08:21.934 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-26T08:08:21.934 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-26T08:08:21.934 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-26T08:08:21.934 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-26T08:08:21.934 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-26T08:08:21.934 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-26T08:08:21.934 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-26T08:08:21.934 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-26T08:08:21.934 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-26T08:08:21.934 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-26T08:08:21.934 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-26T08:08:21.935 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-26T08:08:21.935 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-26T08:08:21.935 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-26T08:08:21.935 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-26T08:08:21.935 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-26T08:08:21.935 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-26T08:08:21.935 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-26T08:08:21.935 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-26T08:08:21.935 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-26T08:08:21.935 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-26T08:08:21.936 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-26T08:08:21.936 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-26T08:08:21.936 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-26T08:08:21.936 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-26T08:08:21.936 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-26T08:08:21.936 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-26T08:08:21.936 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-03-26T08:08:21.936 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-26T08:08:21.936 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-03-26T08:08:21.936 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-26T08:08:21.936 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-26T08:08:21.937 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-26T08:08:21.937 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-26T08:08:21.937 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-26T08:08:21.937 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-26T08:08:21.937 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-26T08:08:21.937 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-03-26T08:08:21.937 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-26T08:08:21.937 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-03-26T08:08:21.937 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-26T08:08:21.937 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-03-26T08:08:21.937 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-26T08:08:21.937 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-03-26T08:08:21.938 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-03-26T08:08:21.938 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-26T08:08:21.938 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-26T08:08:21.938 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-26T08:08:21.938 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-26T08:08:21.938 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-26T08:08:21.938 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-26T08:08:21.938 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-26T08:08:21.938 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-26T08:08:21.938 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-26T08:08:21.939 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-26T08:08:21.939 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-26T08:08:21.939 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-26T08:08:21.939 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-26T08:08:21.939 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-26T08:08:21.939 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-26T08:08:21.939 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-26T08:08:21.939 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-03-26T08:08:21.939 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-26T08:08:21.939 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-26T08:08:21.940 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-26T08:08:21.940 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-26T08:08:21.940 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-26T08:08:21.940 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-26T08:08:21.940 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-26T08:08:21.940 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-26T08:08:21.940 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-26T08:08:21.940 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-26T08:08:21.940 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-26T08:08:21.941 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-26T08:08:21.941 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-26T08:08:21.941 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-26T08:08:21.941 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-26T08:08:21.941 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-26T08:08:21.941 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-26T08:08:21.941 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-26T08:08:21.941 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-26T08:08:21.941 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-26T08:08:21.941 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-26T08:08:21.942 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-26T08:08:21.942 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-26T08:08:21.942 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-26T08:08:21.942 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-26T08:08:21.942 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-26T08:08:21.942 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-26T08:08:21.942 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-26T08:08:21.943 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-26T08:08:21.943 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-26T08:08:21.943 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-26T08:08:21.943 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-26T08:08:21.943 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-26T08:08:21.943 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-26T08:08:21.943 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-26T08:08:21.943 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-26T08:08:21.943 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-26T08:08:21.943 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-26T08:08:21.944 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-26T08:08:21.944 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-26T08:08:21.944 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-26T08:08:21.944 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-26T08:08:21.944 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-26T08:08:21.944 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-26T08:08:21.944 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-26T08:08:21.944 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-26T08:08:21.944 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-26T08:08:21.944 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-26T08:08:21.944 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-26T08:08:21.945 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-26T08:08:21.945 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-26T08:08:21.945 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-26T08:08:21.945 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-26T08:08:21.945 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-26T08:08:21.945 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-26T08:08:21.945 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-26T08:08:21.945 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-26T08:08:21.945 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-26T08:08:21.945 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-03-26T08:08:21.945 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-26T08:08:21.946 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-26T08:08:21.946 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-26T08:08:21.946 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-26T08:08:21.946 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-03-26T08:08:21.946 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-26T08:08:21.946 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-26T08:08:21.946 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-26T08:08:21.946 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-26T08:08:21.946 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-26T08:08:21.946 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-26T08:08:21.947 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-26T08:08:21.947 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-26T08:08:21.947 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-26T08:08:21.947 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-26T08:08:21.947 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-26T08:08:21.947 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-26T08:08:21.947 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-26T08:08:21.947 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-26T08:08:21.947 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-26T08:08:21.947 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-26T08:08:21.947 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-26T08:08:21.948 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-26T08:08:21.948 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-26T08:08:21.948 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-26T08:08:21.948 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-26T08:08:21.948 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-26T08:08:21.948 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-26T08:08:21.948 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-26T08:08:21.948 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-26T08:08:21.948 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-26T08:08:21.948 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-26T08:08:21.948 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-26T08:08:21.949 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-26T08:08:21.949 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-26T08:08:21.949 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-26T08:08:21.949 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-26T08:08:21.949 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-26T08:08:21.949 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-26T08:08:21.949 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-26T08:08:21.949 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-26T08:08:21.949 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-26T08:08:21.949 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-26T08:08:21.949 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-26T08:08:21.949 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-26T08:08:21.950 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-26T08:08:21.950 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-26T08:08:21.950 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-26T08:08:21.950 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-26T08:08:21.950 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-26T08:08:21.950 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-26T08:08:21.950 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-26T08:08:21.950 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-26T08:08:21.950 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-26T08:08:21.951 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-26T08:08:21.951 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-26T08:08:21.951 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-26T08:08:21.951 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-26T08:08:21.951 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-26T08:08:21.951 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-26T08:08:21.951 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-26T08:08:21.951 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-26T08:08:21.951 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-26T08:08:21.951 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-26T08:08:21.951 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-26T08:08:21.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-26T08:08:21.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-26T08:08:21.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-26T08:08:21.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-26T08:08:21.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-26T08:08:21.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-26T08:08:21.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-26T08:08:21.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-26T08:08:21.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-26T08:08:21.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-26T08:08:21.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-26T08:08:21.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-03-26T08:08:21.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-26T08:08:21.952 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-03-26T08:08:21.953 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-26T08:08:21.953 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-03-26T08:08:21.953 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-26T08:08:21.953 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-26T08:08:21.953 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-26T08:08:21.953 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-26T08:08:21.953 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-03-26T08:08:21.953 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-26T08:08:21.953 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-03-26T08:08:21.953 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-26T08:08:21.954 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-26T08:08:21.954 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-26T08:08:21.954 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-03-26T08:08:21.954 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-26T08:08:21.954 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-26T08:08:21.954 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-26T08:08:21.954 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-26T08:08:21.955 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-26T08:08:21.955 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-26T08:08:21.955 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-26T08:08:21.955 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-26T08:08:21.955 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-26T08:08:21.955 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-26T08:08:21.955 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-26T08:08:21.955 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-26T08:08:21.955 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-26T08:08:21.956 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-26T08:08:21.956 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-26T08:08:21.956 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-26T08:08:21.956 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-26T08:08:21.956 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-26T08:08:21.956 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-26T08:08:21.956 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-26T08:08:21.956 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-26T08:08:21.957 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-26T08:08:21.957 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-26T08:08:21.957 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-26T08:08:21.957 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-26T08:08:21.957 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-26T08:08:21.957 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-26T08:08:21.957 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-26T08:08:21.957 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-26T08:08:21.957 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-26T08:08:21.958 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-26T08:08:21.958 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-26T08:08:21.958 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-26T08:08:21.958 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-26T08:08:21.958 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-26T08:08:21.958 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-26T08:08:21.958 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-26T08:08:21.958 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-26T08:08:21.958 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-26T08:08:21.959 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-26T08:08:21.959 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-26T08:08:21.959 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-26T08:08:21.959 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-26T08:08:21.959 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-03-26T08:08:21.959 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-26T08:08:21.959 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-26T08:08:21.959 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-03-26T08:08:21.959 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-03-26T08:08:21.960 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-26T08:08:21.960 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-26T08:08:21.960 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-26T08:08:21.960 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-03-26T08:08:21.960 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-26T08:08:21.960 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-03-26T08:08:21.960 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-26T08:08:21.960 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-26T08:08:21.960 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-03-26T08:08:21.960 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-26T08:08:21.960 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-03-26T08:08:21.960 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-26T08:08:21.960 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-26T08:08:21.960 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-26T08:08:21.961 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-26T08:08:21.961 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-26T08:08:21.961 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-03-26T08:08:21.961 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-26T08:08:21.961 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-03-26T08:08:21.961 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-03-26T08:08:21.961 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-03-26T08:08:21.961 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-26T08:08:21.961 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-03-26T08:08:21.961 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-03-26T08:08:21.961 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-03-26T08:08:21.961 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-03-26T08:08:21.961 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-26T08:08:21.962 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-26T08:08:21.962 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-26T08:08:21.962 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-26T08:08:21.962 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-03-26T08:08:21.962 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-26T08:08:21.962 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-26T08:08:21.962 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-26T08:08:21.962 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-26T08:08:21.962 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-26T08:08:21.962 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-26T08:08:21.962 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-26T08:08:21.962 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-26T08:08:21.962 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-26T08:08:21.962 INFO:teuthology.orchestra.run.smithi052.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-26T08:08:21.967 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-26T08:08:21.967 DEBUG:teuthology.orchestra.run.smithi052:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-26T08:08:21.991 INFO:teuthology.orchestra.run.smithi052.stdout:postinstall scriptlet (using /bin/sh): 2024-03-26T08:08:21.991 INFO:teuthology.orchestra.run.smithi052.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-26T08:08:21.991 INFO:teuthology.orchestra.run.smithi052.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-26T08:08:21.991 INFO:teuthology.orchestra.run.smithi052.stdout:fi 2024-03-26T08:08:21.991 INFO:teuthology.orchestra.run.smithi052.stdout:for file in vmlinuz System.map config; do 2024-03-26T08:08:21.991 INFO:teuthology.orchestra.run.smithi052.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-03-26T08:08:21.991 INFO:teuthology.orchestra.run.smithi052.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-03-26T08:08:21.991 INFO:teuthology.orchestra.run.smithi052.stdout: fi 2024-03-26T08:08:21.992 INFO:teuthology.orchestra.run.smithi052.stdout:done 2024-03-26T08:08:21.992 INFO:teuthology.orchestra.run.smithi052.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-26T08:08:21.992 INFO:teuthology.orchestra.run.smithi052.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-26T08:08:21.992 INFO:teuthology.orchestra.run.smithi052.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-03-26T08:08:21.992 INFO:teuthology.orchestra.run.smithi052.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-26T08:08:21.992 INFO:teuthology.orchestra.run.smithi052.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-03-26T08:08:21.992 INFO:teuthology.orchestra.run.smithi052.stdout:fi 2024-03-26T08:08:21.992 INFO:teuthology.orchestra.run.smithi052.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-26T08:08:21.992 INFO:teuthology.orchestra.run.smithi052.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-26T08:08:21.992 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-03-26T08:08:21.992 INFO:teuthology.orchestra.run.smithi052.stdout:fi 2024-03-26T08:08:21.993 DEBUG:teuthology.orchestra.run.smithi052:> sudo rpm -qi grub2-tools 2024-03-26T08:08:22.072 INFO:teuthology.orchestra.run.smithi052.stdout:Name : grub2-tools 2024-03-26T08:08:22.072 INFO:teuthology.orchestra.run.smithi052.stdout:Epoch : 1 2024-03-26T08:08:22.072 INFO:teuthology.orchestra.run.smithi052.stdout:Version : 2.06 2024-03-26T08:08:22.072 INFO:teuthology.orchestra.run.smithi052.stdout:Release : 68.el9 2024-03-26T08:08:22.072 INFO:teuthology.orchestra.run.smithi052.stdout:Architecture: x86_64 2024-03-26T08:08:22.072 INFO:teuthology.orchestra.run.smithi052.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-26T08:08:22.073 INFO:teuthology.orchestra.run.smithi052.stdout:Group : Unspecified 2024-03-26T08:08:22.073 INFO:teuthology.orchestra.run.smithi052.stdout:Size : 8264254 2024-03-26T08:08:22.073 INFO:teuthology.orchestra.run.smithi052.stdout:License : GPLv3+ 2024-03-26T08:08:22.073 INFO:teuthology.orchestra.run.smithi052.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-26T08:08:22.073 INFO:teuthology.orchestra.run.smithi052.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-26T08:08:22.073 INFO:teuthology.orchestra.run.smithi052.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-26T08:08:22.073 INFO:teuthology.orchestra.run.smithi052.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-26T08:08:22.073 INFO:teuthology.orchestra.run.smithi052.stdout:Packager : builder@centos.org 2024-03-26T08:08:22.073 INFO:teuthology.orchestra.run.smithi052.stdout:Vendor : CentOS 2024-03-26T08:08:22.073 INFO:teuthology.orchestra.run.smithi052.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-26T08:08:22.073 INFO:teuthology.orchestra.run.smithi052.stdout:Summary : Support tools for GRUB. 2024-03-26T08:08:22.073 INFO:teuthology.orchestra.run.smithi052.stdout:Description : 2024-03-26T08:08:22.073 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-26T08:08:22.074 INFO:teuthology.orchestra.run.smithi052.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-26T08:08:22.074 INFO:teuthology.orchestra.run.smithi052.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-26T08:08:22.074 INFO:teuthology.orchestra.run.smithi052.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-26T08:08:22.074 INFO:teuthology.orchestra.run.smithi052.stdout:hardware devices. 2024-03-26T08:08:22.074 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-26T08:08:22.074 INFO:teuthology.orchestra.run.smithi052.stdout:This subpackage provides tools for support of all platforms. 2024-03-26T08:08:22.075 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-26T08:08:22.075 INFO:teuthology.task.kernel:Updating grub on smithi052 to boot 6.8.0-g1df0c7e8a235 2024-03-26T08:08:22.075 DEBUG:teuthology.orchestra.run.smithi052:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-26T08:08:22.708 INFO:teuthology.orchestra.run.smithi052.stderr:Generating grub configuration file ... 2024-03-26T08:08:24.249 INFO:teuthology.orchestra.run.smithi052.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-26T08:08:24.277 INFO:teuthology.orchestra.run.smithi052.stderr:done 2024-03-26T08:08:24.279 DEBUG:teuthology.orchestra.run.smithi052:> mktemp 2024-03-26T08:08:24.336 INFO:teuthology.orchestra.run.smithi052.stdout:/tmp/tmp.QkDz4ctNLk 2024-03-26T08:08:24.336 DEBUG:teuthology.orchestra.run.smithi052:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.QkDz4ctNLk 2024-03-26T08:08:24.407 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 0666 /tmp/tmp.QkDz4ctNLk 2024-03-26T08:08:24.556 DEBUG:teuthology.orchestra.remote:smithi052:/tmp/tmp.QkDz4ctNLk is 6KB 2024-03-26T08:08:24.605 DEBUG:teuthology.orchestra.run.smithi052:> rm -fr /tmp/tmp.QkDz4ctNLk 2024-03-26T08:08:24.620 DEBUG:teuthology.orchestra.run.smithi052:> sudo /bin/ls /boot/loader/entries || true 2024-03-26T08:08:24.687 INFO:teuthology.orchestra.run.smithi052.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-26T08:08:24.688 INFO:teuthology.orchestra.run.smithi052.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-26T08:08:24.688 INFO:teuthology.orchestra.run.smithi052.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-26T08:08:24.688 INFO:teuthology.orchestra.run.smithi052.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235.conf 2024-03-26T08:08:24.689 DEBUG:teuthology.orchestra.run.smithi052:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235 2024-03-26T08:08:24.841 DEBUG:teuthology.orchestra.run.smithi052:> sudo shutdown -r now 2024-03-26T08:08:39.555 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi043.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-26T08:08:39.555 DEBUG:teuthology.orchestra.run.smithi043:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-26T08:08:39.635 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-03-26T08:08:39.635 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-03-26T08:08:39.635 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-03-26T08:08:39.635 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-03-26T08:08:39.635 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-03-26T08:08:39.635 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-03-26T08:08:39.635 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-03-26T08:08:39.636 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-26T08:08:39.636 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-26T08:08:39.636 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-26T08:08:39.636 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-26T08:08:39.636 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-26T08:08:39.636 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-26T08:08:39.636 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-26T08:08:39.636 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-26T08:08:39.636 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-26T08:08:39.636 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-26T08:08:39.636 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-26T08:08:39.636 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-26T08:08:39.636 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-26T08:08:39.636 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-26T08:08:39.637 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-26T08:08:39.637 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-26T08:08:39.637 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-26T08:08:39.637 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-26T08:08:39.637 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-26T08:08:39.637 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-26T08:08:39.637 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-03-26T08:08:39.637 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-03-26T08:08:39.637 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-03-26T08:08:39.637 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-26T08:08:39.637 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-03-26T08:08:39.637 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-26T08:08:39.637 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-26T08:08:39.637 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-26T08:08:39.638 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-03-26T08:08:39.638 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-26T08:08:39.638 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-26T08:08:39.638 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-26T08:08:39.638 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-03-26T08:08:39.638 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-26T08:08:39.638 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-26T08:08:39.638 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-26T08:08:39.638 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-26T08:08:39.638 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-26T08:08:39.638 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-26T08:08:39.639 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-26T08:08:39.639 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-26T08:08:39.639 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-26T08:08:39.639 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-26T08:08:39.639 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-26T08:08:39.639 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-26T08:08:39.639 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-26T08:08:39.639 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-26T08:08:39.639 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-26T08:08:39.639 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-26T08:08:39.639 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-26T08:08:39.639 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-26T08:08:39.639 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-26T08:08:39.639 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-26T08:08:39.640 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-26T08:08:39.640 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-26T08:08:39.640 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-26T08:08:39.640 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-26T08:08:39.640 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-26T08:08:39.640 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-26T08:08:39.640 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-26T08:08:39.640 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-26T08:08:39.640 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-26T08:08:39.640 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-26T08:08:39.640 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-26T08:08:39.640 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-26T08:08:39.640 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-26T08:08:39.640 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-26T08:08:39.640 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-26T08:08:39.641 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-26T08:08:39.641 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-26T08:08:39.641 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-26T08:08:39.641 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-26T08:08:39.641 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-26T08:08:39.641 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-26T08:08:39.641 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-26T08:08:39.641 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-26T08:08:39.641 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-26T08:08:39.641 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-26T08:08:39.641 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-03-26T08:08:39.641 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-03-26T08:08:39.641 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-26T08:08:39.641 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-26T08:08:39.641 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-03-26T08:08:39.642 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-26T08:08:39.642 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-26T08:08:39.642 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-26T08:08:39.642 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-26T08:08:39.642 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-26T08:08:39.642 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-03-26T08:08:39.642 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-26T08:08:39.642 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-26T08:08:39.642 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-03-26T08:08:39.642 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-26T08:08:39.642 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-26T08:08:39.642 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-26T08:08:39.642 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-26T08:08:39.643 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-26T08:08:39.643 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-26T08:08:39.643 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-26T08:08:39.643 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-26T08:08:39.643 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-26T08:08:39.643 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-26T08:08:39.643 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-26T08:08:39.643 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-26T08:08:39.643 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-26T08:08:39.643 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-26T08:08:39.643 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-26T08:08:39.643 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-26T08:08:39.643 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-26T08:08:39.643 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-26T08:08:39.643 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-26T08:08:39.644 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-26T08:08:39.644 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-26T08:08:39.644 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-26T08:08:39.644 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-26T08:08:39.644 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-26T08:08:39.644 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-26T08:08:39.644 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-26T08:08:39.644 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-26T08:08:39.644 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-26T08:08:39.644 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-26T08:08:39.644 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-26T08:08:39.644 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-26T08:08:39.644 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-26T08:08:39.644 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-26T08:08:39.644 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-26T08:08:39.645 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-26T08:08:39.645 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-03-26T08:08:39.645 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-26T08:08:39.645 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-26T08:08:39.645 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-26T08:08:39.645 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-26T08:08:39.645 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-26T08:08:39.645 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-26T08:08:39.645 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-26T08:08:39.645 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-26T08:08:39.645 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-26T08:08:39.645 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-26T08:08:39.645 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-26T08:08:39.645 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-26T08:08:39.646 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-26T08:08:39.646 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-26T08:08:39.646 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-26T08:08:39.646 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-26T08:08:39.646 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-26T08:08:39.646 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-03-26T08:08:39.646 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-26T08:08:39.646 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-03-26T08:08:39.646 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-03-26T08:08:39.646 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-26T08:08:39.647 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-26T08:08:39.647 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-03-26T08:08:39.647 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-26T08:08:39.647 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-03-26T08:08:39.647 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-03-26T08:08:39.647 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-26T08:08:39.647 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-03-26T08:08:39.647 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-03-26T08:08:39.647 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-26T08:08:39.647 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-26T08:08:39.647 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-03-26T08:08:39.647 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-03-26T08:08:39.647 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-26T08:08:39.647 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-26T08:08:39.647 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-03-26T08:08:39.648 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-26T08:08:39.648 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-26T08:08:39.648 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-26T08:08:39.648 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-03-26T08:08:39.648 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-03-26T08:08:39.648 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-26T08:08:39.648 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-03-26T08:08:39.648 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-03-26T08:08:39.648 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-26T08:08:39.648 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-03-26T08:08:39.648 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-26T08:08:39.648 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-03-26T08:08:39.648 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-26T08:08:39.648 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-26T08:08:39.649 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-26T08:08:39.649 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-26T08:08:39.649 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-26T08:08:39.649 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-03-26T08:08:39.649 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-26T08:08:39.649 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-26T08:08:39.649 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-26T08:08:39.649 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-26T08:08:39.649 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-26T08:08:39.649 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-26T08:08:39.649 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-26T08:08:39.649 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-03-26T08:08:39.650 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-26T08:08:39.650 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-26T08:08:39.650 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-26T08:08:39.650 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-03-26T08:08:39.650 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-03-26T08:08:39.650 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-26T08:08:39.650 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-26T08:08:39.650 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-26T08:08:39.650 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-26T08:08:39.650 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-26T08:08:39.650 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-03-26T08:08:39.650 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-03-26T08:08:39.650 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-03-26T08:08:39.651 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-26T08:08:39.651 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-26T08:08:39.651 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-26T08:08:39.651 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-26T08:08:39.651 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-26T08:08:39.651 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-26T08:08:39.651 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-03-26T08:08:39.651 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-03-26T08:08:39.651 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-26T08:08:39.651 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-26T08:08:39.651 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-03-26T08:08:39.651 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-03-26T08:08:39.652 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-03-26T08:08:39.652 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-26T08:08:39.652 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-26T08:08:39.652 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-26T08:08:39.652 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-26T08:08:39.652 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-26T08:08:39.652 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-03-26T08:08:39.652 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-26T08:08:39.652 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-03-26T08:08:39.652 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-26T08:08:39.652 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-03-26T08:08:39.652 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-26T08:08:39.652 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-03-26T08:08:39.652 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-03-26T08:08:39.652 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-26T08:08:39.653 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-03-26T08:08:39.653 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-26T08:08:39.653 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-03-26T08:08:39.653 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-26T08:08:39.653 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-26T08:08:39.653 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-26T08:08:39.653 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-26T08:08:39.653 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-26T08:08:39.653 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-26T08:08:39.653 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-26T08:08:39.653 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-26T08:08:39.653 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-26T08:08:39.653 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-26T08:08:39.653 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-26T08:08:39.654 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-26T08:08:39.654 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-26T08:08:39.654 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-03-26T08:08:39.654 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-03-26T08:08:39.654 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-26T08:08:39.654 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-03-26T08:08:39.654 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-26T08:08:39.654 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-26T08:08:39.654 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-03-26T08:08:39.654 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-26T08:08:39.654 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-26T08:08:39.654 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-26T08:08:39.654 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-26T08:08:39.654 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-26T08:08:39.654 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-26T08:08:39.655 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-26T08:08:39.655 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-26T08:08:39.655 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-26T08:08:39.655 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-26T08:08:39.655 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-26T08:08:39.655 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-26T08:08:39.655 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-26T08:08:39.655 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-26T08:08:39.655 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-26T08:08:39.655 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-26T08:08:39.655 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-26T08:08:39.655 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-26T08:08:39.655 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-26T08:08:39.655 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-26T08:08:39.656 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-26T08:08:39.656 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-26T08:08:39.656 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-26T08:08:39.656 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-26T08:08:39.656 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-26T08:08:39.656 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-26T08:08:39.656 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-26T08:08:39.656 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-26T08:08:39.656 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-26T08:08:39.656 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-26T08:08:39.656 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-26T08:08:39.656 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-26T08:08:39.656 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-26T08:08:39.656 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-26T08:08:39.656 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-26T08:08:39.657 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-26T08:08:39.657 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-26T08:08:39.657 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-26T08:08:39.657 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-26T08:08:39.657 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-26T08:08:39.657 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-26T08:08:39.657 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-26T08:08:39.657 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-26T08:08:39.657 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-26T08:08:39.657 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-26T08:08:39.657 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-26T08:08:39.657 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-26T08:08:39.657 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-26T08:08:39.657 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-26T08:08:39.657 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-26T08:08:39.658 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-26T08:08:39.658 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-26T08:08:39.658 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-26T08:08:39.658 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-26T08:08:39.658 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-26T08:08:39.658 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-26T08:08:39.658 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-26T08:08:39.658 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-26T08:08:39.658 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-26T08:08:39.658 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-26T08:08:39.658 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-26T08:08:39.658 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-26T08:08:39.658 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-26T08:08:39.658 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-26T08:08:39.659 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-03-26T08:08:39.659 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-26T08:08:39.659 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-26T08:08:39.659 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-26T08:08:39.659 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-03-26T08:08:39.659 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-26T08:08:39.659 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-26T08:08:39.659 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-26T08:08:39.659 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-26T08:08:39.659 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-26T08:08:39.659 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-26T08:08:39.659 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-26T08:08:39.659 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-26T08:08:39.659 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-26T08:08:39.659 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-26T08:08:39.660 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-26T08:08:39.660 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-26T08:08:39.660 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-26T08:08:39.660 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-26T08:08:39.660 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-26T08:08:39.660 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-26T08:08:39.660 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-26T08:08:39.660 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-26T08:08:39.660 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-26T08:08:39.660 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-26T08:08:39.660 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-26T08:08:39.660 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-26T08:08:39.660 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-26T08:08:39.660 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-26T08:08:39.661 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-26T08:08:39.661 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-26T08:08:39.661 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-26T08:08:39.661 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-26T08:08:39.661 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-26T08:08:39.661 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-26T08:08:39.661 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-26T08:08:39.661 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-26T08:08:39.661 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-26T08:08:39.661 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-26T08:08:39.661 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-26T08:08:39.661 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-26T08:08:39.661 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-26T08:08:39.662 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-26T08:08:39.662 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-26T08:08:39.662 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-26T08:08:39.662 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-26T08:08:39.662 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-26T08:08:39.662 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-26T08:08:39.662 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-26T08:08:39.662 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-26T08:08:39.662 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-26T08:08:39.662 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-26T08:08:39.662 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-26T08:08:39.662 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-26T08:08:39.663 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-26T08:08:39.663 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-26T08:08:39.663 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-26T08:08:39.663 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-26T08:08:39.663 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-26T08:08:39.663 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-26T08:08:39.663 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-26T08:08:39.663 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-26T08:08:39.663 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-26T08:08:39.663 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-26T08:08:39.663 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-26T08:08:39.663 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-26T08:08:39.663 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-26T08:08:39.664 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-26T08:08:39.664 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-26T08:08:39.664 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-26T08:08:39.664 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-26T08:08:39.664 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-26T08:08:39.664 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-26T08:08:39.664 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-26T08:08:39.665 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-26T08:08:39.665 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-26T08:08:39.665 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-26T08:08:39.665 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-26T08:08:39.665 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-26T08:08:39.665 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-26T08:08:39.666 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-26T08:08:39.666 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-26T08:08:39.666 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-26T08:08:39.666 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-26T08:08:39.666 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-26T08:08:39.666 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-26T08:08:39.666 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-26T08:08:39.667 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-26T08:08:39.667 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-26T08:08:39.667 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-26T08:08:39.667 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-26T08:08:39.667 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-26T08:08:39.667 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-26T08:08:39.667 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-03-26T08:08:39.668 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-26T08:08:39.668 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-26T08:08:39.668 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-26T08:08:39.668 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-26T08:08:39.668 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-26T08:08:39.668 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-26T08:08:39.668 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-26T08:08:39.669 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-26T08:08:39.669 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-26T08:08:39.669 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-26T08:08:39.669 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-26T08:08:39.669 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-26T08:08:39.669 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-26T08:08:39.669 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-26T08:08:39.670 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-26T08:08:39.670 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-26T08:08:39.670 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-26T08:08:39.670 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-26T08:08:39.670 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-26T08:08:39.670 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-26T08:08:39.671 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-26T08:08:39.671 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-26T08:08:39.671 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-26T08:08:39.671 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-26T08:08:39.671 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-26T08:08:39.671 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-26T08:08:39.671 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-26T08:08:39.672 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-26T08:08:39.672 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-26T08:08:39.672 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-26T08:08:39.672 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-26T08:08:39.672 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-26T08:08:39.672 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-26T08:08:39.673 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-26T08:08:39.673 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-26T08:08:39.673 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-26T08:08:39.673 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-26T08:08:39.673 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-26T08:08:39.673 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-26T08:08:39.673 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-26T08:08:39.674 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-26T08:08:39.674 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-03-26T08:08:39.674 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-03-26T08:08:39.674 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-26T08:08:39.674 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-26T08:08:39.674 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-03-26T08:08:39.675 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-26T08:08:39.675 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-26T08:08:39.675 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-26T08:08:39.675 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-26T08:08:39.675 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-26T08:08:39.675 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-26T08:08:39.676 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-26T08:08:39.676 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-26T08:08:39.676 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-26T08:08:39.676 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-26T08:08:39.676 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-26T08:08:39.676 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-26T08:08:39.677 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-26T08:08:39.677 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-26T08:08:39.677 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-26T08:08:39.677 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-26T08:08:39.677 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-26T08:08:39.677 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-26T08:08:39.677 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-26T08:08:39.677 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-26T08:08:39.677 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-26T08:08:39.677 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-26T08:08:39.677 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-26T08:08:39.677 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-03-26T08:08:39.677 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-26T08:08:39.678 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-26T08:08:39.678 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-26T08:08:39.678 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-26T08:08:39.678 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-03-26T08:08:39.678 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-03-26T08:08:39.678 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-26T08:08:39.678 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-26T08:08:39.678 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-26T08:08:39.678 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-26T08:08:39.678 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-26T08:08:39.678 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-26T08:08:39.678 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-26T08:08:39.678 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-03-26T08:08:39.678 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-03-26T08:08:39.679 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-26T08:08:39.679 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-03-26T08:08:39.679 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-26T08:08:39.679 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-03-26T08:08:39.679 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-26T08:08:39.679 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-03-26T08:08:39.679 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-26T08:08:39.679 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-03-26T08:08:39.679 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-03-26T08:08:39.679 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-26T08:08:39.679 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-03-26T08:08:39.679 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-26T08:08:39.679 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-03-26T08:08:39.679 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-26T08:08:39.679 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-03-26T08:08:39.680 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-26T08:08:39.680 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-03-26T08:08:39.680 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-26T08:08:39.680 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-03-26T08:08:39.680 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-03-26T08:08:39.680 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-26T08:08:39.680 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-26T08:08:39.680 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-26T08:08:39.680 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-26T08:08:39.680 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-03-26T08:08:39.680 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-26T08:08:39.680 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-03-26T08:08:39.680 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-26T08:08:39.680 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-26T08:08:39.680 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-26T08:08:39.681 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-26T08:08:39.681 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-26T08:08:39.681 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-26T08:08:39.681 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-26T08:08:39.681 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-26T08:08:39.681 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-03-26T08:08:39.681 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-26T08:08:39.681 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-03-26T08:08:39.681 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-26T08:08:39.681 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-26T08:08:39.681 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-26T08:08:39.681 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-26T08:08:39.681 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-26T08:08:39.681 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-03-26T08:08:39.681 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-26T08:08:39.682 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-26T08:08:39.682 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-26T08:08:39.682 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-26T08:08:39.682 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-03-26T08:08:39.682 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-26T08:08:39.682 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-26T08:08:39.682 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-26T08:08:39.682 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-26T08:08:39.682 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-26T08:08:39.682 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-26T08:08:39.683 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-26T08:08:39.683 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-26T08:08:39.683 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-26T08:08:39.683 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-03-26T08:08:39.683 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-03-26T08:08:39.683 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-26T08:08:39.683 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-26T08:08:39.683 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-26T08:08:39.684 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-26T08:08:39.684 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-26T08:08:39.684 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-26T08:08:39.684 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-26T08:08:39.684 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-26T08:08:39.684 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-26T08:08:39.684 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-26T08:08:39.684 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-26T08:08:39.684 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-26T08:08:39.684 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-26T08:08:39.684 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-26T08:08:39.684 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-26T08:08:39.684 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-26T08:08:39.684 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-03-26T08:08:39.685 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-26T08:08:39.685 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-26T08:08:39.685 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-26T08:08:39.685 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-26T08:08:39.685 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-26T08:08:39.685 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-03-26T08:08:39.685 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-03-26T08:08:39.685 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-26T08:08:39.685 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-26T08:08:39.685 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-26T08:08:39.685 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-26T08:08:39.685 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-26T08:08:39.685 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-26T08:08:39.685 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-26T08:08:39.685 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-03-26T08:08:39.686 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-26T08:08:39.686 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-26T08:08:39.686 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-03-26T08:08:39.686 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-26T08:08:39.686 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-26T08:08:39.686 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-26T08:08:39.686 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-26T08:08:39.686 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-03-26T08:08:39.686 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-03-26T08:08:39.686 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-26T08:08:39.686 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-26T08:08:39.686 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-26T08:08:39.687 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-26T08:08:39.687 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-03-26T08:08:39.687 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-26T08:08:39.687 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-03-26T08:08:39.687 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-26T08:08:39.687 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-26T08:08:39.687 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-26T08:08:39.687 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-26T08:08:39.687 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-26T08:08:39.687 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-26T08:08:39.687 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-26T08:08:39.687 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-03-26T08:08:39.687 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-26T08:08:39.687 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-26T08:08:39.688 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-03-26T08:08:39.688 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-26T08:08:39.688 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-26T08:08:39.688 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-26T08:08:39.688 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-03-26T08:08:39.688 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-26T08:08:39.688 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-03-26T08:08:39.688 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-26T08:08:39.688 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-26T08:08:39.688 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-26T08:08:39.688 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-26T08:08:39.688 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-26T08:08:39.688 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-26T08:08:39.688 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-26T08:08:39.689 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-03-26T08:08:39.689 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-26T08:08:39.689 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-03-26T08:08:39.689 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-03-26T08:08:39.689 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-26T08:08:39.689 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-03-26T08:08:39.689 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-03-26T08:08:39.689 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-26T08:08:39.689 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-26T08:08:39.689 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-26T08:08:39.689 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-03-26T08:08:39.689 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-26T08:08:39.689 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-26T08:08:39.689 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-26T08:08:39.690 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-03-26T08:08:39.690 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-03-26T08:08:39.690 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-26T08:08:39.690 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-26T08:08:39.690 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-26T08:08:39.690 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-26T08:08:39.690 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-03-26T08:08:39.690 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-26T08:08:39.690 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-26T08:08:39.690 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-26T08:08:39.690 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-26T08:08:39.690 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-03-26T08:08:39.690 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-26T08:08:39.690 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-03-26T08:08:39.691 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-26T08:08:39.691 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-03-26T08:08:39.691 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-26T08:08:39.691 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-03-26T08:08:39.691 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-26T08:08:39.691 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-03-26T08:08:39.691 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-26T08:08:39.691 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-26T08:08:39.691 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-26T08:08:39.691 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-03-26T08:08:39.691 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-03-26T08:08:39.691 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-26T08:08:39.691 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-26T08:08:39.691 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-26T08:08:39.691 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-26T08:08:39.692 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-26T08:08:39.692 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-03-26T08:08:39.692 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-26T08:08:39.692 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-26T08:08:39.692 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-03-26T08:08:39.692 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-26T08:08:39.692 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-26T08:08:39.692 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-26T08:08:39.692 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-26T08:08:39.692 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-26T08:08:39.692 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-03-26T08:08:39.692 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-26T08:08:39.692 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-26T08:08:39.692 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-03-26T08:08:39.692 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-03-26T08:08:39.693 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-26T08:08:39.693 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-03-26T08:08:39.693 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-26T08:08:39.693 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-26T08:08:39.693 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-26T08:08:39.693 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-26T08:08:39.693 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-26T08:08:39.693 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-26T08:08:39.693 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-26T08:08:39.693 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-26T08:08:39.693 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-26T08:08:39.693 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-26T08:08:39.693 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-03-26T08:08:39.693 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-03-26T08:08:39.693 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-26T08:08:39.693 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-03-26T08:08:39.694 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-03-26T08:08:39.694 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-26T08:08:39.694 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-26T08:08:39.694 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-26T08:08:39.694 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-26T08:08:39.694 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-26T08:08:39.694 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-26T08:08:39.694 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-03-26T08:08:39.694 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-26T08:08:39.694 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-26T08:08:39.694 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-26T08:08:39.694 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-03-26T08:08:39.694 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-03-26T08:08:39.694 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-26T08:08:39.695 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-26T08:08:39.695 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-26T08:08:39.695 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-03-26T08:08:39.695 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-26T08:08:39.695 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-03-26T08:08:39.695 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-26T08:08:39.695 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-03-26T08:08:39.695 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-26T08:08:39.695 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-03-26T08:08:39.695 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-26T08:08:39.695 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-03-26T08:08:39.695 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-26T08:08:39.695 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-03-26T08:08:39.695 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-26T08:08:39.695 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-03-26T08:08:39.696 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-26T08:08:39.696 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-03-26T08:08:39.696 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-26T08:08:39.696 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-26T08:08:39.696 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-26T08:08:39.696 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-26T08:08:39.696 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-26T08:08:39.696 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-26T08:08:39.696 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-03-26T08:08:39.696 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-26T08:08:39.696 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-26T08:08:39.696 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-26T08:08:39.696 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-03-26T08:08:39.696 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-26T08:08:39.696 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-26T08:08:39.697 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-26T08:08:39.697 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-26T08:08:39.697 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-26T08:08:39.697 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-26T08:08:39.697 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-03-26T08:08:39.697 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-26T08:08:39.697 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-03-26T08:08:39.697 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-26T08:08:39.697 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-26T08:08:39.697 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-03-26T08:08:39.697 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-26T08:08:39.697 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-26T08:08:39.697 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-03-26T08:08:39.697 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-26T08:08:39.697 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-03-26T08:08:39.698 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-26T08:08:39.698 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-03-26T08:08:39.698 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-26T08:08:39.698 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-26T08:08:39.698 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-03-26T08:08:39.698 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-26T08:08:39.698 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-26T08:08:39.698 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-26T08:08:39.698 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-26T08:08:39.698 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-26T08:08:39.698 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-03-26T08:08:39.698 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-26T08:08:39.698 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-03-26T08:08:39.698 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-26T08:08:39.698 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-26T08:08:39.699 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-26T08:08:39.699 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-26T08:08:39.699 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-03-26T08:08:39.699 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-26T08:08:39.699 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-03-26T08:08:39.699 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-26T08:08:39.699 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-03-26T08:08:39.699 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-26T08:08:39.699 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-03-26T08:08:39.699 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-26T08:08:39.699 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-26T08:08:39.699 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-03-26T08:08:39.699 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-26T08:08:39.699 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-26T08:08:39.699 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-26T08:08:39.700 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-26T08:08:39.700 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-03-26T08:08:39.700 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-26T08:08:39.700 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-26T08:08:39.700 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-26T08:08:39.700 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-03-26T08:08:39.700 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-26T08:08:39.700 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-26T08:08:39.700 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-26T08:08:39.700 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-26T08:08:39.700 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-03-26T08:08:39.700 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-26T08:08:39.700 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-03-26T08:08:39.700 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-26T08:08:39.700 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-03-26T08:08:39.701 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-26T08:08:39.701 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-26T08:08:39.701 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-03-26T08:08:39.701 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-26T08:08:39.701 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-26T08:08:39.701 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-03-26T08:08:39.701 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-26T08:08:39.701 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-26T08:08:39.701 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-03-26T08:08:39.701 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-26T08:08:39.701 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-03-26T08:08:39.701 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-26T08:08:39.701 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-26T08:08:39.701 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-03-26T08:08:39.701 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-26T08:08:39.702 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-26T08:08:39.702 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-26T08:08:39.702 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-03-26T08:08:39.702 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-26T08:08:39.702 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-26T08:08:39.702 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-26T08:08:39.702 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-26T08:08:39.702 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-26T08:08:39.702 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-03-26T08:08:39.702 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-26T08:08:39.702 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-26T08:08:39.702 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-03-26T08:08:39.702 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-26T08:08:39.702 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-26T08:08:39.703 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-03-26T08:08:39.703 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-26T08:08:39.703 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-26T08:08:39.703 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-26T08:08:39.703 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-26T08:08:39.703 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-26T08:08:39.703 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-26T08:08:39.703 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-26T08:08:39.703 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-26T08:08:39.703 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-03-26T08:08:39.703 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-26T08:08:39.703 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-26T08:08:39.703 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-26T08:08:39.703 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-26T08:08:39.703 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-26T08:08:39.704 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-26T08:08:39.704 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-26T08:08:39.704 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-26T08:08:39.704 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-26T08:08:39.704 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-26T08:08:39.704 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-26T08:08:39.704 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-26T08:08:39.704 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-03-26T08:08:39.704 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-26T08:08:39.704 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-26T08:08:39.704 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-26T08:08:39.704 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-26T08:08:39.704 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-26T08:08:39.704 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-26T08:08:39.704 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-26T08:08:39.705 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-26T08:08:39.705 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-26T08:08:39.705 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-03-26T08:08:39.705 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-03-26T08:08:39.705 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-26T08:08:39.705 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-26T08:08:39.705 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-26T08:08:39.705 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-03-26T08:08:39.705 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-26T08:08:39.705 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-26T08:08:39.705 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-26T08:08:39.705 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-26T08:08:39.705 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-26T08:08:39.705 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-26T08:08:39.705 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-03-26T08:08:39.706 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-26T08:08:39.706 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-03-26T08:08:39.706 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-03-26T08:08:39.706 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-26T08:08:39.706 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-03-26T08:08:39.706 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-26T08:08:39.706 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-03-26T08:08:39.706 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-26T08:08:39.706 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-03-26T08:08:39.706 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-03-26T08:08:39.706 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-26T08:08:39.706 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-03-26T08:08:39.706 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-03-26T08:08:39.706 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-26T08:08:39.707 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-26T08:08:39.707 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-03-26T08:08:39.707 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-26T08:08:39.707 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-26T08:08:39.707 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-26T08:08:39.707 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-26T08:08:39.707 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-03-26T08:08:39.707 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-03-26T08:08:39.707 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-26T08:08:39.707 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-03-26T08:08:39.707 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-26T08:08:39.707 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-26T08:08:39.707 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-03-26T08:08:39.707 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-26T08:08:39.708 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-26T08:08:39.708 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-26T08:08:39.708 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-03-26T08:08:39.708 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-03-26T08:08:39.708 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-26T08:08:39.708 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-03-26T08:08:39.708 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-03-26T08:08:39.708 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-26T08:08:39.708 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-26T08:08:39.708 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-26T08:08:39.708 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-26T08:08:39.708 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-26T08:08:39.708 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-26T08:08:39.708 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-26T08:08:39.709 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-26T08:08:39.709 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-26T08:08:39.709 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-26T08:08:39.709 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-26T08:08:39.709 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-26T08:08:39.709 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-03-26T08:08:39.709 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-26T08:08:39.709 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-26T08:08:39.709 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-26T08:08:39.709 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-26T08:08:39.709 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-26T08:08:39.709 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-26T08:08:39.709 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-26T08:08:39.709 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-26T08:08:39.710 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-26T08:08:39.710 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-26T08:08:39.710 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-26T08:08:39.710 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-26T08:08:39.710 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-03-26T08:08:39.710 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-26T08:08:39.710 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-26T08:08:39.710 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-03-26T08:08:39.710 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-03-26T08:08:39.710 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-26T08:08:39.710 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-26T08:08:39.710 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-26T08:08:39.710 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-26T08:08:39.710 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-03-26T08:08:39.711 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-26T08:08:39.711 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-26T08:08:39.711 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-03-26T08:08:39.711 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-26T08:08:39.711 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-26T08:08:39.711 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-26T08:08:39.711 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-26T08:08:39.711 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-26T08:08:39.711 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-26T08:08:39.711 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-26T08:08:39.711 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-26T08:08:39.711 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-26T08:08:39.711 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-26T08:08:39.711 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-26T08:08:39.712 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-26T08:08:39.712 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-26T08:08:39.712 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-26T08:08:39.712 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-26T08:08:39.712 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-26T08:08:39.712 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-26T08:08:39.712 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-26T08:08:39.712 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-26T08:08:39.712 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-26T08:08:39.712 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-26T08:08:39.712 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-26T08:08:39.712 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-26T08:08:39.712 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-26T08:08:39.712 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-26T08:08:39.713 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-26T08:08:39.713 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-26T08:08:39.713 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-26T08:08:39.713 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-26T08:08:39.713 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-26T08:08:39.713 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-26T08:08:39.713 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-03-26T08:08:39.713 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-26T08:08:39.713 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-26T08:08:39.713 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-26T08:08:39.713 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-26T08:08:39.713 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-26T08:08:39.713 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-03-26T08:08:39.713 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-26T08:08:39.714 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-26T08:08:39.714 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-03-26T08:08:39.714 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-26T08:08:39.714 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-26T08:08:39.714 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-26T08:08:39.714 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-03-26T08:08:39.714 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-26T08:08:39.714 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-26T08:08:39.714 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-03-26T08:08:39.714 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-26T08:08:39.714 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-03-26T08:08:39.714 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-26T08:08:39.714 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-03-26T08:08:39.714 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-26T08:08:39.715 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-03-26T08:08:39.715 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-26T08:08:39.715 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-26T08:08:39.715 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-03-26T08:08:39.715 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-26T08:08:39.715 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-03-26T08:08:39.715 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-26T08:08:39.715 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-26T08:08:39.715 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-26T08:08:39.715 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-26T08:08:39.715 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-26T08:08:39.715 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-26T08:08:39.715 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-03-26T08:08:39.715 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-26T08:08:39.716 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-26T08:08:39.716 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-26T08:08:39.716 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-26T08:08:39.716 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-26T08:08:39.716 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-03-26T08:08:39.716 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-26T08:08:39.716 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-26T08:08:39.716 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-03-26T08:08:39.716 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-26T08:08:39.716 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-26T08:08:39.716 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-03-26T08:08:39.716 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-26T08:08:39.716 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-26T08:08:39.716 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-26T08:08:39.716 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-26T08:08:39.717 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-26T08:08:39.717 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-26T08:08:39.717 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-26T08:08:39.717 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-03-26T08:08:39.717 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-26T08:08:39.717 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-26T08:08:39.717 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-26T08:08:39.717 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-03-26T08:08:39.717 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-26T08:08:39.717 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-26T08:08:39.717 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-26T08:08:39.717 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-03-26T08:08:39.717 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-26T08:08:39.717 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-03-26T08:08:39.718 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-26T08:08:39.718 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-03-26T08:08:39.718 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-26T08:08:39.718 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-26T08:08:39.718 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-26T08:08:39.718 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-26T08:08:39.718 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-03-26T08:08:39.718 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-26T08:08:39.718 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-03-26T08:08:39.718 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-26T08:08:39.718 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-26T08:08:39.718 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-03-26T08:08:39.718 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-26T08:08:39.719 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-26T08:08:39.719 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-26T08:08:39.719 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-03-26T08:08:39.719 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-26T08:08:39.719 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-26T08:08:39.719 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-03-26T08:08:39.719 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-26T08:08:39.719 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-26T08:08:39.719 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-26T08:08:39.719 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-26T08:08:39.719 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-26T08:08:39.719 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-26T08:08:39.719 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-26T08:08:39.720 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-26T08:08:39.720 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-26T08:08:39.720 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-26T08:08:39.720 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-26T08:08:39.720 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-03-26T08:08:39.720 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-26T08:08:39.720 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-26T08:08:39.720 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-26T08:08:39.720 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-26T08:08:39.720 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-03-26T08:08:39.720 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-26T08:08:39.720 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-03-26T08:08:39.720 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-03-26T08:08:39.721 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-26T08:08:39.721 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-03-26T08:08:39.721 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-26T08:08:39.721 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-26T08:08:39.721 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-26T08:08:39.721 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-26T08:08:39.721 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-26T08:08:39.721 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-26T08:08:39.721 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-03-26T08:08:39.721 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-26T08:08:39.721 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-03-26T08:08:39.721 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-03-26T08:08:39.721 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-03-26T08:08:39.721 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-26T08:08:39.721 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-26T08:08:39.722 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-26T08:08:39.722 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-26T08:08:39.722 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-26T08:08:39.722 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-26T08:08:39.722 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-26T08:08:39.722 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-26T08:08:39.722 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-26T08:08:39.722 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-26T08:08:39.722 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-26T08:08:39.722 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-26T08:08:39.722 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-26T08:08:39.722 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-26T08:08:39.722 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-26T08:08:39.723 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-26T08:08:39.723 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-26T08:08:39.723 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-26T08:08:39.723 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-03-26T08:08:39.723 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-03-26T08:08:39.723 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-26T08:08:39.723 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-26T08:08:39.723 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-26T08:08:39.723 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-26T08:08:39.723 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-03-26T08:08:39.723 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-03-26T08:08:39.723 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-26T08:08:39.723 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-26T08:08:39.723 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-03-26T08:08:39.724 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-26T08:08:39.724 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-03-26T08:08:39.724 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-03-26T08:08:39.724 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-26T08:08:39.724 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-03-26T08:08:39.724 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-26T08:08:39.724 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-03-26T08:08:39.724 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-26T08:08:39.724 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-26T08:08:39.724 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-26T08:08:39.724 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-26T08:08:39.724 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-26T08:08:39.724 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-03-26T08:08:39.724 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-03-26T08:08:39.725 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-26T08:08:39.725 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-26T08:08:39.725 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-26T08:08:39.725 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-26T08:08:39.725 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-03-26T08:08:39.725 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-26T08:08:39.725 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-03-26T08:08:39.725 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-26T08:08:39.725 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-26T08:08:39.725 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-03-26T08:08:39.725 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-26T08:08:39.725 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-26T08:08:39.725 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-26T08:08:39.725 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-26T08:08:39.725 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-26T08:08:39.726 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-26T08:08:39.726 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-26T08:08:39.726 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-26T08:08:39.726 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-26T08:08:39.726 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-26T08:08:39.726 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-03-26T08:08:39.726 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-26T08:08:39.726 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-26T08:08:39.726 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-26T08:08:39.726 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-26T08:08:39.726 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-26T08:08:39.726 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-26T08:08:39.726 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-26T08:08:39.726 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-03-26T08:08:39.727 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-26T08:08:39.727 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-03-26T08:08:39.727 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-26T08:08:39.727 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-26T08:08:39.727 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-26T08:08:39.727 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-26T08:08:39.727 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-26T08:08:39.727 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-26T08:08:39.727 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-26T08:08:39.727 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-26T08:08:39.727 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-26T08:08:39.727 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-26T08:08:39.727 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-26T08:08:39.727 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-26T08:08:39.727 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-26T08:08:39.728 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-26T08:08:39.728 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-26T08:08:39.728 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-26T08:08:39.728 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-26T08:08:39.728 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-26T08:08:39.728 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-26T08:08:39.728 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-26T08:08:39.728 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-26T08:08:39.728 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-26T08:08:39.728 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-26T08:08:39.728 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-26T08:08:39.728 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-26T08:08:39.728 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-26T08:08:39.728 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-26T08:08:39.729 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-26T08:08:39.729 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-26T08:08:39.729 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-26T08:08:39.729 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-26T08:08:39.729 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-26T08:08:39.729 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-26T08:08:39.729 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-26T08:08:39.729 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-26T08:08:39.729 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-26T08:08:39.729 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-26T08:08:39.729 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-26T08:08:39.729 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-26T08:08:39.729 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-26T08:08:39.729 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-26T08:08:39.729 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-26T08:08:39.730 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-26T08:08:39.730 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-26T08:08:39.730 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-26T08:08:39.730 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-03-26T08:08:39.730 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-26T08:08:39.730 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-26T08:08:39.730 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-26T08:08:39.730 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-26T08:08:39.730 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-26T08:08:39.730 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-26T08:08:39.730 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-26T08:08:39.730 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-26T08:08:39.730 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-26T08:08:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-26T08:08:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-26T08:08:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-26T08:08:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-26T08:08:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-26T08:08:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-26T08:08:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-03-26T08:08:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-26T08:08:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-26T08:08:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-26T08:08:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-03-26T08:08:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-03-26T08:08:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-26T08:08:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-26T08:08:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-26T08:08:39.731 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-26T08:08:39.732 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-03-26T08:08:39.732 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-26T08:08:39.732 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-26T08:08:39.732 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-26T08:08:39.732 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-26T08:08:39.732 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-03-26T08:08:39.732 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-03-26T08:08:39.732 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-26T08:08:39.732 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-26T08:08:39.732 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-26T08:08:39.732 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-03-26T08:08:39.732 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-03-26T08:08:39.732 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-03-26T08:08:39.732 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-03-26T08:08:39.733 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-03-26T08:08:39.733 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-26T08:08:39.733 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-26T08:08:39.733 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-03-26T08:08:39.733 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-03-26T08:08:39.733 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-03-26T08:08:39.733 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-26T08:08:39.733 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-26T08:08:39.733 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-03-26T08:08:39.733 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-26T08:08:39.733 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-26T08:08:39.733 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-26T08:08:39.733 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-26T08:08:39.734 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-26T08:08:39.734 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-26T08:08:39.734 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-26T08:08:39.734 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-26T08:08:39.734 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-26T08:08:39.734 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-26T08:08:39.734 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-26T08:08:39.734 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-26T08:08:39.734 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-03-26T08:08:39.734 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-26T08:08:39.734 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-26T08:08:39.734 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-26T08:08:39.734 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-26T08:08:39.734 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-26T08:08:39.735 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-26T08:08:39.735 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-26T08:08:39.735 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-26T08:08:39.735 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-26T08:08:39.735 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-26T08:08:39.735 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-26T08:08:39.735 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-26T08:08:39.735 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-26T08:08:39.735 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-26T08:08:39.735 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-26T08:08:39.735 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-26T08:08:39.735 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-26T08:08:39.735 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-26T08:08:39.735 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-26T08:08:39.735 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-26T08:08:39.736 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-26T08:08:39.736 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-26T08:08:39.736 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-26T08:08:39.736 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-26T08:08:39.736 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-26T08:08:39.736 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-26T08:08:39.736 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-03-26T08:08:39.736 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-26T08:08:39.736 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-26T08:08:39.736 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-26T08:08:39.736 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-26T08:08:39.736 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-03-26T08:08:39.736 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-26T08:08:39.736 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-26T08:08:39.737 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-03-26T08:08:39.737 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-26T08:08:39.737 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-03-26T08:08:39.737 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-03-26T08:08:39.737 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-26T08:08:39.737 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-03-26T08:08:39.737 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-26T08:08:39.737 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-03-26T08:08:39.737 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-26T08:08:39.737 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-26T08:08:39.737 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-03-26T08:08:39.737 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-26T08:08:39.737 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-03-26T08:08:39.737 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-26T08:08:39.738 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-03-26T08:08:39.738 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-26T08:08:39.738 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-03-26T08:08:39.738 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-26T08:08:39.738 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-03-26T08:08:39.738 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-26T08:08:39.738 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-03-26T08:08:39.738 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-26T08:08:39.738 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-03-26T08:08:39.738 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-26T08:08:39.738 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-03-26T08:08:39.738 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-26T08:08:39.738 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-03-26T08:08:39.738 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-03-26T08:08:39.739 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-26T08:08:39.739 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-03-26T08:08:39.739 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-03-26T08:08:39.739 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-26T08:08:39.739 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-03-26T08:08:39.739 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-03-26T08:08:39.739 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-26T08:08:39.739 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-03-26T08:08:39.739 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-26T08:08:39.739 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-03-26T08:08:39.739 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-26T08:08:39.739 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-03-26T08:08:39.740 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-26T08:08:39.740 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-03-26T08:08:39.740 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-26T08:08:39.740 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-03-26T08:08:39.740 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-26T08:08:39.740 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-03-26T08:08:39.740 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-26T08:08:39.740 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-03-26T08:08:39.740 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-26T08:08:39.740 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-03-26T08:08:39.740 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-03-26T08:08:39.740 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-26T08:08:39.740 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-03-26T08:08:39.740 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-26T08:08:39.741 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-03-26T08:08:39.741 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-26T08:08:39.741 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-26T08:08:39.741 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-26T08:08:39.741 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-26T08:08:39.741 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-03-26T08:08:39.741 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-26T08:08:39.741 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-26T08:08:39.741 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-03-26T08:08:39.741 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-26T08:08:39.741 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-03-26T08:08:39.741 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-26T08:08:39.741 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-03-26T08:08:39.741 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-26T08:08:39.742 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-26T08:08:39.742 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-26T08:08:39.742 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-26T08:08:39.742 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-26T08:08:39.742 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-26T08:08:39.742 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-26T08:08:39.742 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-26T08:08:39.742 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-26T08:08:39.742 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-26T08:08:39.742 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-26T08:08:39.742 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-26T08:08:39.742 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-26T08:08:39.742 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-26T08:08:39.743 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-26T08:08:39.743 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-26T08:08:39.743 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-26T08:08:39.743 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-26T08:08:39.743 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-26T08:08:39.743 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-26T08:08:39.743 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-26T08:08:39.743 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-26T08:08:39.743 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-26T08:08:39.743 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-26T08:08:39.743 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-26T08:08:39.743 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-26T08:08:39.743 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-26T08:08:39.743 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-26T08:08:39.743 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-26T08:08:39.744 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-26T08:08:39.744 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-26T08:08:39.744 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-26T08:08:39.744 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-26T08:08:39.744 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-26T08:08:39.744 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-26T08:08:39.744 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-26T08:08:39.744 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-26T08:08:39.744 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-26T08:08:39.744 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-26T08:08:39.744 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-26T08:08:39.744 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-26T08:08:39.744 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-26T08:08:39.744 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-26T08:08:39.744 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-26T08:08:39.745 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-26T08:08:39.745 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-26T08:08:39.745 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-26T08:08:39.745 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-26T08:08:39.745 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-26T08:08:39.745 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-26T08:08:39.745 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-03-26T08:08:39.745 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-03-26T08:08:39.745 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-26T08:08:39.745 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-03-26T08:08:39.745 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-26T08:08:39.745 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-03-26T08:08:39.745 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-26T08:08:39.745 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-26T08:08:39.745 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-26T08:08:39.746 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-26T08:08:39.746 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-26T08:08:39.746 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-03-26T08:08:39.746 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-26T08:08:39.746 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-03-26T08:08:39.746 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-26T08:08:39.746 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-03-26T08:08:39.746 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-26T08:08:39.746 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-03-26T08:08:39.746 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-26T08:08:39.746 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-03-26T08:08:39.746 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-03-26T08:08:39.746 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-26T08:08:39.746 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-03-26T08:08:39.746 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-26T08:08:39.747 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-26T08:08:39.747 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-03-26T08:08:39.747 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-26T08:08:39.747 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-03-26T08:08:39.747 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-26T08:08:39.747 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-03-26T08:08:39.747 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-26T08:08:39.747 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-03-26T08:08:39.747 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-26T08:08:39.747 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-03-26T08:08:39.747 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-26T08:08:39.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-03-26T08:08:39.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-03-26T08:08:39.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-03-26T08:08:39.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-26T08:08:39.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-03-26T08:08:39.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-26T08:08:39.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-03-26T08:08:39.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-03-26T08:08:39.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-26T08:08:39.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-26T08:08:39.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-03-26T08:08:39.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-26T08:08:39.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-26T08:08:39.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-26T08:08:39.748 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-26T08:08:39.749 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-03-26T08:08:39.749 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-26T08:08:39.749 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-26T08:08:39.749 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-03-26T08:08:39.749 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-26T08:08:39.749 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-03-26T08:08:39.749 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-26T08:08:39.749 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-03-26T08:08:39.749 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-26T08:08:39.749 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-26T08:08:39.749 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-26T08:08:39.749 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-26T08:08:39.749 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-03-26T08:08:39.750 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-03-26T08:08:39.750 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-03-26T08:08:39.750 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-26T08:08:39.750 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-03-26T08:08:39.750 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-03-26T08:08:39.750 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-26T08:08:39.750 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-26T08:08:39.750 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-26T08:08:39.750 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-26T08:08:39.750 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-26T08:08:39.750 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-26T08:08:39.750 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-26T08:08:39.750 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-26T08:08:39.750 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-03-26T08:08:39.750 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-26T08:08:39.751 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-26T08:08:39.751 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-26T08:08:39.751 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-26T08:08:39.751 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-26T08:08:39.751 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-03-26T08:08:39.751 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-26T08:08:39.751 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-03-26T08:08:39.751 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-26T08:08:39.751 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-26T08:08:39.751 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-26T08:08:39.751 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-26T08:08:39.751 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-03-26T08:08:39.751 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-26T08:08:39.751 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-03-26T08:08:39.751 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-26T08:08:39.752 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-26T08:08:39.752 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-03-26T08:08:39.752 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-26T08:08:39.752 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-26T08:08:39.752 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-26T08:08:39.752 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-26T08:08:39.752 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-26T08:08:39.752 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-26T08:08:39.752 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-26T08:08:39.752 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-26T08:08:39.752 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-26T08:08:39.752 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-26T08:08:39.753 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-26T08:08:39.753 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-26T08:08:39.753 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-26T08:08:39.753 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-26T08:08:39.753 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-26T08:08:39.753 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-26T08:08:39.753 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-26T08:08:39.753 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-26T08:08:39.753 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-26T08:08:39.753 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-26T08:08:39.753 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-26T08:08:39.753 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-26T08:08:39.753 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-26T08:08:39.753 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-26T08:08:39.753 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-03-26T08:08:39.753 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-26T08:08:39.754 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-03-26T08:08:39.754 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-03-26T08:08:39.754 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-26T08:08:39.754 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-03-26T08:08:39.754 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-26T08:08:39.754 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-26T08:08:39.754 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-26T08:08:39.754 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-26T08:08:39.754 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-03-26T08:08:39.754 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-03-26T08:08:39.754 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-03-26T08:08:39.754 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-26T08:08:39.754 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-26T08:08:39.755 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-26T08:08:39.755 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-26T08:08:39.755 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-26T08:08:39.755 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-03-26T08:08:39.755 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-03-26T08:08:39.755 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-26T08:08:39.755 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-26T08:08:39.755 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-26T08:08:39.755 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-03-26T08:08:39.755 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-26T08:08:39.755 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-26T08:08:39.755 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-26T08:08:39.755 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-26T08:08:39.755 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-26T08:08:39.756 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-26T08:08:39.756 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-26T08:08:39.756 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-26T08:08:39.756 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-26T08:08:39.756 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-26T08:08:39.756 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-03-26T08:08:39.756 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-26T08:08:39.756 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-26T08:08:39.756 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-26T08:08:39.756 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-26T08:08:39.756 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-26T08:08:39.756 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-26T08:08:39.756 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-26T08:08:39.756 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-26T08:08:39.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-26T08:08:39.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-26T08:08:39.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-26T08:08:39.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-26T08:08:39.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-26T08:08:39.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-26T08:08:39.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-26T08:08:39.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-26T08:08:39.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-26T08:08:39.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-26T08:08:39.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-26T08:08:39.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-26T08:08:39.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-26T08:08:39.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-26T08:08:39.757 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-26T08:08:39.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-26T08:08:39.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-26T08:08:39.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-26T08:08:39.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-26T08:08:39.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-26T08:08:39.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-26T08:08:39.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-26T08:08:39.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-26T08:08:39.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-26T08:08:39.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-26T08:08:39.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-26T08:08:39.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-26T08:08:39.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-26T08:08:39.758 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-26T08:08:39.759 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-26T08:08:39.759 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-03-26T08:08:39.759 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-26T08:08:39.759 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-26T08:08:39.759 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-26T08:08:39.759 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-26T08:08:39.759 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-26T08:08:39.759 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-26T08:08:39.759 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-26T08:08:39.759 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-03-26T08:08:39.759 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-26T08:08:39.759 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-26T08:08:39.759 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-26T08:08:39.759 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-26T08:08:39.760 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-26T08:08:39.760 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-26T08:08:39.760 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-26T08:08:39.760 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-26T08:08:39.760 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-26T08:08:39.760 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-26T08:08:39.760 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-26T08:08:39.760 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-26T08:08:39.760 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-26T08:08:39.760 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-26T08:08:39.760 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-26T08:08:39.760 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-26T08:08:39.760 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-26T08:08:39.760 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-26T08:08:39.761 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-26T08:08:39.761 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-26T08:08:39.761 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-26T08:08:39.761 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-26T08:08:39.761 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-26T08:08:39.761 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-26T08:08:39.761 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-26T08:08:39.761 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-26T08:08:39.761 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-26T08:08:39.761 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-03-26T08:08:39.761 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-26T08:08:39.761 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-03-26T08:08:39.761 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-26T08:08:39.761 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-26T08:08:39.761 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-26T08:08:39.762 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-26T08:08:39.762 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-26T08:08:39.762 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-26T08:08:39.762 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-26T08:08:39.762 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-03-26T08:08:39.762 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-26T08:08:39.762 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-03-26T08:08:39.762 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-26T08:08:39.762 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-03-26T08:08:39.762 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-26T08:08:39.762 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-03-26T08:08:39.762 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-03-26T08:08:39.762 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-26T08:08:39.762 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-26T08:08:39.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-26T08:08:39.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-26T08:08:39.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-26T08:08:39.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-26T08:08:39.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-26T08:08:39.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-26T08:08:39.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-26T08:08:39.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-26T08:08:39.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-26T08:08:39.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-26T08:08:39.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-26T08:08:39.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-26T08:08:39.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-26T08:08:39.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-26T08:08:39.763 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-03-26T08:08:39.764 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-26T08:08:39.764 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-26T08:08:39.764 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-26T08:08:39.764 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-26T08:08:39.764 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-26T08:08:39.764 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-26T08:08:39.764 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-26T08:08:39.764 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-26T08:08:39.764 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-26T08:08:39.764 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-26T08:08:39.764 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-26T08:08:39.764 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-26T08:08:39.764 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-26T08:08:39.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-26T08:08:39.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-26T08:08:39.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-26T08:08:39.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-26T08:08:39.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-26T08:08:39.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-26T08:08:39.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-26T08:08:39.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-26T08:08:39.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-26T08:08:39.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-26T08:08:39.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-26T08:08:39.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-26T08:08:39.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-26T08:08:39.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-26T08:08:39.765 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-26T08:08:39.766 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-26T08:08:39.766 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-26T08:08:39.766 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-26T08:08:39.766 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-26T08:08:39.766 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-26T08:08:39.766 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-26T08:08:39.766 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-26T08:08:39.766 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-26T08:08:39.766 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-26T08:08:39.766 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-26T08:08:39.766 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-26T08:08:39.766 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-26T08:08:39.766 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-26T08:08:39.767 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-26T08:08:39.767 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-26T08:08:39.767 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-26T08:08:39.767 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-26T08:08:39.767 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-26T08:08:39.767 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-26T08:08:39.767 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-26T08:08:39.767 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-26T08:08:39.767 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-26T08:08:39.767 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-26T08:08:39.767 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-26T08:08:39.767 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-26T08:08:39.767 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-26T08:08:39.767 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-26T08:08:39.767 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-26T08:08:39.768 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-26T08:08:39.768 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-26T08:08:39.768 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-03-26T08:08:39.768 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-26T08:08:39.768 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-26T08:08:39.768 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-26T08:08:39.768 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-26T08:08:39.768 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-03-26T08:08:39.768 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-26T08:08:39.768 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-26T08:08:39.768 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-26T08:08:39.768 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-26T08:08:39.768 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-26T08:08:39.768 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-26T08:08:39.769 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-26T08:08:39.769 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-26T08:08:39.769 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-26T08:08:39.769 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-26T08:08:39.769 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-26T08:08:39.769 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-26T08:08:39.769 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-26T08:08:39.769 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-26T08:08:39.769 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-26T08:08:39.769 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-26T08:08:39.769 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-26T08:08:39.769 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-26T08:08:39.769 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-26T08:08:39.770 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-26T08:08:39.770 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-26T08:08:39.770 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-26T08:08:39.770 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-26T08:08:39.770 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-26T08:08:39.770 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-26T08:08:39.770 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-26T08:08:39.770 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-26T08:08:39.770 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-26T08:08:39.770 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-26T08:08:39.770 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-26T08:08:39.770 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-26T08:08:39.770 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-26T08:08:39.770 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-26T08:08:39.770 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-26T08:08:39.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-26T08:08:39.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-26T08:08:39.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-26T08:08:39.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-26T08:08:39.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-26T08:08:39.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-26T08:08:39.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-26T08:08:39.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-26T08:08:39.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-26T08:08:39.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-26T08:08:39.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-26T08:08:39.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-26T08:08:39.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-26T08:08:39.771 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-26T08:08:39.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-26T08:08:39.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-26T08:08:39.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-26T08:08:39.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-26T08:08:39.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-26T08:08:39.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-26T08:08:39.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-26T08:08:39.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-26T08:08:39.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-26T08:08:39.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-26T08:08:39.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-26T08:08:39.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-26T08:08:39.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-26T08:08:39.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-26T08:08:39.772 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-26T08:08:39.773 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-26T08:08:39.773 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-26T08:08:39.773 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-26T08:08:39.773 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-26T08:08:39.773 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-26T08:08:39.773 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-26T08:08:39.773 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-26T08:08:39.773 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-26T08:08:39.773 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-03-26T08:08:39.773 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-26T08:08:39.773 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-03-26T08:08:39.773 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-26T08:08:39.773 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-03-26T08:08:39.773 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-26T08:08:39.774 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-26T08:08:39.774 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-26T08:08:39.774 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-26T08:08:39.774 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-03-26T08:08:39.774 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-26T08:08:39.774 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-03-26T08:08:39.774 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-26T08:08:39.774 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-26T08:08:39.774 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-26T08:08:39.774 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-03-26T08:08:39.774 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-26T08:08:39.774 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-26T08:08:39.774 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-26T08:08:39.774 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-26T08:08:39.775 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-26T08:08:39.775 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-26T08:08:39.775 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-26T08:08:39.775 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-26T08:08:39.775 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-26T08:08:39.775 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-26T08:08:39.775 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-26T08:08:39.775 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-26T08:08:39.775 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-26T08:08:39.775 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-26T08:08:39.775 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-26T08:08:39.775 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-26T08:08:39.775 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-26T08:08:39.775 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-26T08:08:39.776 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-26T08:08:39.776 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-26T08:08:39.776 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-26T08:08:39.776 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-26T08:08:39.776 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-26T08:08:39.776 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-26T08:08:39.776 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-26T08:08:39.776 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-26T08:08:39.776 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-26T08:08:39.776 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-26T08:08:39.776 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-26T08:08:39.776 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-26T08:08:39.776 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-26T08:08:39.777 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-26T08:08:39.777 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-26T08:08:39.777 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-26T08:08:39.777 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-26T08:08:39.777 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-26T08:08:39.777 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-26T08:08:39.777 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-26T08:08:39.777 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-26T08:08:39.777 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-26T08:08:39.777 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-26T08:08:39.777 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-26T08:08:39.777 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-26T08:08:39.777 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-03-26T08:08:39.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-26T08:08:39.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-26T08:08:39.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-03-26T08:08:39.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-03-26T08:08:39.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-26T08:08:39.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-26T08:08:39.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-26T08:08:39.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-03-26T08:08:39.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-26T08:08:39.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-03-26T08:08:39.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-26T08:08:39.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-26T08:08:39.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-03-26T08:08:39.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-26T08:08:39.778 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-03-26T08:08:39.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-26T08:08:39.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-26T08:08:39.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-26T08:08:39.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-26T08:08:39.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-26T08:08:39.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-03-26T08:08:39.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-26T08:08:39.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-03-26T08:08:39.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-03-26T08:08:39.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-03-26T08:08:39.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-26T08:08:39.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-03-26T08:08:39.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-03-26T08:08:39.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-03-26T08:08:39.779 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-03-26T08:08:39.780 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-26T08:08:39.780 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-26T08:08:39.780 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-26T08:08:39.780 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-26T08:08:39.780 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-03-26T08:08:39.780 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-26T08:08:39.780 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-26T08:08:39.780 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-26T08:08:39.780 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-26T08:08:39.780 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-26T08:08:39.780 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-26T08:08:39.780 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-26T08:08:39.780 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-26T08:08:39.780 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-26T08:08:39.780 INFO:teuthology.orchestra.run.smithi043.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-26T08:08:39.784 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-26T08:08:39.784 DEBUG:teuthology.orchestra.run.smithi043:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-26T08:08:39.786 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi183.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-26T08:08:39.786 DEBUG:teuthology.orchestra.run.smithi183:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-26T08:08:39.806 INFO:teuthology.orchestra.run.smithi043.stdout:postinstall scriptlet (using /bin/sh): 2024-03-26T08:08:39.807 INFO:teuthology.orchestra.run.smithi043.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-26T08:08:39.807 INFO:teuthology.orchestra.run.smithi043.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-26T08:08:39.807 INFO:teuthology.orchestra.run.smithi043.stdout:fi 2024-03-26T08:08:39.807 INFO:teuthology.orchestra.run.smithi043.stdout:for file in vmlinuz System.map config; do 2024-03-26T08:08:39.807 INFO:teuthology.orchestra.run.smithi043.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-03-26T08:08:39.807 INFO:teuthology.orchestra.run.smithi043.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-03-26T08:08:39.807 INFO:teuthology.orchestra.run.smithi043.stdout: fi 2024-03-26T08:08:39.807 INFO:teuthology.orchestra.run.smithi043.stdout:done 2024-03-26T08:08:39.807 INFO:teuthology.orchestra.run.smithi043.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-26T08:08:39.807 INFO:teuthology.orchestra.run.smithi043.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-26T08:08:39.807 INFO:teuthology.orchestra.run.smithi043.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-03-26T08:08:39.807 INFO:teuthology.orchestra.run.smithi043.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-26T08:08:39.807 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-03-26T08:08:39.808 INFO:teuthology.orchestra.run.smithi043.stdout:fi 2024-03-26T08:08:39.808 INFO:teuthology.orchestra.run.smithi043.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-26T08:08:39.808 INFO:teuthology.orchestra.run.smithi043.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-26T08:08:39.808 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-03-26T08:08:39.808 INFO:teuthology.orchestra.run.smithi043.stdout:fi 2024-03-26T08:08:39.809 DEBUG:teuthology.orchestra.run.smithi043:> sudo rpm -qi grub2-tools 2024-03-26T08:08:39.877 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-03-26T08:08:39.877 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-03-26T08:08:39.877 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-03-26T08:08:39.877 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-03-26T08:08:39.878 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-03-26T08:08:39.878 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-03-26T08:08:39.878 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-03-26T08:08:39.878 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-26T08:08:39.878 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-26T08:08:39.878 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-26T08:08:39.879 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-26T08:08:39.879 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-26T08:08:39.879 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-26T08:08:39.879 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-26T08:08:39.879 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-26T08:08:39.879 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-26T08:08:39.880 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-26T08:08:39.880 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-26T08:08:39.880 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-26T08:08:39.880 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-26T08:08:39.880 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-26T08:08:39.880 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-26T08:08:39.881 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-26T08:08:39.881 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-26T08:08:39.881 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-26T08:08:39.881 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-26T08:08:39.881 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-26T08:08:39.881 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-03-26T08:08:39.882 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-03-26T08:08:39.882 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-03-26T08:08:39.882 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-26T08:08:39.882 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-03-26T08:08:39.882 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-26T08:08:39.883 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-26T08:08:39.883 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-26T08:08:39.883 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-03-26T08:08:39.883 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-26T08:08:39.883 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-26T08:08:39.883 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-26T08:08:39.883 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-03-26T08:08:39.883 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-26T08:08:39.883 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-26T08:08:39.883 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-26T08:08:39.883 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-26T08:08:39.884 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-26T08:08:39.884 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-26T08:08:39.884 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-26T08:08:39.884 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-26T08:08:39.884 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-26T08:08:39.884 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-26T08:08:39.884 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-26T08:08:39.884 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-26T08:08:39.884 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-26T08:08:39.884 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-26T08:08:39.884 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-26T08:08:39.884 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-26T08:08:39.884 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-26T08:08:39.884 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-26T08:08:39.884 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-26T08:08:39.885 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-26T08:08:39.885 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-26T08:08:39.885 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-26T08:08:39.885 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-26T08:08:39.885 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-26T08:08:39.885 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-26T08:08:39.885 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-26T08:08:39.885 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-26T08:08:39.885 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-26T08:08:39.885 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-26T08:08:39.885 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-26T08:08:39.885 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-26T08:08:39.885 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-26T08:08:39.885 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-26T08:08:39.886 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-26T08:08:39.886 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-26T08:08:39.886 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-26T08:08:39.886 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-26T08:08:39.886 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-26T08:08:39.886 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-26T08:08:39.886 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-26T08:08:39.886 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-26T08:08:39.886 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-26T08:08:39.886 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-26T08:08:39.886 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-26T08:08:39.886 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-26T08:08:39.886 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-03-26T08:08:39.886 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-03-26T08:08:39.887 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-26T08:08:39.887 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-26T08:08:39.887 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-03-26T08:08:39.887 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-26T08:08:39.887 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-26T08:08:39.887 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-26T08:08:39.887 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-26T08:08:39.887 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-26T08:08:39.887 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-03-26T08:08:39.887 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-26T08:08:39.887 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-26T08:08:39.887 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-03-26T08:08:39.887 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-26T08:08:39.887 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-26T08:08:39.887 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-26T08:08:39.887 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-26T08:08:39.888 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-26T08:08:39.888 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-26T08:08:39.888 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-26T08:08:39.888 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-26T08:08:39.888 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-26T08:08:39.888 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-26T08:08:39.888 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-26T08:08:39.888 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-26T08:08:39.888 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-26T08:08:39.888 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-26T08:08:39.888 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-26T08:08:39.888 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-26T08:08:39.888 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-26T08:08:39.888 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-26T08:08:39.888 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-26T08:08:39.889 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-26T08:08:39.889 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-26T08:08:39.889 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-26T08:08:39.889 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-26T08:08:39.889 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-26T08:08:39.889 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-26T08:08:39.889 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-26T08:08:39.889 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-26T08:08:39.889 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-26T08:08:39.889 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-26T08:08:39.889 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-26T08:08:39.889 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-26T08:08:39.889 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-26T08:08:39.889 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-26T08:08:39.889 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-26T08:08:39.890 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-26T08:08:39.890 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-03-26T08:08:39.890 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-26T08:08:39.890 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-26T08:08:39.890 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-26T08:08:39.890 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-26T08:08:39.890 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-26T08:08:39.890 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-26T08:08:39.890 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-26T08:08:39.890 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-26T08:08:39.890 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-26T08:08:39.890 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-26T08:08:39.890 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-26T08:08:39.890 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-26T08:08:39.891 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-26T08:08:39.891 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-26T08:08:39.891 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-26T08:08:39.891 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-26T08:08:39.891 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-26T08:08:39.891 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-03-26T08:08:39.891 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-26T08:08:39.891 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-03-26T08:08:39.891 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-03-26T08:08:39.891 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-26T08:08:39.891 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-26T08:08:39.891 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-03-26T08:08:39.891 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-26T08:08:39.891 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-03-26T08:08:39.891 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-03-26T08:08:39.892 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-26T08:08:39.892 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-03-26T08:08:39.892 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-03-26T08:08:39.892 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-26T08:08:39.892 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-26T08:08:39.892 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-03-26T08:08:39.892 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-03-26T08:08:39.892 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-26T08:08:39.892 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-26T08:08:39.892 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-03-26T08:08:39.892 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-26T08:08:39.892 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-26T08:08:39.892 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-26T08:08:39.892 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-03-26T08:08:39.893 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-03-26T08:08:39.893 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-26T08:08:39.893 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-03-26T08:08:39.893 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-03-26T08:08:39.893 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-26T08:08:39.893 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-03-26T08:08:39.893 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-26T08:08:39.893 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-03-26T08:08:39.893 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-26T08:08:39.893 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-26T08:08:39.893 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-26T08:08:39.893 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-26T08:08:39.893 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-26T08:08:39.893 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-03-26T08:08:39.893 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-26T08:08:39.894 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-26T08:08:39.894 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-26T08:08:39.894 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-26T08:08:39.894 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-26T08:08:39.894 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-26T08:08:39.894 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-26T08:08:39.894 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-03-26T08:08:39.894 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-26T08:08:39.894 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-26T08:08:39.894 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-26T08:08:39.894 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-03-26T08:08:39.894 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-03-26T08:08:39.894 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-26T08:08:39.894 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-26T08:08:39.894 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-26T08:08:39.895 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-26T08:08:39.895 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-26T08:08:39.895 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-03-26T08:08:39.895 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-03-26T08:08:39.895 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-03-26T08:08:39.895 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-26T08:08:39.895 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-26T08:08:39.895 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-26T08:08:39.895 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-26T08:08:39.895 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-26T08:08:39.895 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-26T08:08:39.895 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-03-26T08:08:39.895 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-03-26T08:08:39.895 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-26T08:08:39.896 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-26T08:08:39.896 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-03-26T08:08:39.896 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-03-26T08:08:39.896 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-03-26T08:08:39.896 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-26T08:08:39.896 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-26T08:08:39.896 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-26T08:08:39.896 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-26T08:08:39.896 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-26T08:08:39.896 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-03-26T08:08:39.896 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-26T08:08:39.896 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-03-26T08:08:39.896 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-26T08:08:39.896 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-03-26T08:08:39.896 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-26T08:08:39.897 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-03-26T08:08:39.897 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-03-26T08:08:39.897 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-26T08:08:39.897 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-03-26T08:08:39.897 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-26T08:08:39.897 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-03-26T08:08:39.897 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-26T08:08:39.897 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-26T08:08:39.897 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-26T08:08:39.897 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-26T08:08:39.897 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-26T08:08:39.897 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-26T08:08:39.897 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-26T08:08:39.897 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-26T08:08:39.897 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-26T08:08:39.897 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-26T08:08:39.898 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-26T08:08:39.898 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-26T08:08:39.898 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-26T08:08:39.898 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-03-26T08:08:39.898 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-03-26T08:08:39.898 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-26T08:08:39.898 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-03-26T08:08:39.898 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-26T08:08:39.898 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-26T08:08:39.898 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-03-26T08:08:39.898 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-26T08:08:39.898 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-26T08:08:39.898 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-26T08:08:39.899 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-26T08:08:39.899 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-26T08:08:39.899 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-26T08:08:39.899 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-26T08:08:39.899 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-26T08:08:39.899 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-26T08:08:39.899 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-26T08:08:39.899 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-26T08:08:39.899 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-26T08:08:39.899 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-26T08:08:39.899 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-26T08:08:39.899 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-26T08:08:39.899 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-26T08:08:39.899 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-26T08:08:39.900 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-26T08:08:39.900 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-26T08:08:39.900 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-26T08:08:39.900 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-26T08:08:39.900 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-26T08:08:39.900 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-26T08:08:39.900 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-26T08:08:39.900 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-26T08:08:39.900 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-26T08:08:39.900 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-26T08:08:39.900 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-26T08:08:39.900 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-26T08:08:39.900 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-26T08:08:39.900 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-26T08:08:39.900 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-26T08:08:39.901 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-26T08:08:39.901 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-26T08:08:39.901 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-26T08:08:39.901 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-26T08:08:39.901 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-26T08:08:39.901 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-26T08:08:39.901 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-26T08:08:39.901 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-26T08:08:39.901 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-26T08:08:39.901 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-26T08:08:39.901 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-26T08:08:39.901 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-26T08:08:39.901 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-26T08:08:39.902 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-26T08:08:39.902 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-26T08:08:39.902 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-26T08:08:39.902 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-26T08:08:39.902 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-26T08:08:39.902 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-26T08:08:39.902 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-26T08:08:39.902 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-26T08:08:39.902 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-26T08:08:39.902 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-26T08:08:39.902 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-26T08:08:39.902 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-26T08:08:39.902 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-26T08:08:39.902 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-26T08:08:39.902 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-26T08:08:39.903 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-26T08:08:39.903 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-26T08:08:39.903 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-26T08:08:39.903 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-26T08:08:39.903 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-03-26T08:08:39.903 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-26T08:08:39.903 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-26T08:08:39.903 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-26T08:08:39.903 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-03-26T08:08:39.903 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-26T08:08:39.903 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-26T08:08:39.903 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-26T08:08:39.903 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-26T08:08:39.903 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-26T08:08:39.903 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-26T08:08:39.904 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-26T08:08:39.904 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-26T08:08:39.904 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-26T08:08:39.904 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-26T08:08:39.904 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-26T08:08:39.904 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-26T08:08:39.904 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-26T08:08:39.904 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-26T08:08:39.904 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-26T08:08:39.904 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-26T08:08:39.904 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-26T08:08:39.904 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-26T08:08:39.904 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-26T08:08:39.904 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-26T08:08:39.904 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-26T08:08:39.905 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-26T08:08:39.905 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-26T08:08:39.905 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-26T08:08:39.905 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-26T08:08:39.905 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-26T08:08:39.905 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-26T08:08:39.905 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-26T08:08:39.905 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-26T08:08:39.905 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-26T08:08:39.905 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-26T08:08:39.905 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-26T08:08:39.905 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-26T08:08:39.905 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-26T08:08:39.905 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-26T08:08:39.905 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-26T08:08:39.906 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-26T08:08:39.906 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-26T08:08:39.906 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-26T08:08:39.906 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-26T08:08:39.906 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-26T08:08:39.906 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-26T08:08:39.906 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-26T08:08:39.906 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-26T08:08:39.906 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-26T08:08:39.906 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-26T08:08:39.906 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-26T08:08:39.906 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-26T08:08:39.906 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-26T08:08:39.906 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-26T08:08:39.907 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-26T08:08:39.907 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-26T08:08:39.907 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-26T08:08:39.907 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-26T08:08:39.907 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-26T08:08:39.907 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-26T08:08:39.907 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-26T08:08:39.907 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-26T08:08:39.907 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-26T08:08:39.907 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-26T08:08:39.907 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-26T08:08:39.907 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-26T08:08:39.907 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-26T08:08:39.908 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-26T08:08:39.908 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-26T08:08:39.908 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-26T08:08:39.908 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-26T08:08:39.908 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-26T08:08:39.908 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-26T08:08:39.908 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-26T08:08:39.908 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-26T08:08:39.908 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-26T08:08:39.908 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-26T08:08:39.908 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-26T08:08:39.908 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-26T08:08:39.908 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-26T08:08:39.908 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-26T08:08:39.909 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-26T08:08:39.909 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-26T08:08:39.909 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-26T08:08:39.909 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-26T08:08:39.909 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-26T08:08:39.909 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-26T08:08:39.909 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-26T08:08:39.909 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-26T08:08:39.909 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-26T08:08:39.909 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-26T08:08:39.909 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-26T08:08:39.909 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-03-26T08:08:39.909 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-26T08:08:39.909 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-26T08:08:39.910 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-26T08:08:39.910 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-26T08:08:39.910 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-26T08:08:39.910 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-26T08:08:39.910 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-26T08:08:39.910 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-26T08:08:39.910 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-26T08:08:39.910 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-26T08:08:39.910 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-26T08:08:39.910 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-26T08:08:39.910 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-26T08:08:39.910 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-26T08:08:39.910 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-26T08:08:39.910 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-26T08:08:39.911 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-26T08:08:39.911 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-26T08:08:39.911 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-26T08:08:39.911 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-26T08:08:39.911 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-26T08:08:39.911 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-26T08:08:39.911 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-26T08:08:39.911 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-26T08:08:39.911 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-26T08:08:39.911 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-26T08:08:39.911 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-26T08:08:39.911 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-26T08:08:39.911 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-26T08:08:39.911 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-26T08:08:39.912 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-26T08:08:39.912 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-26T08:08:39.912 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-26T08:08:39.912 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-26T08:08:39.912 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-26T08:08:39.912 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-26T08:08:39.912 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-26T08:08:39.912 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-26T08:08:39.912 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-26T08:08:39.912 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-26T08:08:39.912 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-26T08:08:39.912 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-03-26T08:08:39.912 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-03-26T08:08:39.913 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-26T08:08:39.913 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-26T08:08:39.913 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-03-26T08:08:39.913 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-26T08:08:39.913 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-26T08:08:39.913 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-26T08:08:39.913 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-26T08:08:39.913 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-26T08:08:39.913 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-26T08:08:39.913 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-26T08:08:39.913 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-26T08:08:39.913 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-26T08:08:39.913 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-26T08:08:39.913 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-26T08:08:39.913 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-26T08:08:39.913 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-26T08:08:39.914 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-26T08:08:39.914 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-26T08:08:39.914 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-26T08:08:39.914 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-26T08:08:39.914 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-26T08:08:39.914 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-26T08:08:39.914 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-26T08:08:39.914 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-26T08:08:39.914 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-26T08:08:39.914 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-26T08:08:39.914 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-03-26T08:08:39.914 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-26T08:08:39.914 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-26T08:08:39.914 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-26T08:08:39.915 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-26T08:08:39.915 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-03-26T08:08:39.915 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-03-26T08:08:39.915 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-26T08:08:39.915 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-26T08:08:39.915 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-26T08:08:39.915 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-26T08:08:39.915 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-26T08:08:39.915 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-26T08:08:39.915 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-26T08:08:39.915 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-03-26T08:08:39.915 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-03-26T08:08:39.915 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-26T08:08:39.915 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-03-26T08:08:39.916 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-26T08:08:39.916 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-03-26T08:08:39.916 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-26T08:08:39.916 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-03-26T08:08:39.916 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-26T08:08:39.916 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-03-26T08:08:39.916 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-03-26T08:08:39.916 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-26T08:08:39.916 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-03-26T08:08:39.916 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-26T08:08:39.916 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-03-26T08:08:39.916 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-26T08:08:39.916 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-03-26T08:08:39.916 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-26T08:08:39.916 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-03-26T08:08:39.917 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-26T08:08:39.917 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-03-26T08:08:39.917 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-03-26T08:08:39.917 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-26T08:08:39.917 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-26T08:08:39.917 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-26T08:08:39.917 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-26T08:08:39.917 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-03-26T08:08:39.917 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-26T08:08:39.917 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-03-26T08:08:39.917 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-26T08:08:39.917 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-26T08:08:39.917 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-26T08:08:39.917 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-26T08:08:39.917 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-26T08:08:39.918 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-26T08:08:39.918 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-26T08:08:39.918 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-26T08:08:39.918 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-03-26T08:08:39.918 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-26T08:08:39.918 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-03-26T08:08:39.918 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-26T08:08:39.918 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-26T08:08:39.918 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-26T08:08:39.918 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-26T08:08:39.918 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-26T08:08:39.918 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-03-26T08:08:39.918 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-26T08:08:39.918 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-26T08:08:39.918 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-26T08:08:39.919 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-26T08:08:39.919 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-03-26T08:08:39.919 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-26T08:08:39.919 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-26T08:08:39.919 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-26T08:08:39.919 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-26T08:08:39.919 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-26T08:08:39.919 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-26T08:08:39.919 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-26T08:08:39.919 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-26T08:08:39.919 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-26T08:08:39.919 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-03-26T08:08:39.919 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-03-26T08:08:39.919 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-26T08:08:39.919 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-26T08:08:39.920 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-26T08:08:39.920 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-26T08:08:39.920 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-26T08:08:39.920 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-26T08:08:39.920 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-26T08:08:39.920 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-26T08:08:39.920 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-26T08:08:39.920 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-26T08:08:39.920 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-26T08:08:39.920 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-26T08:08:39.920 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-26T08:08:39.921 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-26T08:08:39.921 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-26T08:08:39.921 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-26T08:08:39.921 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-03-26T08:08:39.921 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-26T08:08:39.921 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-26T08:08:39.921 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-26T08:08:39.921 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-26T08:08:39.921 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-26T08:08:39.921 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-03-26T08:08:39.921 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-03-26T08:08:39.921 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-26T08:08:39.921 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-26T08:08:39.921 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-26T08:08:39.921 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-26T08:08:39.922 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-26T08:08:39.922 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-26T08:08:39.922 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-26T08:08:39.922 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-03-26T08:08:39.922 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-26T08:08:39.922 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-26T08:08:39.922 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-03-26T08:08:39.922 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-26T08:08:39.922 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-26T08:08:39.922 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-26T08:08:39.922 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-26T08:08:39.922 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-03-26T08:08:39.922 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-03-26T08:08:39.922 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-26T08:08:39.922 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-26T08:08:39.923 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-26T08:08:39.923 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-26T08:08:39.923 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-03-26T08:08:39.923 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-26T08:08:39.923 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-03-26T08:08:39.923 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-26T08:08:39.923 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-26T08:08:39.923 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-26T08:08:39.923 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-26T08:08:39.923 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-26T08:08:39.923 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-26T08:08:39.923 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-26T08:08:39.923 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-03-26T08:08:39.923 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-26T08:08:39.924 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-26T08:08:39.924 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-03-26T08:08:39.924 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-26T08:08:39.924 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-26T08:08:39.924 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-26T08:08:39.924 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-03-26T08:08:39.924 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-26T08:08:39.924 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-03-26T08:08:39.924 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-26T08:08:39.924 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-26T08:08:39.924 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-26T08:08:39.924 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-26T08:08:39.924 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-26T08:08:39.924 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-26T08:08:39.924 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-26T08:08:39.925 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-03-26T08:08:39.925 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-26T08:08:39.925 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-03-26T08:08:39.925 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-03-26T08:08:39.925 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-26T08:08:39.925 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-03-26T08:08:39.925 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-03-26T08:08:39.925 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-26T08:08:39.925 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-26T08:08:39.925 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-26T08:08:39.925 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-03-26T08:08:39.925 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-26T08:08:39.925 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-26T08:08:39.925 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-26T08:08:39.925 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-03-26T08:08:39.926 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-03-26T08:08:39.926 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-26T08:08:39.926 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-26T08:08:39.926 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-26T08:08:39.926 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-26T08:08:39.926 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-03-26T08:08:39.926 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-26T08:08:39.926 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-26T08:08:39.926 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-26T08:08:39.926 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-26T08:08:39.926 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-03-26T08:08:39.926 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-26T08:08:39.926 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-03-26T08:08:39.926 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-26T08:08:39.926 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-03-26T08:08:39.927 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-26T08:08:39.927 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-03-26T08:08:39.927 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-26T08:08:39.927 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-03-26T08:08:39.927 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-26T08:08:39.927 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-26T08:08:39.927 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-26T08:08:39.927 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-03-26T08:08:39.927 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-03-26T08:08:39.927 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-26T08:08:39.927 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-26T08:08:39.927 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-26T08:08:39.927 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-26T08:08:39.928 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-26T08:08:39.928 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-03-26T08:08:39.928 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-26T08:08:39.928 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-26T08:08:39.928 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-03-26T08:08:39.928 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-26T08:08:39.928 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-26T08:08:39.928 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-26T08:08:39.928 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-26T08:08:39.928 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-26T08:08:39.928 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-03-26T08:08:39.928 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-26T08:08:39.928 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-26T08:08:39.928 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-03-26T08:08:39.928 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-03-26T08:08:39.929 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-26T08:08:39.929 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-03-26T08:08:39.929 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-26T08:08:39.929 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-26T08:08:39.929 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-26T08:08:39.929 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-26T08:08:39.929 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-26T08:08:39.929 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-26T08:08:39.929 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-26T08:08:39.929 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-26T08:08:39.929 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-26T08:08:39.929 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-26T08:08:39.929 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-03-26T08:08:39.929 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-03-26T08:08:39.929 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-26T08:08:39.930 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-03-26T08:08:39.930 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-03-26T08:08:39.930 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-26T08:08:39.930 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-26T08:08:39.930 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-26T08:08:39.930 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-26T08:08:39.930 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-26T08:08:39.930 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-26T08:08:39.930 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-03-26T08:08:39.930 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-26T08:08:39.930 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-26T08:08:39.930 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-26T08:08:39.930 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-03-26T08:08:39.930 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-03-26T08:08:39.931 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-26T08:08:39.931 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-26T08:08:39.931 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-26T08:08:39.931 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-03-26T08:08:39.931 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-26T08:08:39.931 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-03-26T08:08:39.931 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-26T08:08:39.931 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-03-26T08:08:39.931 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-26T08:08:39.931 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-03-26T08:08:39.931 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-26T08:08:39.931 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-03-26T08:08:39.931 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-26T08:08:39.931 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-03-26T08:08:39.932 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-26T08:08:39.932 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-03-26T08:08:39.932 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-26T08:08:39.932 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-03-26T08:08:39.932 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-26T08:08:39.932 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-26T08:08:39.932 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-26T08:08:39.932 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-26T08:08:39.932 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-26T08:08:39.932 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-26T08:08:39.932 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-03-26T08:08:39.932 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-26T08:08:39.932 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-26T08:08:39.932 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-26T08:08:39.932 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-03-26T08:08:39.933 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-26T08:08:39.933 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-26T08:08:39.933 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-26T08:08:39.933 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-26T08:08:39.933 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-26T08:08:39.933 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-26T08:08:39.933 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-03-26T08:08:39.933 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-26T08:08:39.933 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-03-26T08:08:39.933 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-26T08:08:39.933 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-26T08:08:39.933 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-03-26T08:08:39.933 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-26T08:08:39.934 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-26T08:08:39.934 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-03-26T08:08:39.934 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-26T08:08:39.934 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-03-26T08:08:39.934 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-26T08:08:39.934 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-03-26T08:08:39.934 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-26T08:08:39.934 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-26T08:08:39.934 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-03-26T08:08:39.934 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-26T08:08:39.934 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-26T08:08:39.934 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-26T08:08:39.934 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-26T08:08:39.934 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-26T08:08:39.934 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-03-26T08:08:39.935 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-26T08:08:39.935 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-03-26T08:08:39.935 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-26T08:08:39.935 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-26T08:08:39.935 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-26T08:08:39.935 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-26T08:08:39.935 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-03-26T08:08:39.935 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-26T08:08:39.935 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-03-26T08:08:39.935 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-26T08:08:39.935 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-03-26T08:08:39.935 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-26T08:08:39.935 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-03-26T08:08:39.935 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-26T08:08:39.935 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-26T08:08:39.936 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-03-26T08:08:39.936 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-26T08:08:39.936 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-26T08:08:39.936 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-26T08:08:39.936 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-26T08:08:39.936 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-03-26T08:08:39.936 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-26T08:08:39.936 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-26T08:08:39.936 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-26T08:08:39.936 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-03-26T08:08:39.936 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-26T08:08:39.936 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-26T08:08:39.936 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-26T08:08:39.936 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-26T08:08:39.936 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-03-26T08:08:39.937 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-26T08:08:39.937 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-03-26T08:08:39.937 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-26T08:08:39.937 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-03-26T08:08:39.937 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-26T08:08:39.937 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-26T08:08:39.937 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-03-26T08:08:39.937 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-26T08:08:39.937 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-26T08:08:39.937 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-03-26T08:08:39.937 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-26T08:08:39.937 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-26T08:08:39.937 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-03-26T08:08:39.938 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-26T08:08:39.938 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-03-26T08:08:39.938 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-26T08:08:39.938 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-26T08:08:39.938 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-03-26T08:08:39.938 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-26T08:08:39.938 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-26T08:08:39.938 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-26T08:08:39.938 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-03-26T08:08:39.938 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-26T08:08:39.938 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-26T08:08:39.938 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-26T08:08:39.938 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-26T08:08:39.938 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-26T08:08:39.938 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-03-26T08:08:39.939 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-26T08:08:39.939 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-26T08:08:39.939 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-03-26T08:08:39.939 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-26T08:08:39.939 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-26T08:08:39.939 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-03-26T08:08:39.939 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-26T08:08:39.939 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-26T08:08:39.939 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-26T08:08:39.939 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-26T08:08:39.939 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-26T08:08:39.939 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-26T08:08:39.939 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-26T08:08:39.939 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-26T08:08:39.940 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-03-26T08:08:39.940 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-26T08:08:39.940 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-26T08:08:39.940 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-26T08:08:39.940 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-26T08:08:39.940 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-26T08:08:39.940 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-26T08:08:39.940 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-26T08:08:39.940 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-26T08:08:39.940 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-26T08:08:39.940 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-26T08:08:39.940 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-26T08:08:39.941 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-26T08:08:39.941 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-03-26T08:08:39.941 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-26T08:08:39.941 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-26T08:08:39.941 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-26T08:08:39.941 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-26T08:08:39.941 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-26T08:08:39.941 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-26T08:08:39.941 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-26T08:08:39.941 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-26T08:08:39.941 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-26T08:08:39.941 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-03-26T08:08:39.941 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-03-26T08:08:39.941 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-26T08:08:39.941 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-26T08:08:39.942 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-26T08:08:39.942 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-03-26T08:08:39.942 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-26T08:08:39.942 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-26T08:08:39.942 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-26T08:08:39.942 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-26T08:08:39.942 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-26T08:08:39.942 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-26T08:08:39.942 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-03-26T08:08:39.942 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-26T08:08:39.942 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-03-26T08:08:39.942 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-03-26T08:08:39.942 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-26T08:08:39.943 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-03-26T08:08:39.943 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-26T08:08:39.943 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-03-26T08:08:39.943 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-26T08:08:39.943 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-03-26T08:08:39.943 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-03-26T08:08:39.943 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-26T08:08:39.943 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-03-26T08:08:39.943 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-03-26T08:08:39.943 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-26T08:08:39.943 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-26T08:08:39.943 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-03-26T08:08:39.943 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-26T08:08:39.943 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-26T08:08:39.943 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-26T08:08:39.944 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-26T08:08:39.944 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-03-26T08:08:39.944 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-03-26T08:08:39.944 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-26T08:08:39.944 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-03-26T08:08:39.944 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-26T08:08:39.944 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-26T08:08:39.944 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-03-26T08:08:39.944 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-26T08:08:39.944 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-26T08:08:39.944 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-26T08:08:39.944 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-03-26T08:08:39.944 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-03-26T08:08:39.944 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-26T08:08:39.944 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-03-26T08:08:39.945 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-03-26T08:08:39.945 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-26T08:08:39.945 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-26T08:08:39.945 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-26T08:08:39.945 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-26T08:08:39.945 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-26T08:08:39.945 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-26T08:08:39.945 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-26T08:08:39.945 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-26T08:08:39.945 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-26T08:08:39.945 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-26T08:08:39.945 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-26T08:08:39.945 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-26T08:08:39.945 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-03-26T08:08:39.946 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-26T08:08:39.946 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-26T08:08:39.946 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-26T08:08:39.946 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-26T08:08:39.946 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-26T08:08:39.946 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-26T08:08:39.946 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-26T08:08:39.946 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-26T08:08:39.946 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-26T08:08:39.946 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-26T08:08:39.946 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-26T08:08:39.946 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-26T08:08:39.946 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-03-26T08:08:39.946 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-26T08:08:39.947 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-26T08:08:39.947 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-03-26T08:08:39.947 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-03-26T08:08:39.947 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-26T08:08:39.947 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-26T08:08:39.947 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-26T08:08:39.947 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-26T08:08:39.947 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-03-26T08:08:39.947 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-26T08:08:39.947 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-26T08:08:39.947 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-03-26T08:08:39.947 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-26T08:08:39.947 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-26T08:08:39.947 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-26T08:08:39.947 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-26T08:08:39.948 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-26T08:08:39.948 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-26T08:08:39.948 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-26T08:08:39.948 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-26T08:08:39.948 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-26T08:08:39.948 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-26T08:08:39.948 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-26T08:08:39.948 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-26T08:08:39.948 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-26T08:08:39.948 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-26T08:08:39.948 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-26T08:08:39.949 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-26T08:08:39.949 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-26T08:08:39.949 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-26T08:08:39.949 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-26T08:08:39.949 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-26T08:08:39.949 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-26T08:08:39.949 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-26T08:08:39.949 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-26T08:08:39.949 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-26T08:08:39.949 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-26T08:08:39.949 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-26T08:08:39.949 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-26T08:08:39.949 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-26T08:08:39.949 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-26T08:08:39.949 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-26T08:08:39.950 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-26T08:08:39.950 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-03-26T08:08:39.950 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-26T08:08:39.950 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-26T08:08:39.950 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-26T08:08:39.950 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-26T08:08:39.950 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-26T08:08:39.950 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-03-26T08:08:39.950 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-26T08:08:39.950 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-26T08:08:39.950 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-03-26T08:08:39.950 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-26T08:08:39.950 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-26T08:08:39.950 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-26T08:08:39.951 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-03-26T08:08:39.951 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-26T08:08:39.951 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-26T08:08:39.951 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-03-26T08:08:39.951 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-26T08:08:39.951 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-03-26T08:08:39.951 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-26T08:08:39.951 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-03-26T08:08:39.951 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-26T08:08:39.951 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-03-26T08:08:39.951 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-26T08:08:39.951 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-26T08:08:39.951 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-03-26T08:08:39.951 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-26T08:08:39.951 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-03-26T08:08:39.952 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-26T08:08:39.952 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-26T08:08:39.952 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-26T08:08:39.952 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-26T08:08:39.952 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-26T08:08:39.952 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-26T08:08:39.952 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-03-26T08:08:39.952 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-26T08:08:39.952 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-26T08:08:39.952 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-26T08:08:39.952 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-26T08:08:39.952 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-26T08:08:39.952 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-03-26T08:08:39.952 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-26T08:08:39.952 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-26T08:08:39.953 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-03-26T08:08:39.953 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-26T08:08:39.953 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-26T08:08:39.953 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-03-26T08:08:39.953 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-26T08:08:39.953 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-26T08:08:39.953 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-26T08:08:39.953 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-26T08:08:39.953 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-26T08:08:39.953 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-26T08:08:39.953 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-26T08:08:39.953 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-03-26T08:08:39.953 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-26T08:08:39.954 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-26T08:08:39.954 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-26T08:08:39.954 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-03-26T08:08:39.954 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-26T08:08:39.954 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-26T08:08:39.954 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-26T08:08:39.954 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-03-26T08:08:39.954 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-26T08:08:39.954 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-03-26T08:08:39.954 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-26T08:08:39.954 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-03-26T08:08:39.954 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-26T08:08:39.954 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-26T08:08:39.954 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-26T08:08:39.955 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-26T08:08:39.955 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-03-26T08:08:39.955 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-26T08:08:39.955 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-03-26T08:08:39.955 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-26T08:08:39.955 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-26T08:08:39.955 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-03-26T08:08:39.955 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-26T08:08:39.955 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-26T08:08:39.955 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-26T08:08:39.955 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-03-26T08:08:39.955 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-26T08:08:39.955 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-26T08:08:39.955 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-03-26T08:08:39.955 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-26T08:08:39.956 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-26T08:08:39.956 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-26T08:08:39.956 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-26T08:08:39.956 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-26T08:08:39.956 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-26T08:08:39.956 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-26T08:08:39.956 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-26T08:08:39.956 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-26T08:08:39.956 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-26T08:08:39.956 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-26T08:08:39.956 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-03-26T08:08:39.956 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-26T08:08:39.956 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-26T08:08:39.956 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-26T08:08:39.956 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-26T08:08:39.957 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-03-26T08:08:39.957 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-26T08:08:39.957 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-03-26T08:08:39.957 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-03-26T08:08:39.957 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-26T08:08:39.957 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-03-26T08:08:39.957 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-26T08:08:39.957 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-26T08:08:39.957 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-26T08:08:39.957 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-26T08:08:39.957 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-26T08:08:39.957 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-26T08:08:39.957 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-03-26T08:08:39.957 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-26T08:08:39.957 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-03-26T08:08:39.958 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-03-26T08:08:39.958 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-03-26T08:08:39.958 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-26T08:08:39.958 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-26T08:08:39.958 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-26T08:08:39.958 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-26T08:08:39.958 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-26T08:08:39.958 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-26T08:08:39.958 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-26T08:08:39.958 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-26T08:08:39.958 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-26T08:08:39.958 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-26T08:08:39.958 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-26T08:08:39.958 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-26T08:08:39.958 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-26T08:08:39.959 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-26T08:08:39.959 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-26T08:08:39.959 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-26T08:08:39.959 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-26T08:08:39.959 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-26T08:08:39.959 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-03-26T08:08:39.959 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-03-26T08:08:39.959 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-26T08:08:39.959 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-26T08:08:39.959 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-26T08:08:39.959 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-26T08:08:39.959 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-03-26T08:08:39.959 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-03-26T08:08:39.959 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-26T08:08:39.959 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-26T08:08:39.960 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-03-26T08:08:39.960 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-26T08:08:39.960 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-03-26T08:08:39.960 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-03-26T08:08:39.960 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-26T08:08:39.960 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-03-26T08:08:39.960 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-26T08:08:39.960 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-03-26T08:08:39.960 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-26T08:08:39.960 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-26T08:08:39.960 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-26T08:08:39.960 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-26T08:08:39.960 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-26T08:08:39.960 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-03-26T08:08:39.960 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-03-26T08:08:39.961 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-26T08:08:39.961 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-26T08:08:39.961 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-26T08:08:39.961 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-26T08:08:39.961 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-03-26T08:08:39.961 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-26T08:08:39.961 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-03-26T08:08:39.961 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-26T08:08:39.961 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-26T08:08:39.961 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-03-26T08:08:39.961 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-26T08:08:39.961 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-26T08:08:39.961 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-26T08:08:39.961 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-26T08:08:39.961 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-26T08:08:39.962 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-26T08:08:39.962 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-26T08:08:39.962 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-26T08:08:39.962 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-26T08:08:39.962 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-26T08:08:39.962 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-03-26T08:08:39.962 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-26T08:08:39.962 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-26T08:08:39.962 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-26T08:08:39.962 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-26T08:08:39.962 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-26T08:08:39.962 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-26T08:08:39.962 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-26T08:08:39.962 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-03-26T08:08:39.962 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-26T08:08:39.963 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-03-26T08:08:39.963 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-26T08:08:39.963 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-26T08:08:39.963 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-26T08:08:39.963 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-26T08:08:39.963 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-26T08:08:39.963 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-26T08:08:39.963 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-26T08:08:39.963 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-26T08:08:39.963 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-26T08:08:39.963 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-26T08:08:39.963 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-26T08:08:39.963 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-26T08:08:39.963 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-26T08:08:39.963 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-26T08:08:39.964 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-26T08:08:39.964 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-26T08:08:39.964 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-26T08:08:39.964 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-26T08:08:39.964 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-26T08:08:39.964 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-26T08:08:39.964 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-26T08:08:39.964 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-26T08:08:39.964 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-26T08:08:39.964 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-26T08:08:39.964 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-26T08:08:39.964 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-26T08:08:39.964 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-26T08:08:39.964 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-26T08:08:39.964 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-26T08:08:39.965 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-26T08:08:39.965 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-26T08:08:39.965 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-26T08:08:39.965 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-26T08:08:39.965 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-26T08:08:39.965 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-26T08:08:39.965 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-26T08:08:39.965 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-26T08:08:39.965 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-26T08:08:39.965 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-26T08:08:39.965 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-26T08:08:39.965 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-26T08:08:39.965 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-26T08:08:39.965 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-26T08:08:39.965 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-26T08:08:39.966 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-26T08:08:39.966 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-03-26T08:08:39.966 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-26T08:08:39.966 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-26T08:08:39.966 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-26T08:08:39.966 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-26T08:08:39.966 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-26T08:08:39.966 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-26T08:08:39.966 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-26T08:08:39.966 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-26T08:08:39.966 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-26T08:08:39.966 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-26T08:08:39.966 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-26T08:08:39.967 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-26T08:08:39.967 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-26T08:08:39.967 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-26T08:08:39.967 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-26T08:08:39.967 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-03-26T08:08:39.967 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-26T08:08:39.967 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-26T08:08:39.967 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-26T08:08:39.967 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-03-26T08:08:39.967 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-03-26T08:08:39.967 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-26T08:08:39.967 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-26T08:08:39.967 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-26T08:08:39.968 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-26T08:08:39.968 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-03-26T08:08:39.968 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-26T08:08:39.968 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-26T08:08:39.968 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-26T08:08:39.968 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-26T08:08:39.968 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-03-26T08:08:39.968 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-03-26T08:08:39.968 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-26T08:08:39.968 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-26T08:08:39.968 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-26T08:08:39.968 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-03-26T08:08:39.968 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-03-26T08:08:39.969 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-03-26T08:08:39.969 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-03-26T08:08:39.969 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-03-26T08:08:39.969 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-26T08:08:39.969 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-26T08:08:39.969 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-03-26T08:08:39.969 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-03-26T08:08:39.969 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-03-26T08:08:39.969 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-26T08:08:39.969 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-26T08:08:39.969 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-03-26T08:08:39.969 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-26T08:08:39.969 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-26T08:08:39.969 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-26T08:08:39.969 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-26T08:08:39.970 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-26T08:08:39.970 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-26T08:08:39.970 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-26T08:08:39.970 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-26T08:08:39.970 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-26T08:08:39.970 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-26T08:08:39.970 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-26T08:08:39.970 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-26T08:08:39.970 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-03-26T08:08:39.970 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-26T08:08:39.970 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-26T08:08:39.970 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-26T08:08:39.970 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-26T08:08:39.970 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-26T08:08:39.971 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-26T08:08:39.971 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-26T08:08:39.971 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-26T08:08:39.971 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-26T08:08:39.971 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-26T08:08:39.971 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-26T08:08:39.971 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-26T08:08:39.971 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-26T08:08:39.971 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-26T08:08:39.971 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-26T08:08:39.971 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-26T08:08:39.971 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-26T08:08:39.971 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-26T08:08:39.971 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-26T08:08:39.971 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-26T08:08:39.971 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-26T08:08:39.972 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-26T08:08:39.972 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-26T08:08:39.972 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-26T08:08:39.972 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-26T08:08:39.972 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-26T08:08:39.972 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-03-26T08:08:39.972 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-26T08:08:39.972 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-26T08:08:39.972 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-26T08:08:39.972 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-26T08:08:39.972 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-03-26T08:08:39.972 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-26T08:08:39.972 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-26T08:08:39.972 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-03-26T08:08:39.972 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-26T08:08:39.973 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-03-26T08:08:39.973 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-03-26T08:08:39.973 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-26T08:08:39.973 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-03-26T08:08:39.973 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-26T08:08:39.973 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-03-26T08:08:39.973 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-26T08:08:39.973 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-26T08:08:39.973 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-03-26T08:08:39.973 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-26T08:08:39.973 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-03-26T08:08:39.973 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-26T08:08:39.973 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-03-26T08:08:39.973 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-26T08:08:39.973 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-03-26T08:08:39.974 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-26T08:08:39.974 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-03-26T08:08:39.974 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-26T08:08:39.974 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-03-26T08:08:39.974 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-26T08:08:39.974 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-03-26T08:08:39.974 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-26T08:08:39.974 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-03-26T08:08:39.974 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-26T08:08:39.974 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-03-26T08:08:39.974 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-03-26T08:08:39.974 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-26T08:08:39.974 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-03-26T08:08:39.974 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-03-26T08:08:39.975 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-26T08:08:39.975 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-03-26T08:08:39.975 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-03-26T08:08:39.975 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-26T08:08:39.975 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-03-26T08:08:39.975 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-26T08:08:39.975 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-03-26T08:08:39.975 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-26T08:08:39.975 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-03-26T08:08:39.975 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-26T08:08:39.975 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-03-26T08:08:39.975 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-26T08:08:39.975 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-03-26T08:08:39.975 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-26T08:08:39.975 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-03-26T08:08:39.976 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-26T08:08:39.976 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-03-26T08:08:39.976 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-26T08:08:39.976 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-03-26T08:08:39.976 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-03-26T08:08:39.976 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-26T08:08:39.976 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-03-26T08:08:39.976 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-26T08:08:39.976 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-03-26T08:08:39.976 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-26T08:08:39.976 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-26T08:08:39.976 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-26T08:08:39.976 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-26T08:08:39.976 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-03-26T08:08:39.976 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-26T08:08:39.977 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-26T08:08:39.977 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-03-26T08:08:39.977 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-26T08:08:39.977 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-03-26T08:08:39.977 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-26T08:08:39.977 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-03-26T08:08:39.977 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-26T08:08:39.977 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-26T08:08:39.977 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-26T08:08:39.977 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-26T08:08:39.977 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-26T08:08:39.977 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-26T08:08:39.977 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-26T08:08:39.977 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-26T08:08:39.977 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-26T08:08:39.978 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-26T08:08:39.978 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-26T08:08:39.978 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-26T08:08:39.978 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-26T08:08:39.978 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-26T08:08:39.978 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-26T08:08:39.978 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-26T08:08:39.978 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-26T08:08:39.978 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-26T08:08:39.978 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-26T08:08:39.978 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-26T08:08:39.978 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-26T08:08:39.978 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-26T08:08:39.978 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-26T08:08:39.979 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-26T08:08:39.979 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-26T08:08:39.979 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-26T08:08:39.979 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-26T08:08:39.979 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-26T08:08:39.979 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-26T08:08:39.979 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-26T08:08:39.979 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-26T08:08:39.979 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-26T08:08:39.979 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-26T08:08:39.979 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-26T08:08:39.979 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-26T08:08:39.979 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-26T08:08:39.979 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-26T08:08:39.979 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-26T08:08:39.979 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-26T08:08:39.980 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-26T08:08:39.980 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-26T08:08:39.980 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-26T08:08:39.980 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-26T08:08:39.980 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-26T08:08:39.980 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-26T08:08:39.980 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-26T08:08:39.980 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-26T08:08:39.980 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-26T08:08:39.980 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-26T08:08:39.980 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-26T08:08:39.980 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-03-26T08:08:39.980 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-03-26T08:08:39.980 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-26T08:08:39.980 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-03-26T08:08:39.981 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-26T08:08:39.981 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-03-26T08:08:39.981 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-26T08:08:39.981 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-26T08:08:39.981 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-26T08:08:39.981 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-26T08:08:39.981 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-26T08:08:39.981 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-03-26T08:08:39.981 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-26T08:08:39.981 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-03-26T08:08:39.981 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-26T08:08:39.981 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-03-26T08:08:39.981 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-26T08:08:39.981 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-03-26T08:08:39.981 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-26T08:08:39.982 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-03-26T08:08:39.982 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-03-26T08:08:39.982 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-26T08:08:39.982 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-03-26T08:08:39.982 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-26T08:08:39.982 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-26T08:08:39.982 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-03-26T08:08:39.982 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-26T08:08:39.982 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-03-26T08:08:39.982 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-26T08:08:39.982 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-03-26T08:08:39.982 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-26T08:08:39.982 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-03-26T08:08:39.982 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-26T08:08:39.983 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-03-26T08:08:39.983 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-26T08:08:39.983 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-03-26T08:08:39.983 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-03-26T08:08:39.983 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-03-26T08:08:39.983 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-26T08:08:39.983 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-03-26T08:08:39.983 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-26T08:08:39.983 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-03-26T08:08:39.983 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-03-26T08:08:39.983 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-26T08:08:39.984 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-26T08:08:39.984 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-03-26T08:08:39.984 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-26T08:08:39.984 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-26T08:08:39.984 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-26T08:08:39.984 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-26T08:08:39.984 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-03-26T08:08:39.984 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-26T08:08:39.984 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-26T08:08:39.984 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-03-26T08:08:39.984 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-26T08:08:39.984 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-03-26T08:08:39.984 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-26T08:08:39.984 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-03-26T08:08:39.985 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-26T08:08:39.985 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-26T08:08:39.985 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-26T08:08:39.985 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-26T08:08:39.985 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-03-26T08:08:39.985 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-03-26T08:08:39.985 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-03-26T08:08:39.985 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-26T08:08:39.985 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-03-26T08:08:39.985 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-03-26T08:08:39.985 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-26T08:08:39.985 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-26T08:08:39.985 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-26T08:08:39.985 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-26T08:08:39.986 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-26T08:08:39.986 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-26T08:08:39.986 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-26T08:08:39.986 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-26T08:08:39.986 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-03-26T08:08:39.986 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-26T08:08:39.986 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-26T08:08:39.986 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-26T08:08:39.986 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-26T08:08:39.986 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-26T08:08:39.986 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-03-26T08:08:39.986 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-26T08:08:39.986 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-03-26T08:08:39.986 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-26T08:08:39.987 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-26T08:08:39.987 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-26T08:08:39.987 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-26T08:08:39.987 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-03-26T08:08:39.987 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-26T08:08:39.987 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-03-26T08:08:39.987 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-26T08:08:39.987 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-26T08:08:39.987 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-03-26T08:08:39.987 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-26T08:08:39.987 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-26T08:08:39.987 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-26T08:08:39.987 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-26T08:08:39.987 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-26T08:08:39.988 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-26T08:08:39.988 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-26T08:08:39.988 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-26T08:08:39.988 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-26T08:08:39.988 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-26T08:08:39.988 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-26T08:08:39.988 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-26T08:08:39.988 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-26T08:08:39.988 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-26T08:08:39.988 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-26T08:08:39.988 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-26T08:08:39.988 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-26T08:08:39.988 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-26T08:08:39.988 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-26T08:08:39.988 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-26T08:08:39.989 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-26T08:08:39.989 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-26T08:08:39.989 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-26T08:08:39.989 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-26T08:08:39.989 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-03-26T08:08:39.989 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-26T08:08:39.989 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-03-26T08:08:39.989 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-03-26T08:08:39.989 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-26T08:08:39.989 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-03-26T08:08:39.989 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-26T08:08:39.989 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-26T08:08:39.989 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-26T08:08:39.989 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-26T08:08:39.989 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-03-26T08:08:39.990 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-03-26T08:08:39.990 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-03-26T08:08:39.990 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-26T08:08:39.990 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-26T08:08:39.990 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-26T08:08:39.990 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-26T08:08:39.990 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-26T08:08:39.990 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-03-26T08:08:39.990 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-03-26T08:08:39.990 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-26T08:08:39.990 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-26T08:08:39.990 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-26T08:08:39.990 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-03-26T08:08:39.991 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-26T08:08:39.991 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-26T08:08:39.991 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-26T08:08:39.991 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-26T08:08:39.991 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-26T08:08:39.991 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-26T08:08:39.991 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-26T08:08:39.991 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-26T08:08:39.991 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-26T08:08:39.991 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-26T08:08:39.991 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-03-26T08:08:39.991 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-26T08:08:39.991 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-26T08:08:39.991 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-26T08:08:39.991 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-26T08:08:39.992 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-26T08:08:39.992 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-26T08:08:39.992 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-26T08:08:39.992 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-26T08:08:39.992 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-26T08:08:39.992 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-26T08:08:39.992 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-26T08:08:39.992 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-26T08:08:39.992 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-26T08:08:39.992 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-26T08:08:39.992 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-26T08:08:39.992 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-26T08:08:39.992 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-26T08:08:39.992 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-26T08:08:39.993 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-26T08:08:39.993 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-26T08:08:39.993 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-26T08:08:39.993 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-26T08:08:39.993 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-26T08:08:39.993 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-26T08:08:39.993 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-26T08:08:39.993 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-26T08:08:39.993 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-26T08:08:39.993 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-26T08:08:39.993 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-26T08:08:39.993 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-26T08:08:39.993 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-26T08:08:39.993 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-26T08:08:39.994 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-26T08:08:39.994 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-26T08:08:39.994 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-26T08:08:39.994 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-26T08:08:39.994 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-26T08:08:39.994 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-26T08:08:39.994 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-03-26T08:08:39.994 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-26T08:08:39.994 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-26T08:08:39.994 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-26T08:08:39.994 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-26T08:08:39.994 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-26T08:08:39.995 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-26T08:08:39.995 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-26T08:08:39.995 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-03-26T08:08:39.995 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-26T08:08:39.995 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-26T08:08:39.995 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-26T08:08:39.995 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-26T08:08:39.995 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-26T08:08:39.995 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-26T08:08:39.995 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-26T08:08:39.996 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-26T08:08:39.996 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-26T08:08:39.996 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-26T08:08:39.996 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-26T08:08:39.996 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-26T08:08:39.996 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-26T08:08:39.996 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-26T08:08:39.996 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-26T08:08:39.996 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-26T08:08:39.996 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-26T08:08:39.996 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-26T08:08:39.996 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-26T08:08:39.996 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-26T08:08:39.996 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-26T08:08:39.997 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-26T08:08:39.997 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-26T08:08:39.997 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-26T08:08:39.997 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-26T08:08:39.997 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-26T08:08:39.997 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-26T08:08:39.997 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-03-26T08:08:39.997 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-26T08:08:39.997 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-03-26T08:08:39.997 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-26T08:08:39.997 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-26T08:08:39.997 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-26T08:08:39.997 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-26T08:08:39.998 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-26T08:08:39.998 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-26T08:08:39.998 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-26T08:08:39.998 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-03-26T08:08:39.998 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-26T08:08:39.998 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-03-26T08:08:39.998 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-26T08:08:39.998 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-03-26T08:08:39.998 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-26T08:08:39.998 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-03-26T08:08:39.998 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-03-26T08:08:39.998 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-26T08:08:39.998 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-26T08:08:39.998 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-26T08:08:39.999 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-26T08:08:39.999 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-26T08:08:39.999 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-26T08:08:39.999 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-26T08:08:39.999 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-26T08:08:39.999 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-26T08:08:39.999 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-26T08:08:39.999 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-26T08:08:39.999 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-26T08:08:39.999 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-26T08:08:39.999 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-26T08:08:39.999 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-26T08:08:39.999 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-26T08:08:39.999 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-03-26T08:08:39.999 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-26T08:08:40.000 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-26T08:08:40.000 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-26T08:08:40.000 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-26T08:08:40.000 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-26T08:08:40.000 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-26T08:08:40.000 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-26T08:08:40.000 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-26T08:08:40.000 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-26T08:08:40.000 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-26T08:08:40.000 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-26T08:08:40.000 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-26T08:08:40.000 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-26T08:08:40.000 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-26T08:08:40.001 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-26T08:08:40.001 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-26T08:08:40.001 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-26T08:08:40.001 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-26T08:08:40.001 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-26T08:08:40.001 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-26T08:08:40.001 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-26T08:08:40.001 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-26T08:08:40.001 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-26T08:08:40.001 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-26T08:08:40.001 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-26T08:08:40.001 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-26T08:08:40.001 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-26T08:08:40.001 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-26T08:08:40.002 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-26T08:08:40.002 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-26T08:08:40.002 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-26T08:08:40.002 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-26T08:08:40.002 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-26T08:08:40.002 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-26T08:08:40.002 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-26T08:08:40.002 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-26T08:08:40.002 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-26T08:08:40.002 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-26T08:08:40.002 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-26T08:08:40.002 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-26T08:08:40.002 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-26T08:08:40.003 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-26T08:08:40.003 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-26T08:08:40.003 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-26T08:08:40.003 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-26T08:08:40.003 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-26T08:08:40.003 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-26T08:08:40.003 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-26T08:08:40.003 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-26T08:08:40.003 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-26T08:08:40.003 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-26T08:08:40.003 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-26T08:08:40.003 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-26T08:08:40.003 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-26T08:08:40.003 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-26T08:08:40.004 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-26T08:08:40.004 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-26T08:08:40.004 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-26T08:08:40.004 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-03-26T08:08:40.004 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-26T08:08:40.004 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-26T08:08:40.004 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-26T08:08:40.004 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-26T08:08:40.004 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-03-26T08:08:40.004 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-26T08:08:40.004 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-26T08:08:40.004 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-26T08:08:40.004 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-26T08:08:40.004 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-26T08:08:40.004 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-26T08:08:40.005 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-26T08:08:40.005 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-26T08:08:40.005 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-26T08:08:40.005 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-26T08:08:40.005 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-26T08:08:40.005 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-26T08:08:40.005 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-26T08:08:40.005 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-26T08:08:40.005 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-26T08:08:40.005 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-26T08:08:40.005 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-26T08:08:40.005 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-26T08:08:40.005 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-26T08:08:40.005 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-26T08:08:40.005 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-26T08:08:40.006 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-26T08:08:40.006 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-26T08:08:40.006 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-26T08:08:40.006 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-26T08:08:40.006 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-26T08:08:40.006 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-26T08:08:40.006 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-26T08:08:40.006 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-26T08:08:40.006 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-26T08:08:40.006 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-26T08:08:40.006 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-26T08:08:40.006 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-26T08:08:40.006 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-26T08:08:40.006 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-26T08:08:40.007 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-26T08:08:40.007 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-26T08:08:40.007 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-26T08:08:40.007 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-26T08:08:40.007 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-26T08:08:40.007 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-26T08:08:40.007 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-26T08:08:40.007 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-26T08:08:40.007 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-26T08:08:40.007 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-26T08:08:40.007 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-26T08:08:40.007 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-26T08:08:40.007 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-26T08:08:40.007 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-26T08:08:40.008 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-26T08:08:40.008 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-26T08:08:40.008 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-26T08:08:40.008 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-26T08:08:40.008 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-26T08:08:40.008 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-26T08:08:40.008 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-26T08:08:40.008 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-26T08:08:40.008 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-26T08:08:40.008 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-26T08:08:40.008 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-26T08:08:40.008 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-26T08:08:40.008 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-26T08:08:40.008 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-26T08:08:40.008 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-26T08:08:40.009 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-26T08:08:40.009 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-26T08:08:40.009 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-26T08:08:40.009 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-26T08:08:40.009 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-26T08:08:40.009 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-26T08:08:40.009 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-26T08:08:40.009 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-03-26T08:08:40.009 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-26T08:08:40.009 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-03-26T08:08:40.009 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-26T08:08:40.009 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-03-26T08:08:40.009 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-26T08:08:40.009 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-26T08:08:40.009 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-26T08:08:40.010 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-26T08:08:40.010 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-03-26T08:08:40.010 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-26T08:08:40.010 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-03-26T08:08:40.010 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-26T08:08:40.010 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-26T08:08:40.010 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-26T08:08:40.010 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-03-26T08:08:40.010 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-26T08:08:40.010 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-26T08:08:40.010 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-26T08:08:40.010 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-26T08:08:40.010 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-26T08:08:40.010 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-26T08:08:40.011 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-26T08:08:40.011 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-26T08:08:40.011 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-26T08:08:40.011 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-26T08:08:40.011 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-26T08:08:40.011 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-26T08:08:40.011 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-26T08:08:40.011 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-26T08:08:40.011 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-26T08:08:40.011 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-26T08:08:40.011 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-26T08:08:40.011 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-26T08:08:40.011 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-26T08:08:40.011 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-26T08:08:40.012 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-26T08:08:40.012 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-26T08:08:40.012 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-26T08:08:40.012 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-26T08:08:40.012 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-26T08:08:40.012 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-26T08:08:40.012 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-26T08:08:40.012 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-26T08:08:40.012 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-26T08:08:40.012 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-26T08:08:40.012 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-26T08:08:40.012 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-26T08:08:40.012 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-26T08:08:40.012 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-26T08:08:40.012 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-26T08:08:40.013 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-26T08:08:40.013 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-26T08:08:40.013 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-26T08:08:40.013 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-26T08:08:40.013 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-26T08:08:40.013 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-26T08:08:40.013 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-26T08:08:40.013 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-26T08:08:40.013 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-03-26T08:08:40.013 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-26T08:08:40.013 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-26T08:08:40.013 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-03-26T08:08:40.013 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-03-26T08:08:40.013 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-26T08:08:40.013 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-26T08:08:40.014 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-26T08:08:40.014 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-03-26T08:08:40.014 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-26T08:08:40.014 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-03-26T08:08:40.014 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-26T08:08:40.014 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-26T08:08:40.015 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-03-26T08:08:40.015 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-26T08:08:40.015 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-03-26T08:08:40.015 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-26T08:08:40.015 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-26T08:08:40.015 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-26T08:08:40.015 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-26T08:08:40.016 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-26T08:08:40.016 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-03-26T08:08:40.016 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-26T08:08:40.016 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-03-26T08:08:40.016 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-03-26T08:08:40.016 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-03-26T08:08:40.016 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-26T08:08:40.016 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-03-26T08:08:40.016 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-03-26T08:08:40.016 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-03-26T08:08:40.016 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-03-26T08:08:40.016 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-26T08:08:40.016 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-26T08:08:40.016 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-26T08:08:40.016 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-26T08:08:40.017 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-03-26T08:08:40.017 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-26T08:08:40.017 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-26T08:08:40.017 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-26T08:08:40.017 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-26T08:08:40.017 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-26T08:08:40.017 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-26T08:08:40.017 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-26T08:08:40.017 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-26T08:08:40.017 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-26T08:08:40.017 INFO:teuthology.orchestra.run.smithi183.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-26T08:08:40.021 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-26T08:08:40.021 DEBUG:teuthology.orchestra.run.smithi183:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-26T08:08:40.022 INFO:teuthology.orchestra.run.smithi043.stdout:Name : grub2-tools 2024-03-26T08:08:40.023 INFO:teuthology.orchestra.run.smithi043.stdout:Epoch : 1 2024-03-26T08:08:40.023 INFO:teuthology.orchestra.run.smithi043.stdout:Version : 2.06 2024-03-26T08:08:40.023 INFO:teuthology.orchestra.run.smithi043.stdout:Release : 68.el9 2024-03-26T08:08:40.023 INFO:teuthology.orchestra.run.smithi043.stdout:Architecture: x86_64 2024-03-26T08:08:40.023 INFO:teuthology.orchestra.run.smithi043.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-26T08:08:40.023 INFO:teuthology.orchestra.run.smithi043.stdout:Group : Unspecified 2024-03-26T08:08:40.023 INFO:teuthology.orchestra.run.smithi043.stdout:Size : 8264254 2024-03-26T08:08:40.023 INFO:teuthology.orchestra.run.smithi043.stdout:License : GPLv3+ 2024-03-26T08:08:40.023 INFO:teuthology.orchestra.run.smithi043.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-26T08:08:40.023 INFO:teuthology.orchestra.run.smithi043.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-26T08:08:40.023 INFO:teuthology.orchestra.run.smithi043.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-26T08:08:40.023 INFO:teuthology.orchestra.run.smithi043.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-26T08:08:40.024 INFO:teuthology.orchestra.run.smithi043.stdout:Packager : builder@centos.org 2024-03-26T08:08:40.024 INFO:teuthology.orchestra.run.smithi043.stdout:Vendor : CentOS 2024-03-26T08:08:40.024 INFO:teuthology.orchestra.run.smithi043.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-26T08:08:40.024 INFO:teuthology.orchestra.run.smithi043.stdout:Summary : Support tools for GRUB. 2024-03-26T08:08:40.024 INFO:teuthology.orchestra.run.smithi043.stdout:Description : 2024-03-26T08:08:40.024 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:08:40.024 INFO:teuthology.orchestra.run.smithi043.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-26T08:08:40.024 INFO:teuthology.orchestra.run.smithi043.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-26T08:08:40.024 INFO:teuthology.orchestra.run.smithi043.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-26T08:08:40.024 INFO:teuthology.orchestra.run.smithi043.stdout:hardware devices. 2024-03-26T08:08:40.024 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:08:40.025 INFO:teuthology.orchestra.run.smithi043.stdout:This subpackage provides tools for support of all platforms. 2024-03-26T08:08:40.025 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-26T08:08:40.025 INFO:teuthology.task.kernel:Updating grub on smithi043 to boot 6.8.0-g1df0c7e8a235 2024-03-26T08:08:40.025 DEBUG:teuthology.orchestra.run.smithi043:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-26T08:08:40.043 INFO:teuthology.orchestra.run.smithi183.stdout:postinstall scriptlet (using /bin/sh): 2024-03-26T08:08:40.043 INFO:teuthology.orchestra.run.smithi183.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-26T08:08:40.043 INFO:teuthology.orchestra.run.smithi183.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-26T08:08:40.043 INFO:teuthology.orchestra.run.smithi183.stdout:fi 2024-03-26T08:08:40.043 INFO:teuthology.orchestra.run.smithi183.stdout:for file in vmlinuz System.map config; do 2024-03-26T08:08:40.043 INFO:teuthology.orchestra.run.smithi183.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-03-26T08:08:40.043 INFO:teuthology.orchestra.run.smithi183.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-03-26T08:08:40.043 INFO:teuthology.orchestra.run.smithi183.stdout: fi 2024-03-26T08:08:40.043 INFO:teuthology.orchestra.run.smithi183.stdout:done 2024-03-26T08:08:40.043 INFO:teuthology.orchestra.run.smithi183.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-26T08:08:40.044 INFO:teuthology.orchestra.run.smithi183.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-26T08:08:40.044 INFO:teuthology.orchestra.run.smithi183.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-03-26T08:08:40.044 INFO:teuthology.orchestra.run.smithi183.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-26T08:08:40.044 INFO:teuthology.orchestra.run.smithi183.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-03-26T08:08:40.044 INFO:teuthology.orchestra.run.smithi183.stdout:fi 2024-03-26T08:08:40.044 INFO:teuthology.orchestra.run.smithi183.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-26T08:08:40.044 INFO:teuthology.orchestra.run.smithi183.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-26T08:08:40.044 INFO:teuthology.orchestra.run.smithi183.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-03-26T08:08:40.044 INFO:teuthology.orchestra.run.smithi183.stdout:fi 2024-03-26T08:08:40.045 DEBUG:teuthology.orchestra.run.smithi183:> sudo rpm -qi grub2-tools 2024-03-26T08:08:40.119 INFO:teuthology.orchestra.run.smithi183.stdout:Name : grub2-tools 2024-03-26T08:08:40.119 INFO:teuthology.orchestra.run.smithi183.stdout:Epoch : 1 2024-03-26T08:08:40.119 INFO:teuthology.orchestra.run.smithi183.stdout:Version : 2.06 2024-03-26T08:08:40.119 INFO:teuthology.orchestra.run.smithi183.stdout:Release : 68.el9 2024-03-26T08:08:40.120 INFO:teuthology.orchestra.run.smithi183.stdout:Architecture: x86_64 2024-03-26T08:08:40.120 INFO:teuthology.orchestra.run.smithi183.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-26T08:08:40.120 INFO:teuthology.orchestra.run.smithi183.stdout:Group : Unspecified 2024-03-26T08:08:40.120 INFO:teuthology.orchestra.run.smithi183.stdout:Size : 8264254 2024-03-26T08:08:40.120 INFO:teuthology.orchestra.run.smithi183.stdout:License : GPLv3+ 2024-03-26T08:08:40.120 INFO:teuthology.orchestra.run.smithi183.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-26T08:08:40.121 INFO:teuthology.orchestra.run.smithi183.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-26T08:08:40.121 INFO:teuthology.orchestra.run.smithi183.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-26T08:08:40.121 INFO:teuthology.orchestra.run.smithi183.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-26T08:08:40.121 INFO:teuthology.orchestra.run.smithi183.stdout:Packager : builder@centos.org 2024-03-26T08:08:40.121 INFO:teuthology.orchestra.run.smithi183.stdout:Vendor : CentOS 2024-03-26T08:08:40.121 INFO:teuthology.orchestra.run.smithi183.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-26T08:08:40.122 INFO:teuthology.orchestra.run.smithi183.stdout:Summary : Support tools for GRUB. 2024-03-26T08:08:40.122 INFO:teuthology.orchestra.run.smithi183.stdout:Description : 2024-03-26T08:08:40.122 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-26T08:08:40.122 INFO:teuthology.orchestra.run.smithi183.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-26T08:08:40.122 INFO:teuthology.orchestra.run.smithi183.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-26T08:08:40.122 INFO:teuthology.orchestra.run.smithi183.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-26T08:08:40.123 INFO:teuthology.orchestra.run.smithi183.stdout:hardware devices. 2024-03-26T08:08:40.123 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-26T08:08:40.123 INFO:teuthology.orchestra.run.smithi183.stdout:This subpackage provides tools for support of all platforms. 2024-03-26T08:08:40.124 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-26T08:08:40.124 INFO:teuthology.task.kernel:Updating grub on smithi183 to boot 6.8.0-g1df0c7e8a235 2024-03-26T08:08:40.125 DEBUG:teuthology.orchestra.run.smithi183:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-26T08:08:40.655 INFO:teuthology.orchestra.run.smithi043.stderr:Generating grub configuration file ... 2024-03-26T08:08:40.728 INFO:teuthology.orchestra.run.smithi183.stderr:Generating grub configuration file ... 2024-03-26T08:08:42.113 INFO:teuthology.orchestra.run.smithi183.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-26T08:08:42.146 INFO:teuthology.orchestra.run.smithi183.stderr:done 2024-03-26T08:08:42.148 DEBUG:teuthology.orchestra.run.smithi183:> mktemp 2024-03-26T08:08:42.163 INFO:teuthology.orchestra.run.smithi183.stdout:/tmp/tmp.uKWNd4DMQy 2024-03-26T08:08:42.163 DEBUG:teuthology.orchestra.run.smithi183:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.uKWNd4DMQy 2024-03-26T08:08:42.235 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0666 /tmp/tmp.uKWNd4DMQy 2024-03-26T08:08:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-26T08:08:42.309 INFO:teuthology.orchestra.run.smithi043.stderr:done 2024-03-26T08:08:42.311 DEBUG:teuthology.orchestra.run.smithi043:> mktemp 2024-03-26T08:08:42.327 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.7kkrv90MGH 2024-03-26T08:08:42.327 DEBUG:teuthology.orchestra.run.smithi043:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.7kkrv90MGH 2024-03-26T08:08:42.381 DEBUG:teuthology.orchestra.remote:smithi183:/tmp/tmp.uKWNd4DMQy is 6KB 2024-03-26T08:08:42.398 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0666 /tmp/tmp.7kkrv90MGH 2024-03-26T08:08:42.401 DEBUG:teuthology.orchestra.run.smithi183:> rm -fr /tmp/tmp.uKWNd4DMQy 2024-03-26T08:08:42.414 DEBUG:teuthology.orchestra.run.smithi183:> sudo /bin/ls /boot/loader/entries || true 2024-03-26T08:08:42.487 INFO:teuthology.orchestra.run.smithi183.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-26T08:08:42.487 INFO:teuthology.orchestra.run.smithi183.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-26T08:08:42.487 INFO:teuthology.orchestra.run.smithi183.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-26T08:08:42.487 INFO:teuthology.orchestra.run.smithi183.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235.conf 2024-03-26T08:08:42.489 DEBUG:teuthology.orchestra.run.smithi183:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235 2024-03-26T08:08:42.542 DEBUG:teuthology.orchestra.remote:smithi043:/tmp/tmp.7kkrv90MGH is 6KB 2024-03-26T08:08:42.591 DEBUG:teuthology.orchestra.run.smithi043:> rm -fr /tmp/tmp.7kkrv90MGH 2024-03-26T08:08:42.606 DEBUG:teuthology.orchestra.run.smithi043:> sudo /bin/ls /boot/loader/entries || true 2024-03-26T08:08:42.623 DEBUG:teuthology.orchestra.run.smithi183:> sudo shutdown -r now 2024-03-26T08:08:42.674 INFO:teuthology.orchestra.run.smithi043.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-26T08:08:42.674 INFO:teuthology.orchestra.run.smithi043.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-26T08:08:42.674 INFO:teuthology.orchestra.run.smithi043.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-26T08:08:42.674 INFO:teuthology.orchestra.run.smithi043.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235.conf 2024-03-26T08:08:42.675 DEBUG:teuthology.orchestra.run.smithi043:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235 2024-03-26T08:08:42.857 DEBUG:teuthology.orchestra.run.smithi043:> sudo shutdown -r now 2024-03-26T08:08:54.854 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-26T08:08:54.855 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-03-26T08:08:54.856 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T08:09:12.643 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-03-26T08:09:12.644 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-03-26T08:09:12.645 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T08:09:12.860 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-26T08:09:12.860 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-03-26T08:09:12.861 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T08:09:54.900 DEBUG:teuthology.orchestra.remote:timed out 2024-03-26T08:10:03.902 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-03-26T08:10:03.903 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T08:10:04.380 DEBUG:teuthology.orchestra.run.smithi052:> true 2024-03-26T08:10:05.037 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-03-26T08:10:05.037 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.8.0-g1df0c7e8a235"... 2024-03-26T08:10:05.038 DEBUG:teuthology.orchestra.run.smithi052:> uname -r 2024-03-26T08:10:05.083 INFO:teuthology.orchestra.run.smithi052.stdout:6.8.0-g1df0c7e8a235 2024-03-26T08:10:05.083 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-26T08:10:05.083 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-26T08:10:05.083 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-26T08:10:06.086 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-26T08:10:06.086 DEBUG:teuthology.orchestra.run.smithi052:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-26T08:10:06.234 INFO:teuthology.orchestra.run.smithi052.stdout:ttyS1 2024-03-26T08:10:06.293 DEBUG:teuthology.parallel:result is None 2024-03-26T08:10:12.647 DEBUG:teuthology.orchestra.remote:timed out 2024-03-26T08:10:12.863 DEBUG:teuthology.orchestra.remote:timed out 2024-03-26T08:10:21.648 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-03-26T08:10:21.649 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T08:10:21.865 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-03-26T08:10:21.865 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T08:10:22.309 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-03-26T08:10:23.025 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-03-26T08:10:23.025 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.8.0-g1df0c7e8a235"... 2024-03-26T08:10:23.025 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-03-26T08:10:23.082 INFO:teuthology.orchestra.run.smithi043.stdout:6.8.0-g1df0c7e8a235 2024-03-26T08:10:23.082 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-26T08:10:23.082 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-26T08:10:23.082 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-26T08:10:24.083 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-26T08:10:24.083 DEBUG:teuthology.orchestra.run.smithi043:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-26T08:10:24.184 INFO:teuthology.orchestra.run.smithi043.stdout:ttyS1 2024-03-26T08:10:24.229 DEBUG:teuthology.parallel:result is None 2024-03-26T08:10:25.244 DEBUG:teuthology.orchestra.run.smithi183:> true 2024-03-26T08:10:25.918 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-03-26T08:10:25.918 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.8.0-g1df0c7e8a235"... 2024-03-26T08:10:25.918 DEBUG:teuthology.orchestra.run.smithi183:> uname -r 2024-03-26T08:10:25.963 INFO:teuthology.orchestra.run.smithi183.stdout:6.8.0-g1df0c7e8a235 2024-03-26T08:10:25.963 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-26T08:10:25.963 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-26T08:10:25.963 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-26T08:10:26.964 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-03-26T08:10:26.964 DEBUG:teuthology.orchestra.run.smithi183:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-26T08:10:27.156 INFO:teuthology.orchestra.run.smithi183.stdout:ttyS1 2024-03-26T08:10:27.208 DEBUG:teuthology.parallel:result is None 2024-03-26T08:10:27.208 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-26T08:10:27.216 INFO:teuthology.task.internal:Creating test directory... 2024-03-26T08:10:27.217 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-26T08:10:27.223 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-26T08:10:27.232 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-26T08:10:27.287 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-26T08:10:27.294 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-26T08:10:27.321 INFO:teuthology.task.internal:Creating archive directory... 2024-03-26T08:10:27.321 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-26T08:10:27.326 DEBUG:teuthology.orchestra.run.smithi052:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-26T08:10:27.333 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-26T08:10:27.398 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-26T08:10:27.405 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-26T08:10:27.405 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-26T08:10:27.415 DEBUG:teuthology.orchestra.run.smithi052:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-26T08:10:27.422 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-26T08:10:27.489 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T08:10:27.506 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T08:10:27.525 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T08:10:27.531 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T08:10:27.540 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T08:10:27.577 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T08:10:27.582 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-26T08:10:27.590 INFO:teuthology.task.internal:Configuring sudo... 2024-03-26T08:10:27.590 DEBUG:teuthology.orchestra.run.smithi043:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-26T08:10:27.595 DEBUG:teuthology.orchestra.run.smithi052:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-26T08:10:27.600 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-26T08:10:27.694 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-26T08:10:27.704 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-26T08:10:27.704 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-26T08:10:27.708 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-26T08:10:27.714 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-26T08:10:27.763 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T08:10:27.865 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T08:10:27.968 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T08:10:28.037 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T08:10:28.138 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-26T08:10:28.138 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-26T08:10:28.222 DEBUG:teuthology.orchestra.run.smithi052:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T08:10:28.313 DEBUG:teuthology.orchestra.run.smithi052:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T08:10:28.460 DEBUG:teuthology.orchestra.run.smithi052:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T08:10:28.551 DEBUG:teuthology.orchestra.run.smithi052:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T08:10:28.656 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-26T08:10:28.656 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-26T08:10:28.755 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T08:10:28.835 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T08:10:28.914 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T08:10:28.982 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T08:10:29.072 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-03-26T08:10:29.072 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-26T08:10:29.143 DEBUG:teuthology.orchestra.run.smithi043:> sudo service rsyslog restart 2024-03-26T08:10:29.150 DEBUG:teuthology.orchestra.run.smithi052:> sudo service rsyslog restart 2024-03-26T08:10:29.158 DEBUG:teuthology.orchestra.run.smithi183:> sudo service rsyslog restart 2024-03-26T08:10:29.265 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T08:10:29.280 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T08:10:29.288 INFO:teuthology.orchestra.run.smithi052.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T08:10:29.655 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-26T08:10:29.663 INFO:teuthology.task.internal:Starting timer... 2024-03-26T08:10:29.663 INFO:teuthology.run_tasks:Running task pcp... 2024-03-26T08:10:29.698 INFO:teuthology.run_tasks:Running task selinux... 2024-03-26T08:10:29.731 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-26T08:10:29.731 DEBUG:teuthology.orchestra.run.smithi043:> sudo service auditd rotate 2024-03-26T08:10:29.850 INFO:teuthology.orchestra.run.smithi043.stdout:Rotating logs: 2024-03-26T08:10:29.855 DEBUG:teuthology.orchestra.run.smithi052:> sudo service auditd rotate 2024-03-26T08:10:29.992 INFO:teuthology.orchestra.run.smithi052.stdout:Rotating logs: 2024-03-26T08:10:29.998 DEBUG:teuthology.orchestra.run.smithi183:> sudo service auditd rotate 2024-03-26T08:10:30.108 INFO:teuthology.orchestra.run.smithi183.stdout:Rotating logs: 2024-03-26T08:10:30.111 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-26T08:10:30.111 DEBUG:teuthology.orchestra.run.smithi043:> /usr/sbin/getenforce 2024-03-26T08:10:30.176 INFO:teuthology.orchestra.run.smithi043.stdout:Permissive 2024-03-26T08:10:30.177 DEBUG:teuthology.orchestra.run.smithi052:> /usr/sbin/getenforce 2024-03-26T08:10:30.240 INFO:teuthology.orchestra.run.smithi052.stdout:Permissive 2024-03-26T08:10:30.241 DEBUG:teuthology.orchestra.run.smithi183:> /usr/sbin/getenforce 2024-03-26T08:10:30.286 INFO:teuthology.orchestra.run.smithi183.stdout:Permissive 2024-03-26T08:10:30.286 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi043.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi052.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi183.front.sepia.ceph.com': 'permissive'} 2024-03-26T08:10:30.286 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T08:10:30.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T08:10:30.378 DEBUG:teuthology.orchestra.run.smithi052:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T08:10:30.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T08:10:30.461 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T08:10:30.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T08:10:30.541 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-26T08:10:30.541 DEBUG:teuthology.orchestra.run.smithi043:> sudo /usr/sbin/setenforce permissive 2024-03-26T08:10:30.657 DEBUG:teuthology.orchestra.run.smithi052:> sudo /usr/sbin/setenforce permissive 2024-03-26T08:10:30.745 DEBUG:teuthology.orchestra.run.smithi183:> sudo /usr/sbin/setenforce permissive 2024-03-26T08:10:30.835 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-26T08:10:30.962 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-26T08:10:30.968 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-26T08:10:31.125 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-26T08:10:31.141 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-26T08:10:31.142 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi043.front.sepia.ceph.com,smithi052.front.sepia.ceph.com,smithi183.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-26T08:16:45.688 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi043.front.sepia.ceph.com'), Remote(name='ubuntu@smithi052.front.sepia.ceph.com'), Remote(name='ubuntu@smithi183.front.sepia.ceph.com')] 2024-03-26T08:16:45.725 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-03-26T08:16:45.727 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T08:16:45.876 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-03-26T08:16:46.034 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-03-26T08:16:46.034 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-03-26T08:16:46.035 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T08:16:46.184 DEBUG:teuthology.orchestra.run.smithi052:> true 2024-03-26T08:16:46.341 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-03-26T08:16:46.342 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-03-26T08:16:46.343 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T08:16:46.489 DEBUG:teuthology.orchestra.run.smithi183:> true 2024-03-26T08:16:46.635 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-03-26T08:16:46.636 INFO:teuthology.run_tasks:Running task clock... 2024-03-26T08:16:46.647 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-26T08:16:46.647 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-26T08:16:46.647 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T08:16:46.655 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-26T08:16:46.655 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T08:16:46.663 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-26T08:16:46.663 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T08:16:46.755 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-26T08:16:46.759 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-26T08:16:46.768 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-26T08:16:46.810 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-26T08:16:46.815 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-26T08:16:46.819 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-26T08:16:46.911 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: ntpd: command not found 2024-03-26T08:16:46.912 INFO:teuthology.orchestra.run.smithi052.stderr:sudo: ntpd: command not found 2024-03-26T08:16:46.915 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: ntpd: command not found 2024-03-26T08:16:46.952 INFO:teuthology.orchestra.run.smithi052.stdout:506 Cannot talk to daemon 2024-03-26T08:16:46.955 INFO:teuthology.orchestra.run.smithi183.stdout:506 Cannot talk to daemon 2024-03-26T08:16:46.957 INFO:teuthology.orchestra.run.smithi043.stdout:506 Cannot talk to daemon 2024-03-26T08:16:47.006 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-26T08:16:47.007 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-26T08:16:47.014 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-26T08:16:47.062 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-26T08:16:47.063 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-26T08:16:47.071 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-26T08:16:47.190 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-03-26T08:16:47.199 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T08:16:47.199 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-03-26T08:16:47.200 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T08:16:47.200 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T08:16:47.200 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T08:16:47.200 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T08:16:47.201 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-03-26T08:16:47.206 INFO:teuthology.orchestra.run.smithi052.stderr:bash: line 1: ntpq: command not found 2024-03-26T08:16:47.209 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T08:16:47.209 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-03-26T08:16:47.209 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T08:16:47.209 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T08:16:47.209 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T08:16:47.209 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T08:16:47.215 INFO:teuthology.orchestra.run.smithi052.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T08:16:47.215 INFO:teuthology.orchestra.run.smithi052.stdout:=============================================================================== 2024-03-26T08:16:47.215 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T08:16:47.215 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T08:16:47.215 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T08:16:47.215 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T08:16:47.217 INFO:teuthology.run_tasks:Running task install... 2024-03-26T08:16:47.226 DEBUG:teuthology.task.install:project ceph 2024-03-26T08:16:47.226 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'}} 2024-03-26T08:16:47.226 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'} 2024-03-26T08:16:47.226 INFO:teuthology.task.install:Using flavor: default 2024-03-26T08:16:47.247 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-26T08:16:47.247 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-26T08:16:47.247 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': None, 'wait_for_package': False} 2024-03-26T08:16:47.248 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T08:16:47.250 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': None, 'wait_for_package': False} 2024-03-26T08:16:47.250 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T08:16:47.251 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': None, 'wait_for_package': False} 2024-03-26T08:16:47.251 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T08:16:47.398 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/ 2024-03-26T08:16:47.398 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2145.g3be9a5c9 2024-03-26T08:16:47.466 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/ 2024-03-26T08:16:47.466 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2145.g3be9a5c9 2024-03-26T08:16:47.482 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/ 2024-03-26T08:16:47.483 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2145.g3be9a5c9 2024-03-26T08:16:47.508 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-26T08:16:47.508 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-03-26T08:16:47.508 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-26T08:16:47.585 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-26T08:16:47.585 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-26T08:16:47.585 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-26T08:16:47.606 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-26T08:16:47.607 DEBUG:teuthology.orchestra.run.smithi183:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3be9a5c9bc793e11e2800a8c0c696e8b46742033/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-26T08:16:47.652 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-26T08:16:47.652 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-26T08:16:47.652 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-26T08:16:47.692 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-26T08:16:47.693 DEBUG:teuthology.orchestra.run.smithi043:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3be9a5c9bc793e11e2800a8c0c696e8b46742033/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-26T08:16:47.745 DEBUG:teuthology.orchestra.run.smithi183:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-26T08:16:47.759 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-26T08:16:47.759 DEBUG:teuthology.orchestra.run.smithi052:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3be9a5c9bc793e11e2800a8c0c696e8b46742033/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-26T08:16:47.829 DEBUG:teuthology.orchestra.run.smithi043:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-26T08:16:47.875 DEBUG:teuthology.orchestra.run.smithi183:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-26T08:16:47.903 DEBUG:teuthology.orchestra.run.smithi052:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-26T08:16:47.962 INFO:teuthology.orchestra.run.smithi183.stdout:check_obsoletes = 1 2024-03-26T08:16:47.967 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum clean all 2024-03-26T08:16:47.968 DEBUG:teuthology.orchestra.run.smithi043:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-26T08:16:48.048 DEBUG:teuthology.orchestra.run.smithi052:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-26T08:16:48.065 INFO:teuthology.orchestra.run.smithi043.stdout:check_obsoletes = 1 2024-03-26T08:16:48.070 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum clean all 2024-03-26T08:16:48.145 INFO:teuthology.orchestra.run.smithi052.stdout:check_obsoletes = 1 2024-03-26T08:16:48.149 DEBUG:teuthology.orchestra.run.smithi052:> sudo yum clean all 2024-03-26T08:16:48.328 INFO:teuthology.orchestra.run.smithi183.stdout:74 files removed 2024-03-26T08:16:48.374 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-26T08:16:48.422 INFO:teuthology.orchestra.run.smithi043.stdout:74 files removed 2024-03-26T08:16:48.462 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-26T08:16:48.519 INFO:teuthology.orchestra.run.smithi052.stdout:74 files removed 2024-03-26T08:16:48.559 DEBUG:teuthology.orchestra.run.smithi052:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-26T08:16:49.274 INFO:teuthology.orchestra.run.smithi183.stdout:ceph packages for x86_64 213 kB/s | 83 kB 00:00 2024-03-26T08:16:49.560 INFO:teuthology.orchestra.run.smithi043.stdout:ceph packages for x86_64 143 kB/s | 83 kB 00:00 2024-03-26T08:16:49.579 INFO:teuthology.orchestra.run.smithi052.stdout:ceph packages for x86_64 169 kB/s | 83 kB 00:00 2024-03-26T08:16:49.783 INFO:teuthology.orchestra.run.smithi183.stdout:ceph noarch packages 24 kB/s | 10 kB 00:00 2024-03-26T08:16:50.095 INFO:teuthology.orchestra.run.smithi052.stdout:ceph noarch packages 24 kB/s | 10 kB 00:00 2024-03-26T08:16:50.096 INFO:teuthology.orchestra.run.smithi043.stdout:ceph noarch packages 23 kB/s | 10 kB 00:00 2024-03-26T08:16:50.275 INFO:teuthology.orchestra.run.smithi183.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-03-26T08:16:50.619 INFO:teuthology.orchestra.run.smithi043.stdout:ceph source packages 4.3 kB/s | 1.9 kB 00:00 2024-03-26T08:16:50.663 INFO:teuthology.orchestra.run.smithi052.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-03-26T08:16:50.870 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-26T08:16:51.180 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-26T08:16:51.308 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-26T08:16:52.612 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS-9 - BaseOS 4.3 MB/s | 2.0 MB 00:00 2024-03-26T08:16:52.873 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS-9 - BaseOS 5.1 MB/s | 2.0 MB 00:00 2024-03-26T08:16:53.059 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS-9 - BaseOS 4.5 MB/s | 2.0 MB 00:00 2024-03-26T08:16:53.946 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.0 MB 00:00 2024-03-26T08:16:53.965 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - BaseOS 8.6 MB/s | 8.0 MB 00:00 2024-03-26T08:16:55.419 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 3.7 MB/s | 8.0 MB 00:02 2024-03-26T08:16:58.176 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - AppStream 6.2 MB/s | 19 MB 00:03 2024-03-26T08:16:59.203 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 7.4 MB/s | 19 MB 00:02 2024-03-26T08:17:02.976 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - CRB 6.3 MB/s | 6.1 MB 00:00 2024-03-26T08:17:04.395 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 4.9 MB/s | 6.1 MB 00:01 2024-03-26T08:17:04.585 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 16 kB 00:00 2024-03-26T08:17:06.054 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 55 kB/s | 16 kB 00:00 2024-03-26T08:17:07.014 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 27 MB/s | 21 MB 00:00 2024-03-26T08:17:10.763 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:15 2024-03-26T08:17:13.064 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-03-26T08:17:15.213 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.1 MB 00:00 2024-03-26T08:17:15.843 INFO:teuthology.orchestra.run.smithi043.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T08:17:15.845 INFO:teuthology.orchestra.run.smithi043.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T08:17:15.869 INFO:teuthology.orchestra.run.smithi043.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-26T08:17:15.870 INFO:teuthology.orchestra.run.smithi043.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-26T08:17:15.872 INFO:teuthology.orchestra.run.smithi043.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-26T08:17:15.873 INFO:teuthology.orchestra.run.smithi043.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-26T08:17:15.876 INFO:teuthology.orchestra.run.smithi043.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-26T08:17:15.880 INFO:teuthology.orchestra.run.smithi043.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-26T08:17:15.881 INFO:teuthology.orchestra.run.smithi043.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-26T08:17:15.976 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-26T08:17:16.071 INFO:teuthology.orchestra.run.smithi043.stdout:============================================================================================ 2024-03-26T08:17:16.071 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-03-26T08:17:16.072 INFO:teuthology.orchestra.run.smithi043.stdout:============================================================================================ 2024-03-26T08:17:16.072 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-03-26T08:17:16.072 INFO:teuthology.orchestra.run.smithi043.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-26T08:17:16.072 INFO:teuthology.orchestra.run.smithi043.stdout: ceph x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 6.6 k 2024-03-26T08:17:16.072 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.4 M 2024-03-26T08:17:16.072 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 890 k 2024-03-26T08:17:16.072 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 145 k 2024-03-26T08:17:16.072 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 1.5 M 2024-03-26T08:17:16.072 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 139 k 2024-03-26T08:17:16.072 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 1.8 M 2024-03-26T08:17:16.072 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 7.4 M 2024-03-26T08:17:16.072 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 49 k 2024-03-26T08:17:16.073 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 9.9 M 2024-03-26T08:17:16.073 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 48 M 2024-03-26T08:17:16.073 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 264 k 2024-03-26T08:17:16.073 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 748 k 2024-03-26T08:17:16.073 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 220 k 2024-03-26T08:17:16.073 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-shell x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 27 k 2024-03-26T08:17:16.073 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-top noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 18 k 2024-03-26T08:17:16.073 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-26T08:17:16.073 INFO:teuthology.orchestra.run.smithi043.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-26T08:17:16.073 INFO:teuthology.orchestra.run.smithi043.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-26T08:17:16.073 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 31 k 2024-03-26T08:17:16.073 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 746 k 2024-03-26T08:17:16.073 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 127 k 2024-03-26T08:17:16.074 INFO:teuthology.orchestra.run.smithi043.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-26T08:17:16.074 INFO:teuthology.orchestra.run.smithi043.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-26T08:17:16.074 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-26T08:17:16.074 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-26T08:17:16.074 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-26T08:17:16.074 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-26T08:17:16.074 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 162 k 2024-03-26T08:17:16.074 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 321 k 2024-03-26T08:17:16.074 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 298 k 2024-03-26T08:17:16.074 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 99 k 2024-03-26T08:17:16.074 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 85 k 2024-03-26T08:17:16.075 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-26T08:17:16.075 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 170 k 2024-03-26T08:17:16.075 INFO:teuthology.orchestra.run.smithi043.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-26T08:17:16.075 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrading: 2024-03-26T08:17:16.075 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-26T08:17:16.075 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-26T08:17:16.075 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-26T08:17:16.075 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-26T08:17:16.075 INFO:teuthology.orchestra.run.smithi043.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-26T08:17:16.075 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-26T08:17:16.075 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-26T08:17:16.075 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-26T08:17:16.076 INFO:teuthology.orchestra.run.smithi043.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-26T08:17:16.076 INFO:teuthology.orchestra.run.smithi043.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-26T08:17:16.076 INFO:teuthology.orchestra.run.smithi043.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-26T08:17:16.076 INFO:teuthology.orchestra.run.smithi043.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-26T08:17:16.076 INFO:teuthology.orchestra.run.smithi043.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-26T08:17:16.076 INFO:teuthology.orchestra.run.smithi043.stdout: librados2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.4 M 2024-03-26T08:17:16.076 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-26T08:17:16.076 INFO:teuthology.orchestra.run.smithi043.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-26T08:17:16.076 INFO:teuthology.orchestra.run.smithi043.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-26T08:17:16.076 INFO:teuthology.orchestra.run.smithi043.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-26T08:17:16.076 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-26T08:17:16.076 INFO:teuthology.orchestra.run.smithi043.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-26T08:17:16.077 INFO:teuthology.orchestra.run.smithi043.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-26T08:17:16.077 INFO:teuthology.orchestra.run.smithi043.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-26T08:17:16.077 INFO:teuthology.orchestra.run.smithi043.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-26T08:17:16.077 INFO:teuthology.orchestra.run.smithi043.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-26T08:17:16.077 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-03-26T08:17:16.077 INFO:teuthology.orchestra.run.smithi043.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-26T08:17:16.077 INFO:teuthology.orchestra.run.smithi043.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-26T08:17:16.077 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-26T08:17:16.077 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 20 M 2024-03-26T08:17:16.077 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 34 k 2024-03-26T08:17:16.077 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 2.3 M 2024-03-26T08:17:16.077 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 248 k 2024-03-26T08:17:16.078 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 4.7 M 2024-03-26T08:17:16.078 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 17 M 2024-03-26T08:17:16.078 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 16 k 2024-03-26T08:17:16.078 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 25 k 2024-03-26T08:17:16.078 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-26T08:17:16.078 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-26T08:17:16.078 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-26T08:17:16.078 INFO:teuthology.orchestra.run.smithi043.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-26T08:17:16.078 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-26T08:17:16.078 INFO:teuthology.orchestra.run.smithi043.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-26T08:17:16.078 INFO:teuthology.orchestra.run.smithi043.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-26T08:17:16.078 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-26T08:17:16.079 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-26T08:17:16.079 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 163 k 2024-03-26T08:17:16.079 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-26T08:17:16.079 INFO:teuthology.orchestra.run.smithi043.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-26T08:17:16.079 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-26T08:17:16.079 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-26T08:17:16.079 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-26T08:17:16.079 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 501 k 2024-03-26T08:17:16.079 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-26T08:17:16.079 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.1 M 2024-03-26T08:17:16.079 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-26T08:17:16.079 INFO:teuthology.orchestra.run.smithi043.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-26T08:17:16.079 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-26T08:17:16.079 INFO:teuthology.orchestra.run.smithi043.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-26T08:17:16.080 INFO:teuthology.orchestra.run.smithi043.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-26T08:17:16.080 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-26T08:17:16.080 INFO:teuthology.orchestra.run.smithi043.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-26T08:17:16.080 INFO:teuthology.orchestra.run.smithi043.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-26T08:17:16.080 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-26T08:17:16.080 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-26T08:17:16.080 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-26T08:17:16.080 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-26T08:17:16.080 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-26T08:17:16.080 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-26T08:17:16.080 INFO:teuthology.orchestra.run.smithi043.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-26T08:17:16.080 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-26T08:17:16.080 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-26T08:17:16.081 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-26T08:17:16.081 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 45 k 2024-03-26T08:17:16.081 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 131 k 2024-03-26T08:17:16.081 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-26T08:17:16.081 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-26T08:17:16.081 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-26T08:17:16.081 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-26T08:17:16.081 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-26T08:17:16.081 INFO:teuthology.orchestra.run.smithi043.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-26T08:17:16.081 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-26T08:17:16.081 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-26T08:17:16.081 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-26T08:17:16.081 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-26T08:17:16.081 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-26T08:17:16.082 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-26T08:17:16.082 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-26T08:17:16.082 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-26T08:17:16.082 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-26T08:17:16.082 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-26T08:17:16.082 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-26T08:17:16.082 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-26T08:17:16.082 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-26T08:17:16.082 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-26T08:17:16.082 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-26T08:17:16.082 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-26T08:17:16.082 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-26T08:17:16.082 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-26T08:17:16.083 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-26T08:17:16.083 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-26T08:17:16.083 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-26T08:17:16.083 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-26T08:17:16.083 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-26T08:17:16.083 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-26T08:17:16.083 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-26T08:17:16.083 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-26T08:17:16.083 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-26T08:17:16.083 INFO:teuthology.orchestra.run.smithi043.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-26T08:17:16.083 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-26T08:17:16.083 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-26T08:17:16.083 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-26T08:17:16.084 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-26T08:17:16.084 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-26T08:17:16.084 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-26T08:17:16.084 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-26T08:17:16.084 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-26T08:17:16.084 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-26T08:17:16.084 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-26T08:17:16.084 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-26T08:17:16.084 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-26T08:17:16.084 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-26T08:17:16.084 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-26T08:17:16.084 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-26T08:17:16.085 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-26T08:17:16.085 INFO:teuthology.orchestra.run.smithi043.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-26T08:17:16.085 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-26T08:17:16.085 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-26T08:17:16.085 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-26T08:17:16.085 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-26T08:17:16.085 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-26T08:17:16.085 INFO:teuthology.orchestra.run.smithi043.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-26T08:17:16.085 INFO:teuthology.orchestra.run.smithi043.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-26T08:17:16.085 INFO:teuthology.orchestra.run.smithi043.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-26T08:17:16.085 INFO:teuthology.orchestra.run.smithi043.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-26T08:17:16.085 INFO:teuthology.orchestra.run.smithi043.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-26T08:17:16.085 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-26T08:17:16.085 INFO:teuthology.orchestra.run.smithi043.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-26T08:17:16.086 INFO:teuthology.orchestra.run.smithi043.stdout:Installing weak dependencies: 2024-03-26T08:17:16.086 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-26T08:17:16.086 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:17:16.086 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-03-26T08:17:16.086 INFO:teuthology.orchestra.run.smithi043.stdout:============================================================================================ 2024-03-26T08:17:16.086 INFO:teuthology.orchestra.run.smithi043.stdout:Install 150 Packages 2024-03-26T08:17:16.086 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrade 24 Packages 2024-03-26T08:17:16.086 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:17:16.087 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 225 M 2024-03-26T08:17:16.087 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-03-26T08:17:16.982 INFO:teuthology.orchestra.run.smithi043.stdout:(1/174): ceph-19.0.0-2145.g3be9a5c9.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-03-26T08:17:17.130 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-03-26T08:17:17.257 INFO:teuthology.orchestra.run.smithi043.stdout:(2/174): ceph-fuse-19.0.0-2145.g3be9a5c9.el9.x8 3.2 MB/s | 890 kB 00:00 2024-03-26T08:17:17.416 INFO:teuthology.orchestra.run.smithi043.stdout:(3/174): ceph-immutable-object-cache-19.0.0-214 922 kB/s | 145 kB 00:00 2024-03-26T08:17:17.583 INFO:teuthology.orchestra.run.smithi043.stdout:(4/174): ceph-base-19.0.0-2145.g3be9a5c9.el9.x8 6.3 MB/s | 5.4 MB 00:00 2024-03-26T08:17:17.691 INFO:teuthology.orchestra.run.smithi043.stdout:(5/174): ceph-mds-19.0.0-2145.g3be9a5c9.el9.x86 8.4 MB/s | 2.3 MB 00:00 2024-03-26T08:17:17.766 INFO:teuthology.orchestra.run.smithi043.stdout:(6/174): ceph-mgr-19.0.0-2145.g3be9a5c9.el9.x86 8.2 MB/s | 1.5 MB 00:00 2024-03-26T08:17:19.600 INFO:teuthology.orchestra.run.smithi043.stdout:(7/174): ceph-osd-19.0.0-2145.g3be9a5c9.el9.x86 9.2 MB/s | 17 MB 00:01 2024-03-26T08:17:19.725 INFO:teuthology.orchestra.run.smithi043.stdout:(8/174): ceph-mon-19.0.0-2145.g3be9a5c9.el9.x86 2.3 MB/s | 4.7 MB 00:02 2024-03-26T08:17:19.817 INFO:teuthology.orchestra.run.smithi043.stdout:(9/174): ceph-selinux-19.0.0-2145.g3be9a5c9.el9 277 kB/s | 25 kB 00:00 2024-03-26T08:17:20.182 INFO:teuthology.orchestra.run.smithi052.stdout:Extra Packages for Enterprise Linux 7.2 MB/s | 21 MB 00:02 2024-03-26T08:17:20.484 INFO:teuthology.orchestra.run.smithi043.stdout:(10/174): ceph-radosgw-19.0.0-2145.g3be9a5c9.el 11 MB/s | 9.9 MB 00:00 2024-03-26T08:17:20.609 INFO:teuthology.orchestra.run.smithi043.stdout:(11/174): cephfs-mirror-19.0.0-2145.g3be9a5c9.e 1.7 MB/s | 220 kB 00:00 2024-03-26T08:17:20.793 INFO:teuthology.orchestra.run.smithi043.stdout:(12/174): cephfs-shell-19.0.0-2145.g3be9a5c9.el 148 kB/s | 27 kB 00:00 2024-03-26T08:17:21.068 INFO:teuthology.orchestra.run.smithi043.stdout:(13/174): libcephfs-devel-19.0.0-2145.g3be9a5c9 114 kB/s | 31 kB 00:00 2024-03-26T08:17:21.627 INFO:teuthology.orchestra.run.smithi043.stdout:(14/174): ceph-common-19.0.0-2145.g3be9a5c9.el9 4.2 MB/s | 20 MB 00:04 2024-03-26T08:17:21.702 INFO:teuthology.orchestra.run.smithi043.stdout:(15/174): libcephfs2-19.0.0-2145.g3be9a5c9.el9. 1.2 MB/s | 746 kB 00:00 2024-03-26T08:17:21.736 INFO:teuthology.orchestra.run.smithi043.stdout:(16/174): libcephsqlite-19.0.0-2145.g3be9a5c9.e 1.5 MB/s | 163 kB 00:00 2024-03-26T08:17:21.794 INFO:teuthology.orchestra.run.smithi043.stdout:(17/174): librados-devel-19.0.0-2145.g3be9a5c9. 1.4 MB/s | 127 kB 00:00 2024-03-26T08:17:21.878 INFO:teuthology.orchestra.run.smithi043.stdout:(18/174): libradosstriper1-19.0.0-2145.g3be9a5c 3.5 MB/s | 501 kB 00:00 2024-03-26T08:17:21.970 INFO:teuthology.orchestra.run.smithi043.stdout:(19/174): python3-ceph-argparse-19.0.0-2145.g3b 496 kB/s | 45 kB 00:00 2024-03-26T08:17:22.098 INFO:teuthology.orchestra.run.smithi043.stdout:(20/174): python3-ceph-common-19.0.0-2145.g3be9 1.0 MB/s | 131 kB 00:00 2024-03-26T08:17:22.354 INFO:teuthology.orchestra.run.smithi043.stdout:(21/174): librgw2-19.0.0-2145.g3be9a5c9.el9.x86 9.2 MB/s | 5.1 MB 00:00 2024-03-26T08:17:22.387 INFO:teuthology.orchestra.run.smithi043.stdout:(22/174): python3-cephfs-19.0.0-2145.g3be9a5c9. 563 kB/s | 162 kB 00:00 2024-03-26T08:17:22.445 INFO:teuthology.orchestra.run.smithi043.stdout:(23/174): python3-rados-19.0.0-2145.g3be9a5c9.e 3.5 MB/s | 321 kB 00:00 2024-03-26T08:17:22.529 INFO:teuthology.orchestra.run.smithi043.stdout:(24/174): python3-rbd-19.0.0-2145.g3be9a5c9.el9 2.1 MB/s | 298 kB 00:00 2024-03-26T08:17:22.563 INFO:teuthology.orchestra.run.smithi043.stdout:(25/174): python3-rgw-19.0.0-2145.g3be9a5c9.el9 854 kB/s | 99 kB 00:00 2024-03-26T08:17:22.605 INFO:teuthology.orchestra.run.smithi043.stdout:(26/174): rbd-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.1 MB/s | 85 kB 00:00 2024-03-26T08:17:22.755 INFO:teuthology.orchestra.run.smithi043.stdout:(27/174): rbd-nbd-19.0.0-2145.g3be9a5c9.el9.x86 1.1 MB/s | 170 kB 00:00 2024-03-26T08:17:22.869 INFO:teuthology.orchestra.run.smithi043.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2145.g 300 kB/s | 34 kB 00:00 2024-03-26T08:17:22.956 INFO:teuthology.orchestra.run.smithi043.stdout:(29/174): rbd-mirror-19.0.0-2145.g3be9a5c9.el9. 7.9 MB/s | 3.1 MB 00:00 2024-03-26T08:17:22.981 INFO:teuthology.orchestra.run.smithi043.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2145.g3be9a5c 1.2 MB/s | 139 kB 00:00 2024-03-26T08:17:23.732 INFO:teuthology.orchestra.run.smithi043.stdout:(31/174): ceph-test-19.0.0-2145.g3be9a5c9.el9.x 12 MB/s | 48 MB 00:03 2024-03-26T08:17:23.807 INFO:teuthology.orchestra.run.smithi043.stdout:(32/174): ceph-mgr-dashboard-19.0.0-2145.g3be9a 2.1 MB/s | 1.8 MB 00:00 2024-03-26T08:17:23.965 INFO:teuthology.orchestra.run.smithi043.stdout:(33/174): ceph-mgr-rook-19.0.0-2145.g3be9a5c9.e 312 kB/s | 49 kB 00:00 2024-03-26T08:17:23.999 INFO:teuthology.orchestra.run.smithi043.stdout:(34/174): ceph-mgr-modules-core-19.0.0-2145.g3b 933 kB/s | 248 kB 00:00 2024-03-26T08:17:24.024 INFO:teuthology.orchestra.run.smithi043.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2145.g3 286 kB/s | 16 kB 00:00 2024-03-26T08:17:24.175 INFO:teuthology.orchestra.run.smithi043.stdout:(36/174): ceph-mgr-diskprediction-local-19.0.0- 6.2 MB/s | 7.4 MB 00:01 2024-03-26T08:17:24.208 INFO:teuthology.orchestra.run.smithi043.stdout:(37/174): ceph-volume-19.0.0-2145.g3be9a5c9.el9 1.2 MB/s | 264 kB 00:00 2024-03-26T08:17:24.250 INFO:teuthology.orchestra.run.smithi043.stdout:(38/174): cephadm-19.0.0-2145.g3be9a5c9.el9.noa 3.3 MB/s | 748 kB 00:00 2024-03-26T08:17:24.334 INFO:teuthology.orchestra.run.smithi043.stdout:(39/174): cephfs-top-19.0.0-2145.g3be9a5c9.el9. 116 kB/s | 18 kB 00:00 2024-03-26T08:17:24.526 INFO:teuthology.orchestra.run.smithi043.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 4.4 MB/s | 920 kB 00:00 2024-03-26T08:17:24.567 INFO:teuthology.orchestra.run.smithi043.stdout:(41/174): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 662 kB 00:00 2024-03-26T08:17:24.593 INFO:teuthology.orchestra.run.smithi043.stdout:(42/174): boost-program-options-1.75.0-8.el9.x8 529 kB/s | 106 kB 00:00 2024-03-26T08:17:24.626 INFO:teuthology.orchestra.run.smithi043.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 3.1 MB/s | 307 kB 00:00 2024-03-26T08:17:24.662 INFO:teuthology.orchestra.run.smithi043.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 503 kB/s | 32 kB 00:00 2024-03-26T08:17:24.777 INFO:teuthology.orchestra.run.smithi043.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 118 kB/s | 17 kB 00:00 2024-03-26T08:17:24.844 INFO:teuthology.orchestra.run.smithi043.stdout:(46/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-03-26T08:17:24.894 INFO:teuthology.orchestra.run.smithi043.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 44 kB/s | 10 kB 00:00 2024-03-26T08:17:24.969 INFO:teuthology.orchestra.run.smithi043.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 843 kB/s | 159 kB 00:00 2024-03-26T08:17:25.003 INFO:teuthology.orchestra.run.smithi043.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 417 kB/s | 44 kB 00:00 2024-03-26T08:17:25.045 INFO:teuthology.orchestra.run.smithi043.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 4.4 MB/s | 661 kB 00:00 2024-03-26T08:17:25.086 INFO:teuthology.orchestra.run.smithi043.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 5.1 MB/s | 578 kB 00:00 2024-03-26T08:17:25.112 INFO:teuthology.orchestra.run.smithi043.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 765 kB/s | 46 kB 00:00 2024-03-26T08:17:25.137 INFO:teuthology.orchestra.run.smithi043.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.2 MB/s | 282 kB 00:00 2024-03-26T08:17:25.170 INFO:teuthology.orchestra.run.smithi043.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.3 MB/s | 187 kB 00:00 2024-03-26T08:17:25.196 INFO:teuthology.orchestra.run.smithi043.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 615 kB/s | 34 kB 00:00 2024-03-26T08:17:25.229 INFO:teuthology.orchestra.run.smithi043.stdout:(56/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.4 MB/s | 217 kB 00:00 2024-03-26T08:17:25.413 INFO:teuthology.orchestra.run.smithi043.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-03-26T08:17:25.472 INFO:teuthology.orchestra.run.smithi043.stdout:(58/174): protobuf-3.14.0-13.el9.x86_64.rpm 3.8 MB/s | 1.0 MB 00:00 2024-03-26T08:17:25.496 INFO:teuthology.orchestra.run.smithi043.stdout:(59/174): python3-attrs-20.3.0-7.el9.noarch.rpm 308 kB/s | 78 kB 00:00 2024-03-26T08:17:25.673 INFO:teuthology.orchestra.run.smithi043.stdout:(60/174): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 5.8 MB 00:00 2024-03-26T08:17:25.705 INFO:teuthology.orchestra.run.smithi043.stdout:(61/174): python3-jsonpointer-2.0-4.el9.noarch. 94 kB/s | 18 kB 00:00 2024-03-26T08:17:25.739 INFO:teuthology.orchestra.run.smithi043.stdout:(62/174): python3-jsonpatch-1.21-16.el9.noarch. 108 kB/s | 24 kB 00:00 2024-03-26T08:17:25.764 INFO:teuthology.orchestra.run.smithi043.stdout:(63/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 144 kB 00:00 2024-03-26T08:17:25.790 INFO:teuthology.orchestra.run.smithi043.stdout:(64/174): python3-markupsafe-1.1.1-12.el9.x86_6 383 kB/s | 32 kB 00:00 2024-03-26T08:17:25.932 INFO:teuthology.orchestra.run.smithi043.stdout:(65/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 4.8 MB 00:00 2024-03-26T08:17:25.956 INFO:teuthology.orchestra.run.smithi043.stdout:(66/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 217 kB 00:00 2024-03-26T08:17:25.982 INFO:teuthology.orchestra.run.smithi043.stdout:(67/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 174 kB 00:00 2024-03-26T08:17:26.024 INFO:teuthology.orchestra.run.smithi043.stdout:(68/174): python3-packaging-20.9-5.el9.noarch.r 1.7 MB/s | 69 kB 00:00 2024-03-26T08:17:26.072 INFO:teuthology.orchestra.run.smithi043.stdout:(69/174): python3-prettytable-0.7.2-27.el9.noar 466 kB/s | 41 kB 00:00 2024-03-26T08:17:26.108 INFO:teuthology.orchestra.run.smithi043.stdout:(70/174): python3-protobuf-3.14.0-13.el9.noarch 1.9 MB/s | 238 kB 00:00 2024-03-26T08:17:26.141 INFO:teuthology.orchestra.run.smithi043.stdout:(71/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-03-26T08:17:26.166 INFO:teuthology.orchestra.run.smithi043.stdout:(72/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 211 kB 00:00 2024-03-26T08:17:26.192 INFO:teuthology.orchestra.run.smithi043.stdout:(73/174): python3-pytz-2021.1-5.el9.noarch.rpm 592 kB/s | 47 kB 00:00 2024-03-26T08:17:26.207 INFO:teuthology.orchestra.run.smithi052.stdout:lab-extras 21 kB/s | 1.7 kB 00:00 2024-03-26T08:17:26.233 INFO:teuthology.orchestra.run.smithi043.stdout:(74/174): python3-requests-oauthlib-1.3.0-12.el 654 kB/s | 43 kB 00:00 2024-03-26T08:17:26.567 INFO:teuthology.orchestra.run.smithi043.stdout:(75/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 39 MB/s | 16 MB 00:00 2024-03-26T08:17:26.592 INFO:teuthology.orchestra.run.smithi043.stdout:(76/174): python3-toml-0.10.2-6.el9.noarch.rpm 101 kB/s | 39 kB 00:00 2024-03-26T08:17:26.617 INFO:teuthology.orchestra.run.smithi043.stdout:(77/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 111 kB/s | 41 kB 00:00 2024-03-26T08:17:26.659 INFO:teuthology.orchestra.run.smithi043.stdout:(78/174): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 56 kB 00:00 2024-03-26T08:17:26.693 INFO:teuthology.orchestra.run.smithi043.stdout:(79/174): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 300 kB 00:00 2024-03-26T08:17:26.718 INFO:teuthology.orchestra.run.smithi043.stdout:(80/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 636 kB/s | 63 kB 00:00 2024-03-26T08:17:26.743 INFO:teuthology.orchestra.run.smithi043.stdout:(81/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 553 kB/s | 44 kB 00:00 2024-03-26T08:17:26.777 INFO:teuthology.orchestra.run.smithi043.stdout:(82/174): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-03-26T08:17:26.794 INFO:teuthology.orchestra.run.smithi043.stdout:(83/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.2 MB/s | 241 kB 00:00 2024-03-26T08:17:26.852 INFO:teuthology.orchestra.run.smithi043.stdout:(84/174): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-03-26T08:17:26.903 INFO:teuthology.orchestra.run.smithi043.stdout:(85/174): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 103 kB 00:00 2024-03-26T08:17:26.928 INFO:teuthology.orchestra.run.smithi043.stdout:(86/174): python3-pycparser-2.20-6.el9.noarch.r 1.0 MB/s | 124 kB 00:00 2024-03-26T08:17:26.961 INFO:teuthology.orchestra.run.smithi043.stdout:(87/174): python3-urllib3-1.26.5-5.el9.noarch.r 1.7 MB/s | 187 kB 00:00 2024-03-26T08:17:27.103 INFO:teuthology.orchestra.run.smithi043.stdout:(88/174): jq-1.6-16.el9.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2024-03-26T08:17:27.128 INFO:teuthology.orchestra.run.smithi043.stdout:(89/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-03-26T08:17:27.179 INFO:teuthology.orchestra.run.smithi043.stdout:(90/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.2 MB/s | 803 kB 00:00 2024-03-26T08:17:27.204 INFO:teuthology.orchestra.run.smithi043.stdout:(91/174): elfutils-libelf-devel-0.190-2.el9.x86 292 kB/s | 21 kB 00:00 2024-03-26T08:17:27.229 INFO:teuthology.orchestra.run.smithi043.stdout:(92/174): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-03-26T08:17:27.288 INFO:teuthology.orchestra.run.smithi043.stdout:(93/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 235 kB/s | 25 kB 00:00 2024-03-26T08:17:27.421 INFO:teuthology.orchestra.run.smithi043.stdout:(94/174): postgresql-13.14-1.el9.x86_64.rpm 8.3 MB/s | 1.6 MB 00:00 2024-03-26T08:17:27.480 INFO:teuthology.orchestra.run.smithi043.stdout:(95/174): postgresql-contrib-13.14-1.el9.x86_64 4.7 MB/s | 911 kB 00:00 2024-03-26T08:17:27.522 INFO:teuthology.orchestra.run.smithi043.stdout:(96/174): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 133 kB 00:00 2024-03-26T08:17:27.631 INFO:teuthology.orchestra.run.smithi043.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-03-26T08:17:27.723 INFO:teuthology.orchestra.run.smithi043.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 2.7 MB/s | 248 kB 00:00 2024-03-26T08:17:27.823 INFO:teuthology.orchestra.run.smithi043.stdout:(99/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 6.7 MB/s | 4.1 MB 00:00 2024-03-26T08:17:27.915 INFO:teuthology.orchestra.run.smithi043.stdout:(100/174): qatlib-service-24.02.0-1.el9.x86_64. 396 kB/s | 36 kB 00:00 2024-03-26T08:17:27.948 INFO:teuthology.orchestra.run.smithi043.stdout:(101/174): qatlib-24.02.0-1.el9.x86_64.rpm 983 kB/s | 221 kB 00:00 2024-03-26T08:17:28.091 INFO:teuthology.orchestra.run.smithi043.stdout:(102/174): postgresql-server-13.14-1.el9.x86_64 9.5 MB/s | 5.8 MB 00:00 2024-03-26T08:17:28.116 INFO:teuthology.orchestra.run.smithi043.stdout:(103/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 238 kB/s | 47 kB 00:00 2024-03-26T08:17:28.149 INFO:teuthology.orchestra.run.smithi043.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 387 kB/s | 22 kB 00:00 2024-03-26T08:17:28.175 INFO:teuthology.orchestra.run.smithi043.stdout:(105/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 416 kB/s | 94 kB 00:00 2024-03-26T08:17:28.250 INFO:teuthology.orchestra.run.smithi043.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 6.3 MB/s | 863 kB 00:00 2024-03-26T08:17:28.709 INFO:teuthology.orchestra.run.smithi043.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 57 kB/s | 19 kB 00:00 2024-03-26T08:17:28.759 INFO:teuthology.orchestra.run.smithi043.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 530 kB/s | 309 kB 00:00 2024-03-26T08:17:28.858 INFO:teuthology.orchestra.run.smithi043.stdout:(109/174): abseil-cpp-20211102.0-3.el9.x86_64.r 775 kB/s | 548 kB 00:00 2024-03-26T08:17:28.893 INFO:teuthology.orchestra.run.smithi043.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 195 kB/s | 26 kB 00:00 2024-03-26T08:17:28.921 INFO:teuthology.orchestra.run.smithi052.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T08:17:28.923 INFO:teuthology.orchestra.run.smithi052.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T08:17:28.946 INFO:teuthology.orchestra.run.smithi052.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-26T08:17:28.947 INFO:teuthology.orchestra.run.smithi052.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-26T08:17:28.949 INFO:teuthology.orchestra.run.smithi052.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-26T08:17:28.951 INFO:teuthology.orchestra.run.smithi052.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-26T08:17:28.951 INFO:teuthology.orchestra.run.smithi043.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 528 kB/s | 49 kB 00:00 2024-03-26T08:17:28.953 INFO:teuthology.orchestra.run.smithi052.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-26T08:17:28.957 INFO:teuthology.orchestra.run.smithi052.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-26T08:17:28.959 INFO:teuthology.orchestra.run.smithi052.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-26T08:17:28.985 INFO:teuthology.orchestra.run.smithi043.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 740 kB/s | 67 kB 00:00 2024-03-26T08:17:29.052 INFO:teuthology.orchestra.run.smithi043.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-03-26T08:17:29.055 INFO:teuthology.orchestra.run.smithi052.stdout:Dependencies resolved. 2024-03-26T08:17:29.074 INFO:teuthology.orchestra.run.smithi052.stdout:============================================================================================ 2024-03-26T08:17:29.075 INFO:teuthology.orchestra.run.smithi052.stdout: Package Arch Version Repository Size 2024-03-26T08:17:29.075 INFO:teuthology.orchestra.run.smithi052.stdout:============================================================================================ 2024-03-26T08:17:29.075 INFO:teuthology.orchestra.run.smithi052.stdout:Installing: 2024-03-26T08:17:29.075 INFO:teuthology.orchestra.run.smithi052.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-26T08:17:29.075 INFO:teuthology.orchestra.run.smithi052.stdout: ceph x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 6.6 k 2024-03-26T08:17:29.075 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-base x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.4 M 2024-03-26T08:17:29.075 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 890 k 2024-03-26T08:17:29.075 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 145 k 2024-03-26T08:17:29.075 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 1.5 M 2024-03-26T08:17:29.075 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 139 k 2024-03-26T08:17:29.075 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 1.8 M 2024-03-26T08:17:29.075 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 7.4 M 2024-03-26T08:17:29.076 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-rook noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 49 k 2024-03-26T08:17:29.076 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-radosgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 9.9 M 2024-03-26T08:17:29.076 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-test x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 48 M 2024-03-26T08:17:29.076 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-volume noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 264 k 2024-03-26T08:17:29.076 INFO:teuthology.orchestra.run.smithi052.stdout: cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 748 k 2024-03-26T08:17:29.076 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 220 k 2024-03-26T08:17:29.076 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-shell x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 27 k 2024-03-26T08:17:29.076 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-top noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 18 k 2024-03-26T08:17:29.076 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-26T08:17:29.076 INFO:teuthology.orchestra.run.smithi052.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-26T08:17:29.076 INFO:teuthology.orchestra.run.smithi052.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-26T08:17:29.076 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 31 k 2024-03-26T08:17:29.076 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 746 k 2024-03-26T08:17:29.077 INFO:teuthology.orchestra.run.smithi052.stdout: librados-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 127 k 2024-03-26T08:17:29.077 INFO:teuthology.orchestra.run.smithi052.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-26T08:17:29.077 INFO:teuthology.orchestra.run.smithi052.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-26T08:17:29.077 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-26T08:17:29.077 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-26T08:17:29.077 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-26T08:17:29.077 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-26T08:17:29.077 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cephfs x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 162 k 2024-03-26T08:17:29.077 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rados x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 321 k 2024-03-26T08:17:29.077 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 298 k 2024-03-26T08:17:29.077 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 99 k 2024-03-26T08:17:29.077 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 85 k 2024-03-26T08:17:29.077 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-26T08:17:29.077 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-nbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 170 k 2024-03-26T08:17:29.078 INFO:teuthology.orchestra.run.smithi052.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-26T08:17:29.078 INFO:teuthology.orchestra.run.smithi052.stdout:Upgrading: 2024-03-26T08:17:29.078 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-26T08:17:29.078 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-26T08:17:29.078 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-26T08:17:29.078 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-26T08:17:29.078 INFO:teuthology.orchestra.run.smithi052.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-26T08:17:29.078 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-26T08:17:29.078 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-26T08:17:29.078 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-26T08:17:29.078 INFO:teuthology.orchestra.run.smithi052.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-26T08:17:29.078 INFO:teuthology.orchestra.run.smithi052.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-26T08:17:29.078 INFO:teuthology.orchestra.run.smithi052.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-26T08:17:29.079 INFO:teuthology.orchestra.run.smithi052.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-26T08:17:29.079 INFO:teuthology.orchestra.run.smithi052.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-26T08:17:29.079 INFO:teuthology.orchestra.run.smithi052.stdout: librados2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.4 M 2024-03-26T08:17:29.079 INFO:teuthology.orchestra.run.smithi052.stdout: librbd1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-26T08:17:29.079 INFO:teuthology.orchestra.run.smithi052.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-26T08:17:29.079 INFO:teuthology.orchestra.run.smithi052.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-26T08:17:29.079 INFO:teuthology.orchestra.run.smithi052.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-26T08:17:29.079 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-26T08:17:29.079 INFO:teuthology.orchestra.run.smithi052.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-26T08:17:29.079 INFO:teuthology.orchestra.run.smithi052.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-26T08:17:29.079 INFO:teuthology.orchestra.run.smithi052.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-26T08:17:29.079 INFO:teuthology.orchestra.run.smithi052.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-26T08:17:29.079 INFO:teuthology.orchestra.run.smithi052.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-26T08:17:29.080 INFO:teuthology.orchestra.run.smithi052.stdout:Installing dependencies: 2024-03-26T08:17:29.080 INFO:teuthology.orchestra.run.smithi052.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-26T08:17:29.080 INFO:teuthology.orchestra.run.smithi052.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-26T08:17:29.080 INFO:teuthology.orchestra.run.smithi052.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-26T08:17:29.080 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 20 M 2024-03-26T08:17:29.080 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 34 k 2024-03-26T08:17:29.080 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mds x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 2.3 M 2024-03-26T08:17:29.080 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 248 k 2024-03-26T08:17:29.080 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mon x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 4.7 M 2024-03-26T08:17:29.080 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-osd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 17 M 2024-03-26T08:17:29.080 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 16 k 2024-03-26T08:17:29.080 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-selinux x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 25 k 2024-03-26T08:17:29.080 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-26T08:17:29.080 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-26T08:17:29.081 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-26T08:17:29.081 INFO:teuthology.orchestra.run.smithi052.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-26T08:17:29.081 INFO:teuthology.orchestra.run.smithi052.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-26T08:17:29.081 INFO:teuthology.orchestra.run.smithi052.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-26T08:17:29.081 INFO:teuthology.orchestra.run.smithi052.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-26T08:17:29.081 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-26T08:17:29.081 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-26T08:17:29.081 INFO:teuthology.orchestra.run.smithi052.stdout: libcephsqlite x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 163 k 2024-03-26T08:17:29.081 INFO:teuthology.orchestra.run.smithi052.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-26T08:17:29.081 INFO:teuthology.orchestra.run.smithi052.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-26T08:17:29.081 INFO:teuthology.orchestra.run.smithi052.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-26T08:17:29.081 INFO:teuthology.orchestra.run.smithi052.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-26T08:17:29.081 INFO:teuthology.orchestra.run.smithi052.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-26T08:17:29.081 INFO:teuthology.orchestra.run.smithi052.stdout: libradosstriper1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 501 k 2024-03-26T08:17:29.082 INFO:teuthology.orchestra.run.smithi052.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-26T08:17:29.082 INFO:teuthology.orchestra.run.smithi052.stdout: librgw2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.1 M 2024-03-26T08:17:29.082 INFO:teuthology.orchestra.run.smithi052.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-26T08:17:29.082 INFO:teuthology.orchestra.run.smithi052.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-26T08:17:29.082 INFO:teuthology.orchestra.run.smithi052.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-26T08:17:29.082 INFO:teuthology.orchestra.run.smithi052.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-26T08:17:29.082 INFO:teuthology.orchestra.run.smithi052.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-26T08:17:29.082 INFO:teuthology.orchestra.run.smithi052.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-26T08:17:29.082 INFO:teuthology.orchestra.run.smithi052.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-26T08:17:29.082 INFO:teuthology.orchestra.run.smithi052.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-26T08:17:29.082 INFO:teuthology.orchestra.run.smithi052.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-26T08:17:29.082 INFO:teuthology.orchestra.run.smithi052.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-26T08:17:29.082 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-26T08:17:29.083 INFO:teuthology.orchestra.run.smithi052.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-26T08:17:29.083 INFO:teuthology.orchestra.run.smithi052.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-26T08:17:29.083 INFO:teuthology.orchestra.run.smithi052.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-26T08:17:29.083 INFO:teuthology.orchestra.run.smithi052.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-26T08:17:29.083 INFO:teuthology.orchestra.run.smithi052.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-26T08:17:29.083 INFO:teuthology.orchestra.run.smithi052.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-26T08:17:29.083 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-26T08:17:29.083 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-argparse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 45 k 2024-03-26T08:17:29.083 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 131 k 2024-03-26T08:17:29.083 INFO:teuthology.orchestra.run.smithi052.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-26T08:17:29.083 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-26T08:17:29.083 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-26T08:17:29.083 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-26T08:17:29.084 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-26T08:17:29.084 INFO:teuthology.orchestra.run.smithi052.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-26T08:17:29.084 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-26T08:17:29.084 INFO:teuthology.orchestra.run.smithi052.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-26T08:17:29.084 INFO:teuthology.orchestra.run.smithi052.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-26T08:17:29.084 INFO:teuthology.orchestra.run.smithi052.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-26T08:17:29.084 INFO:teuthology.orchestra.run.smithi052.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-26T08:17:29.084 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-26T08:17:29.084 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-26T08:17:29.084 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-26T08:17:29.084 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-26T08:17:29.084 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-26T08:17:29.084 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-26T08:17:29.085 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-26T08:17:29.085 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-26T08:17:29.085 INFO:teuthology.orchestra.run.smithi052.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-26T08:17:29.085 INFO:teuthology.orchestra.run.smithi052.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-26T08:17:29.085 INFO:teuthology.orchestra.run.smithi052.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-26T08:17:29.085 INFO:teuthology.orchestra.run.smithi052.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-26T08:17:29.085 INFO:teuthology.orchestra.run.smithi052.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-26T08:17:29.085 INFO:teuthology.orchestra.run.smithi052.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-26T08:17:29.085 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-26T08:17:29.085 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-26T08:17:29.085 INFO:teuthology.orchestra.run.smithi052.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-26T08:17:29.085 INFO:teuthology.orchestra.run.smithi052.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-26T08:17:29.085 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-26T08:17:29.085 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-26T08:17:29.086 INFO:teuthology.orchestra.run.smithi052.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-26T08:17:29.086 INFO:teuthology.orchestra.run.smithi052.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-26T08:17:29.086 INFO:teuthology.orchestra.run.smithi052.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-26T08:17:29.086 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-26T08:17:29.086 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-26T08:17:29.086 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-26T08:17:29.086 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-26T08:17:29.086 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-26T08:17:29.086 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-26T08:17:29.086 INFO:teuthology.orchestra.run.smithi052.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-26T08:17:29.086 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-26T08:17:29.086 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-26T08:17:29.087 INFO:teuthology.orchestra.run.smithi052.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-26T08:17:29.087 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-26T08:17:29.087 INFO:teuthology.orchestra.run.smithi052.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-26T08:17:29.087 INFO:teuthology.orchestra.run.smithi052.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-26T08:17:29.087 INFO:teuthology.orchestra.run.smithi052.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-26T08:17:29.087 INFO:teuthology.orchestra.run.smithi052.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-26T08:17:29.087 INFO:teuthology.orchestra.run.smithi052.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-26T08:17:29.087 INFO:teuthology.orchestra.run.smithi052.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-26T08:17:29.087 INFO:teuthology.orchestra.run.smithi052.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-26T08:17:29.087 INFO:teuthology.orchestra.run.smithi052.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-26T08:17:29.087 INFO:teuthology.orchestra.run.smithi052.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-26T08:17:29.087 INFO:teuthology.orchestra.run.smithi052.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-26T08:17:29.088 INFO:teuthology.orchestra.run.smithi052.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-26T08:17:29.088 INFO:teuthology.orchestra.run.smithi052.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-26T08:17:29.088 INFO:teuthology.orchestra.run.smithi052.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-26T08:17:29.088 INFO:teuthology.orchestra.run.smithi052.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-26T08:17:29.088 INFO:teuthology.orchestra.run.smithi052.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-26T08:17:29.088 INFO:teuthology.orchestra.run.smithi052.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-26T08:17:29.088 INFO:teuthology.orchestra.run.smithi052.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-26T08:17:29.088 INFO:teuthology.orchestra.run.smithi052.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-26T08:17:29.088 INFO:teuthology.orchestra.run.smithi052.stdout:Installing weak dependencies: 2024-03-26T08:17:29.088 INFO:teuthology.orchestra.run.smithi052.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-26T08:17:29.088 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-26T08:17:29.088 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction Summary 2024-03-26T08:17:29.088 INFO:teuthology.orchestra.run.smithi052.stdout:============================================================================================ 2024-03-26T08:17:29.089 INFO:teuthology.orchestra.run.smithi052.stdout:Install 150 Packages 2024-03-26T08:17:29.089 INFO:teuthology.orchestra.run.smithi052.stdout:Upgrade 24 Packages 2024-03-26T08:17:29.089 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-26T08:17:29.089 INFO:teuthology.orchestra.run.smithi052.stdout:Total download size: 225 M 2024-03-26T08:17:29.089 INFO:teuthology.orchestra.run.smithi052.stdout:Downloading Packages: 2024-03-26T08:17:29.185 INFO:teuthology.orchestra.run.smithi043.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.1 MB/s | 838 kB 00:00 2024-03-26T08:17:29.252 INFO:teuthology.orchestra.run.smithi043.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 2.7 MB/s | 548 kB 00:00 2024-03-26T08:17:29.278 INFO:teuthology.orchestra.run.smithi043.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 478 kB/s | 43 kB 00:00 2024-03-26T08:17:29.336 INFO:teuthology.orchestra.run.smithi043.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 389 kB/s | 32 kB 00:00 2024-03-26T08:17:29.361 INFO:teuthology.orchestra.run.smithi043.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 181 kB/s | 15 kB 00:00 2024-03-26T08:17:29.470 INFO:teuthology.orchestra.run.smithi043.stdout:(119/174): libarrow-9.0.0-10.el9.x86_64.rpm 5.8 MB/s | 4.4 MB 00:00 2024-03-26T08:17:29.504 INFO:teuthology.orchestra.run.smithi043.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.0 MB/s | 172 kB 00:00 2024-03-26T08:17:29.537 INFO:teuthology.orchestra.run.smithi043.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 2.0 MB/s | 358 kB 00:00 2024-03-26T08:17:29.579 INFO:teuthology.orchestra.run.smithi043.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-03-26T08:17:29.596 INFO:teuthology.orchestra.run.smithi043.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 386 kB/s | 35 kB 00:00 2024-03-26T08:17:29.663 INFO:teuthology.orchestra.run.smithi043.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 1.6 MB/s | 203 kB 00:00 2024-03-26T08:17:29.697 INFO:teuthology.orchestra.run.smithi043.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-03-26T08:17:29.739 INFO:teuthology.orchestra.run.smithi043.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 143 kB/s | 11 kB 00:00 2024-03-26T08:17:29.814 INFO:teuthology.orchestra.run.smithi043.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-03-26T08:17:29.839 INFO:teuthology.orchestra.run.smithi043.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 125 kB/s | 18 kB 00:00 2024-03-26T08:17:29.865 INFO:teuthology.orchestra.run.smithi043.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 186 kB/s | 23 kB 00:00 2024-03-26T08:17:29.898 INFO:teuthology.orchestra.run.smithi043.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 233 kB/s | 19 kB 00:00 2024-03-26T08:17:29.948 INFO:teuthology.orchestra.run.smithi043.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 238 kB/s | 20 kB 00:00 2024-03-26T08:17:29.982 INFO:teuthology.orchestra.run.smithi043.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 558 kB/s | 46 kB 00:00 2024-03-26T08:17:30.049 INFO:teuthology.orchestra.run.smithi043.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 5.6 MB/s | 1.0 MB 00:00 2024-03-26T08:17:30.074 INFO:teuthology.orchestra.run.smithi043.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 633 kB/s | 58 kB 00:00 2024-03-26T08:17:30.099 INFO:teuthology.orchestra.run.smithi043.stdout:(135/174): python3-more-itertools-8.12.0-2.el9. 526 kB/s | 79 kB 00:00 2024-03-26T08:17:30.158 INFO:teuthology.orchestra.run.smithi043.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-03-26T08:17:30.185 INFO:teuthology.orchestra.run.smithi043.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 152 kB/s | 16 kB 00:00 2024-03-26T08:17:30.209 INFO:teuthology.orchestra.run.smithi043.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 831 kB/s | 90 kB 00:00 2024-03-26T08:17:30.242 INFO:teuthology.orchestra.run.smithi043.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 339 kB/s | 28 kB 00:00 2024-03-26T08:17:30.276 INFO:teuthology.orchestra.run.smithi043.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 337 kB/s | 31 kB 00:00 2024-03-26T08:17:30.318 INFO:teuthology.orchestra.run.smithi043.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-03-26T08:17:30.343 INFO:teuthology.orchestra.run.smithi043.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 591 kB/s | 59 kB 00:00 2024-03-26T08:17:30.368 INFO:teuthology.orchestra.run.smithi043.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 390 kB/s | 36 kB 00:00 2024-03-26T08:17:30.417 INFO:teuthology.orchestra.run.smithi052.stdout:(1/174): ceph-19.0.0-2145.g3be9a5c9.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-03-26T08:17:30.436 INFO:teuthology.orchestra.run.smithi043.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 438 kB/s | 51 kB 00:00 2024-03-26T08:17:30.469 INFO:teuthology.orchestra.run.smithi043.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-03-26T08:17:30.494 INFO:teuthology.orchestra.run.smithi043.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 718 kB/s | 90 kB 00:00 2024-03-26T08:17:30.536 INFO:teuthology.orchestra.run.smithi043.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-03-26T08:17:30.556 INFO:teuthology.orchestra.run.smithi043.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 231 kB/s | 20 kB 00:00 2024-03-26T08:17:30.615 INFO:teuthology.orchestra.run.smithi043.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-03-26T08:17:30.690 INFO:teuthology.orchestra.run.smithi043.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-03-26T08:17:30.732 INFO:teuthology.orchestra.run.smithi043.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.7 MB/s | 321 kB 00:00 2024-03-26T08:17:30.891 INFO:teuthology.orchestra.run.smithi183.stdout:Extra Packages for Enterprise Linux 811 kB/s | 21 MB 00:26 2024-03-26T08:17:30.941 INFO:teuthology.orchestra.run.smithi043.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 11 MB/s | 2.3 MB 00:00 2024-03-26T08:17:31.041 INFO:teuthology.orchestra.run.smithi043.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 19 MB/s | 1.8 MB 00:00 2024-03-26T08:17:31.083 INFO:teuthology.orchestra.run.smithi043.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 1.0 MB/s | 42 kB 00:00 2024-03-26T08:17:31.134 INFO:teuthology.orchestra.run.smithi043.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 4.9 MB/s | 248 kB 00:00 2024-03-26T08:17:31.192 INFO:teuthology.orchestra.run.smithi043.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2024-03-26T08:17:31.201 INFO:teuthology.orchestra.run.smithi052.stdout:(2/174): ceph-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.1 MB/s | 890 kB 00:00 2024-03-26T08:17:31.234 INFO:teuthology.orchestra.run.smithi043.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 889 kB/s | 36 kB 00:00 2024-03-26T08:17:31.284 INFO:teuthology.orchestra.run.smithi043.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.8 MB/s | 192 kB 00:00 2024-03-26T08:17:31.334 INFO:teuthology.orchestra.run.smithi043.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 5.1 MB/s | 256 kB 00:00 2024-03-26T08:17:31.377 INFO:teuthology.orchestra.run.smithi043.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 561 kB/s | 23 kB 00:00 2024-03-26T08:17:31.409 INFO:teuthology.orchestra.run.smithi052.stdout:(3/174): ceph-immutable-object-cache-19.0.0-214 702 kB/s | 145 kB 00:00 2024-03-26T08:17:31.469 INFO:teuthology.orchestra.run.smithi043.stdout:(161/174): librados2-19.0.0-2145.g3be9a5c9.el9. 3.7 MB/s | 3.4 MB 00:00 2024-03-26T08:17:31.493 INFO:teuthology.orchestra.run.smithi043.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 936 kB/s | 108 kB 00:00 2024-03-26T08:17:31.569 INFO:teuthology.orchestra.run.smithi043.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 136 kB 00:00 2024-03-26T08:17:31.603 INFO:teuthology.orchestra.run.smithi043.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 155 kB 00:00 2024-03-26T08:17:31.628 INFO:teuthology.orchestra.run.smithi043.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.1 MB/s | 64 kB 00:00 2024-03-26T08:17:31.653 INFO:teuthology.orchestra.run.smithi043.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 582 kB/s | 29 kB 00:00 2024-03-26T08:17:31.668 INFO:teuthology.orchestra.run.smithi052.stdout:(4/174): ceph-common-19.0.0-2145.g3be9a5c9.el9. 13 MB/s | 20 MB 00:01 2024-03-26T08:17:31.712 INFO:teuthology.orchestra.run.smithi043.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 15 MB/s | 1.2 MB 00:00 2024-03-26T08:17:31.810 INFO:teuthology.orchestra.run.smithi052.stdout:(5/174): ceph-mgr-19.0.0-2145.g3be9a5c9.el9.x86 11 MB/s | 1.5 MB 00:00 2024-03-26T08:17:31.812 INFO:teuthology.orchestra.run.smithi043.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 14 MB/s | 2.2 MB 00:00 2024-03-26T08:17:31.896 INFO:teuthology.orchestra.run.smithi043.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 12 MB/s | 2.3 MB 00:00 2024-03-26T08:17:31.959 INFO:teuthology.orchestra.run.smithi043.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 3.1 MB/s | 465 kB 00:00 2024-03-26T08:17:31.988 INFO:teuthology.orchestra.run.smithi043.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 342 kB/s | 31 kB 00:00 2024-03-26T08:17:32.038 INFO:teuthology.orchestra.run.smithi043.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 14 MB/s | 1.1 MB 00:00 2024-03-26T08:17:32.063 INFO:teuthology.orchestra.run.smithi043.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.0 MB/s | 74 kB 00:00 2024-03-26T08:17:32.135 INFO:teuthology.orchestra.run.smithi052.stdout:(6/174): ceph-mon-19.0.0-2145.g3be9a5c9.el9.x86 14 MB/s | 4.7 MB 00:00 2024-03-26T08:17:32.202 INFO:teuthology.orchestra.run.smithi052.stdout:(7/174): ceph-mds-19.0.0-2145.g3be9a5c9.el9.x86 2.9 MB/s | 2.3 MB 00:00 2024-03-26T08:17:32.536 INFO:teuthology.orchestra.run.smithi052.stdout:(8/174): ceph-base-19.0.0-2145.g3be9a5c9.el9.x8 2.2 MB/s | 5.4 MB 00:02 2024-03-26T08:17:32.628 INFO:teuthology.orchestra.run.smithi052.stdout:(9/174): ceph-selinux-19.0.0-2145.g3be9a5c9.el9 277 kB/s | 25 kB 00:00 2024-03-26T08:17:33.153 INFO:teuthology.orchestra.run.smithi052.stdout:(10/174): ceph-osd-19.0.0-2145.g3be9a5c9.el9.x8 17 MB/s | 17 MB 00:01 2024-03-26T08:17:33.211 INFO:teuthology.orchestra.run.smithi052.stdout:(11/174): cephfs-mirror-19.0.0-2145.g3be9a5c9.e 3.8 MB/s | 220 kB 00:00 2024-03-26T08:17:33.230 INFO:teuthology.orchestra.run.smithi043.stdout:(174/174): librbd1-19.0.0-2145.g3be9a5c9.el9.x8 1.2 MB/s | 3.1 MB 00:02 2024-03-26T08:17:33.247 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-03-26T08:17:33.247 INFO:teuthology.orchestra.run.smithi043.stdout:Total 13 MB/s | 225 MB 00:17 2024-03-26T08:17:33.270 INFO:teuthology.orchestra.run.smithi052.stdout:(12/174): cephfs-shell-19.0.0-2145.g3be9a5c9.el 473 kB/s | 27 kB 00:00 2024-03-26T08:17:33.320 INFO:teuthology.orchestra.run.smithi052.stdout:(13/174): libcephfs-devel-19.0.0-2145.g3be9a5c9 636 kB/s | 31 kB 00:00 2024-03-26T08:17:33.421 INFO:teuthology.orchestra.run.smithi052.stdout:(14/174): libcephfs2-19.0.0-2145.g3be9a5c9.el9. 7.4 MB/s | 746 kB 00:00 2024-03-26T08:17:33.471 INFO:teuthology.orchestra.run.smithi052.stdout:(15/174): libcephsqlite-19.0.0-2145.g3be9a5c9.e 3.3 MB/s | 163 kB 00:00 2024-03-26T08:17:33.529 INFO:teuthology.orchestra.run.smithi052.stdout:(16/174): librados-devel-19.0.0-2145.g3be9a5c9. 2.2 MB/s | 127 kB 00:00 2024-03-26T08:17:33.713 INFO:teuthology.orchestra.run.smithi052.stdout:(17/174): ceph-radosgw-19.0.0-2145.g3be9a5c9.el 6.5 MB/s | 9.9 MB 00:01 2024-03-26T08:17:33.763 INFO:teuthology.orchestra.run.smithi052.stdout:(18/174): libradosstriper1-19.0.0-2145.g3be9a5c 2.1 MB/s | 501 kB 00:00 2024-03-26T08:17:33.813 INFO:teuthology.orchestra.run.smithi052.stdout:(19/174): python3-ceph-argparse-19.0.0-2145.g3b 924 kB/s | 45 kB 00:00 2024-03-26T08:17:33.881 INFO:teuthology.orchestra.run.smithi052.stdout:(20/174): python3-ceph-common-19.0.0-2145.g3be9 2.0 MB/s | 131 kB 00:00 2024-03-26T08:17:33.956 INFO:teuthology.orchestra.run.smithi052.stdout:(21/174): python3-cephfs-19.0.0-2145.g3be9a5c9. 2.1 MB/s | 162 kB 00:00 2024-03-26T08:17:34.039 INFO:teuthology.orchestra.run.smithi052.stdout:(22/174): python3-rados-19.0.0-2145.g3be9a5c9.e 3.8 MB/s | 321 kB 00:00 2024-03-26T08:17:34.090 INFO:teuthology.orchestra.run.smithi052.stdout:(23/174): python3-rbd-19.0.0-2145.g3be9a5c9.el9 6.0 MB/s | 298 kB 00:00 2024-03-26T08:17:34.232 INFO:teuthology.orchestra.run.smithi052.stdout:(24/174): librgw2-19.0.0-2145.g3be9a5c9.el9.x86 10 MB/s | 5.1 MB 00:00 2024-03-26T08:17:34.265 INFO:teuthology.orchestra.run.smithi052.stdout:(25/174): python3-rgw-19.0.0-2145.g3be9a5c9.el9 570 kB/s | 99 kB 00:00 2024-03-26T08:17:34.299 INFO:teuthology.orchestra.run.smithi052.stdout:(26/174): rbd-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.3 MB/s | 85 kB 00:00 2024-03-26T08:17:34.382 INFO:teuthology.orchestra.run.smithi052.stdout:(27/174): rbd-nbd-19.0.0-2145.g3be9a5c9.el9.x86 2.0 MB/s | 170 kB 00:00 2024-03-26T08:17:34.474 INFO:teuthology.orchestra.run.smithi052.stdout:(28/174): rbd-mirror-19.0.0-2145.g3be9a5c9.el9. 15 MB/s | 3.1 MB 00:00 2024-03-26T08:17:34.500 INFO:teuthology.orchestra.run.smithi052.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2145.g 292 kB/s | 34 kB 00:00 2024-03-26T08:17:34.525 INFO:teuthology.orchestra.run.smithi052.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2145.g3be9a5c 2.8 MB/s | 139 kB 00:00 2024-03-26T08:17:34.725 INFO:teuthology.orchestra.run.smithi052.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2145.g3be9a 8.0 MB/s | 1.8 MB 00:00 2024-03-26T08:17:34.834 INFO:teuthology.orchestra.run.smithi052.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2145.g3b 2.3 MB/s | 248 kB 00:00 2024-03-26T08:17:34.955 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-03-26T08:17:35.001 INFO:teuthology.orchestra.run.smithi052.stdout:(33/174): ceph-mgr-diskprediction-local-19.0.0- 16 MB/s | 7.4 MB 00:00 2024-03-26T08:17:35.026 INFO:teuthology.orchestra.run.smithi052.stdout:(34/174): ceph-mgr-rook-19.0.0-2145.g3be9a5c9.e 258 kB/s | 49 kB 00:00 2024-03-26T08:17:35.043 INFO:teuthology.orchestra.run.smithi052.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2145.g3 403 kB/s | 16 kB 00:00 2024-03-26T08:17:35.101 INFO:teuthology.orchestra.run.smithi052.stdout:(36/174): ceph-volume-19.0.0-2145.g3be9a5c9.el9 3.5 MB/s | 264 kB 00:00 2024-03-26T08:17:35.144 INFO:teuthology.orchestra.run.smithi052.stdout:(37/174): cephadm-19.0.0-2145.g3be9a5c9.el9.noa 7.3 MB/s | 748 kB 00:00 2024-03-26T08:17:35.169 INFO:teuthology.orchestra.run.smithi052.stdout:(38/174): cephfs-top-19.0.0-2145.g3be9a5c9.el9. 274 kB/s | 18 kB 00:00 2024-03-26T08:17:35.363 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-03-26T08:17:35.363 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-03-26T08:17:35.394 INFO:teuthology.orchestra.run.smithi052.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.6 MB/s | 662 kB 00:00 2024-03-26T08:17:35.445 INFO:teuthology.orchestra.run.smithi052.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 4.4 MB/s | 920 kB 00:00 2024-03-26T08:17:35.470 INFO:teuthology.orchestra.run.smithi052.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.5 MB/s | 106 kB 00:00 2024-03-26T08:17:35.503 INFO:teuthology.orchestra.run.smithi052.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 307 kB 00:00 2024-03-26T08:17:35.520 INFO:teuthology.orchestra.run.smithi052.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 679 kB/s | 32 kB 00:00 2024-03-26T08:17:35.621 INFO:teuthology.orchestra.run.smithi052.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-03-26T08:17:35.638 INFO:teuthology.orchestra.run.smithi052.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 175 kB/s | 17 kB 00:00 2024-03-26T08:17:35.680 INFO:teuthology.orchestra.run.smithi052.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.9 MB/s | 159 kB 00:00 2024-03-26T08:17:35.705 INFO:teuthology.orchestra.run.smithi052.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 153 kB/s | 10 kB 00:00 2024-03-26T08:17:35.722 INFO:teuthology.orchestra.run.smithi052.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 44 kB 00:00 2024-03-26T08:17:35.764 INFO:teuthology.orchestra.run.smithi052.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-03-26T08:17:35.806 INFO:teuthology.orchestra.run.smithi052.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 7.4 MB/s | 578 kB 00:00 2024-03-26T08:17:35.831 INFO:teuthology.orchestra.run.smithi052.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 761 kB/s | 46 kB 00:00 2024-03-26T08:17:35.865 INFO:teuthology.orchestra.run.smithi052.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2024-03-26T08:17:35.898 INFO:teuthology.orchestra.run.smithi052.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.9 MB/s | 187 kB 00:00 2024-03-26T08:17:35.934 INFO:teuthology.orchestra.run.smithi052.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2024-03-26T08:17:35.949 INFO:teuthology.orchestra.run.smithi052.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 729 kB/s | 34 kB 00:00 2024-03-26T08:17:36.107 INFO:teuthology.orchestra.run.smithi052.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-03-26T08:17:36.149 INFO:teuthology.orchestra.run.smithi052.stdout:(57/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.7 MB/s | 1.0 MB 00:00 2024-03-26T08:17:36.175 INFO:teuthology.orchestra.run.smithi052.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-26T08:17:36.367 INFO:teuthology.orchestra.run.smithi052.stdout:(59/174): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 5.8 MB 00:00 2024-03-26T08:17:36.392 INFO:teuthology.orchestra.run.smithi052.stdout:(60/174): python3-jsonpatch-1.21-16.el9.noarch. 126 kB/s | 24 kB 00:00 2024-03-26T08:17:36.434 INFO:teuthology.orchestra.run.smithi052.stdout:(61/174): python3-jsonpointer-2.0-4.el9.noarch. 447 kB/s | 18 kB 00:00 2024-03-26T08:17:36.459 INFO:teuthology.orchestra.run.smithi052.stdout:(62/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 144 kB 00:00 2024-03-26T08:17:36.492 INFO:teuthology.orchestra.run.smithi052.stdout:(63/174): python3-markupsafe-1.1.1-12.el9.x86_6 980 kB/s | 32 kB 00:00 2024-03-26T08:17:36.626 INFO:teuthology.orchestra.run.smithi052.stdout:(64/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 4.8 MB 00:00 2024-03-26T08:17:36.640 INFO:teuthology.orchestra.run.smithi183.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-03-26T08:17:36.651 INFO:teuthology.orchestra.run.smithi052.stdout:(65/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 217 kB 00:00 2024-03-26T08:17:36.693 INFO:teuthology.orchestra.run.smithi052.stdout:(66/174): python3-oauthlib-3.1.1-5.el9.noarch.r 4.2 MB/s | 174 kB 00:00 2024-03-26T08:17:36.718 INFO:teuthology.orchestra.run.smithi052.stdout:(67/174): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 69 kB 00:00 2024-03-26T08:17:36.752 INFO:teuthology.orchestra.run.smithi052.stdout:(68/174): python3-prettytable-0.7.2-27.el9.noar 1.3 MB/s | 41 kB 00:00 2024-03-26T08:17:36.786 INFO:teuthology.orchestra.run.smithi052.stdout:(69/174): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-03-26T08:17:36.811 INFO:teuthology.orchestra.run.smithi052.stdout:(70/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 132 kB 00:00 2024-03-26T08:17:36.845 INFO:teuthology.orchestra.run.smithi052.stdout:(71/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 211 kB 00:00 2024-03-26T08:17:36.862 INFO:teuthology.orchestra.run.smithi052.stdout:(72/174): python3-pytz-2021.1-5.el9.noarch.rpm 991 kB/s | 47 kB 00:00 2024-03-26T08:17:36.879 INFO:teuthology.orchestra.run.smithi052.stdout:(73/174): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 43 kB 00:00 2024-03-26T08:17:37.062 INFO:teuthology.orchestra.run.smithi052.stdout:(74/174): python3-toml-0.10.2-6.el9.noarch.rpm 245 kB/s | 39 kB 00:00 2024-03-26T08:17:37.246 INFO:teuthology.orchestra.run.smithi052.stdout:(75/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2024-03-26T08:17:37.263 INFO:teuthology.orchestra.run.smithi052.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 210 kB/s | 41 kB 00:00 2024-03-26T08:17:37.296 INFO:teuthology.orchestra.run.smithi052.stdout:(77/174): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 300 kB 00:00 2024-03-26T08:17:37.313 INFO:teuthology.orchestra.run.smithi052.stdout:(78/174): uuid-1.6.2-55.el9.x86_64.rpm 1.2 MB/s | 56 kB 00:00 2024-03-26T08:17:37.330 INFO:teuthology.orchestra.run.smithi052.stdout:(79/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.9 MB/s | 63 kB 00:00 2024-03-26T08:17:37.347 INFO:teuthology.orchestra.run.smithi052.stdout:(80/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-26T08:17:37.373 INFO:teuthology.orchestra.run.smithi052.stdout:(81/174): fuse-2.9.9-15.el9.x86_64.rpm 1.9 MB/s | 79 kB 00:00 2024-03-26T08:17:37.406 INFO:teuthology.orchestra.run.smithi052.stdout:(82/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.1 MB/s | 241 kB 00:00 2024-03-26T08:17:37.465 INFO:teuthology.orchestra.run.smithi052.stdout:(83/174): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.1 MB 00:00 2024-03-26T08:17:37.490 INFO:teuthology.orchestra.run.smithi052.stdout:(84/174): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 103 kB 00:00 2024-03-26T08:17:37.515 INFO:teuthology.orchestra.run.smithi052.stdout:(85/174): python3-pycparser-2.20-6.el9.noarch.r 2.5 MB/s | 124 kB 00:00 2024-03-26T08:17:37.549 INFO:teuthology.orchestra.run.smithi052.stdout:(86/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2024-03-26T08:17:37.741 INFO:teuthology.orchestra.run.smithi052.stdout:(87/174): jq-1.6-16.el9.x86_64.rpm 841 kB/s | 187 kB 00:00 2024-03-26T08:17:37.833 INFO:teuthology.orchestra.run.smithi052.stdout:(88/174): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-03-26T08:17:37.900 INFO:teuthology.orchestra.run.smithi052.stdout:(89/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.2 MB/s | 803 kB 00:00 2024-03-26T08:17:37.929 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-03-26T08:17:37.933 INFO:teuthology.orchestra.run.smithi052.stdout:(90/174): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-03-26T08:17:37.950 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-03-26T08:17:38.159 INFO:teuthology.orchestra.run.smithi052.stdout:(91/174): elfutils-libelf-devel-0.190-2.el9.x86 83 kB/s | 21 kB 00:00 2024-03-26T08:17:38.193 INFO:teuthology.orchestra.run.smithi052.stdout:(92/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 99 kB/s | 25 kB 00:00 2024-03-26T08:17:38.910 INFO:teuthology.orchestra.run.smithi052.stdout:(93/174): postgresql-13.14-1.el9.x86_64.rpm 2.2 MB/s | 1.6 MB 00:00 2024-03-26T08:17:39.051 INFO:teuthology.orchestra.run.smithi052.stdout:(94/174): postgresql-contrib-13.14-1.el9.x86_64 6.3 MB/s | 911 kB 00:00 2024-03-26T08:17:39.168 INFO:teuthology.orchestra.run.smithi052.stdout:(95/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 4.1 MB/s | 4.1 MB 00:01 2024-03-26T08:17:39.202 INFO:teuthology.orchestra.run.smithi052.stdout:(96/174): postgresql-private-libs-13.14-1.el9.x 893 kB/s | 133 kB 00:00 2024-03-26T08:17:39.284 INFO:teuthology.orchestra.run.smithi183.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T08:17:39.285 INFO:teuthology.orchestra.run.smithi183.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T08:17:39.302 INFO:teuthology.orchestra.run.smithi183.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-26T08:17:39.303 INFO:teuthology.orchestra.run.smithi183.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-26T08:17:39.304 INFO:teuthology.orchestra.run.smithi183.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-26T08:17:39.306 INFO:teuthology.orchestra.run.smithi183.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-26T08:17:39.308 INFO:teuthology.orchestra.run.smithi183.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-26T08:17:39.311 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-26T08:17:39.312 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-26T08:17:39.319 INFO:teuthology.orchestra.run.smithi052.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.1 MB/s | 245 kB 00:00 2024-03-26T08:17:39.397 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-03-26T08:17:39.418 INFO:teuthology.orchestra.run.smithi183.stdout:============================================================================================ 2024-03-26T08:17:39.418 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2024-03-26T08:17:39.418 INFO:teuthology.orchestra.run.smithi183.stdout:============================================================================================ 2024-03-26T08:17:39.418 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-03-26T08:17:39.418 INFO:teuthology.orchestra.run.smithi183.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-26T08:17:39.418 INFO:teuthology.orchestra.run.smithi183.stdout: ceph x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 6.6 k 2024-03-26T08:17:39.418 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.4 M 2024-03-26T08:17:39.419 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 890 k 2024-03-26T08:17:39.419 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 145 k 2024-03-26T08:17:39.419 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 1.5 M 2024-03-26T08:17:39.419 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 139 k 2024-03-26T08:17:39.419 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 1.8 M 2024-03-26T08:17:39.419 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 7.4 M 2024-03-26T08:17:39.419 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 49 k 2024-03-26T08:17:39.419 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 9.9 M 2024-03-26T08:17:39.419 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 48 M 2024-03-26T08:17:39.420 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 264 k 2024-03-26T08:17:39.420 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 748 k 2024-03-26T08:17:39.420 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 220 k 2024-03-26T08:17:39.420 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 27 k 2024-03-26T08:17:39.420 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 18 k 2024-03-26T08:17:39.420 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-26T08:17:39.420 INFO:teuthology.orchestra.run.smithi183.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-26T08:17:39.420 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-26T08:17:39.421 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 31 k 2024-03-26T08:17:39.421 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 746 k 2024-03-26T08:17:39.421 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 127 k 2024-03-26T08:17:39.421 INFO:teuthology.orchestra.run.smithi183.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-26T08:17:39.421 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-26T08:17:39.421 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-26T08:17:39.421 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-26T08:17:39.421 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-26T08:17:39.422 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-26T08:17:39.422 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 162 k 2024-03-26T08:17:39.422 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 321 k 2024-03-26T08:17:39.422 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 298 k 2024-03-26T08:17:39.422 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 99 k 2024-03-26T08:17:39.422 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 85 k 2024-03-26T08:17:39.422 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-26T08:17:39.422 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 170 k 2024-03-26T08:17:39.422 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-26T08:17:39.422 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrading: 2024-03-26T08:17:39.422 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-26T08:17:39.423 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-26T08:17:39.423 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-26T08:17:39.423 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-26T08:17:39.423 INFO:teuthology.orchestra.run.smithi183.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-26T08:17:39.423 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-26T08:17:39.423 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-26T08:17:39.423 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-26T08:17:39.423 INFO:teuthology.orchestra.run.smithi183.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-26T08:17:39.423 INFO:teuthology.orchestra.run.smithi183.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-26T08:17:39.423 INFO:teuthology.orchestra.run.smithi183.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-26T08:17:39.423 INFO:teuthology.orchestra.run.smithi183.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-26T08:17:39.423 INFO:teuthology.orchestra.run.smithi183.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-26T08:17:39.423 INFO:teuthology.orchestra.run.smithi183.stdout: librados2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.4 M 2024-03-26T08:17:39.424 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-26T08:17:39.424 INFO:teuthology.orchestra.run.smithi183.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-26T08:17:39.424 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-26T08:17:39.424 INFO:teuthology.orchestra.run.smithi183.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-26T08:17:39.424 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-26T08:17:39.424 INFO:teuthology.orchestra.run.smithi183.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-26T08:17:39.424 INFO:teuthology.orchestra.run.smithi183.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-26T08:17:39.424 INFO:teuthology.orchestra.run.smithi183.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-26T08:17:39.424 INFO:teuthology.orchestra.run.smithi183.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-26T08:17:39.424 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-26T08:17:39.424 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2024-03-26T08:17:39.424 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-26T08:17:39.424 INFO:teuthology.orchestra.run.smithi183.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-26T08:17:39.425 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-26T08:17:39.425 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 20 M 2024-03-26T08:17:39.425 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 34 k 2024-03-26T08:17:39.425 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 2.3 M 2024-03-26T08:17:39.425 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 248 k 2024-03-26T08:17:39.425 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 4.7 M 2024-03-26T08:17:39.425 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 17 M 2024-03-26T08:17:39.425 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 16 k 2024-03-26T08:17:39.425 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 25 k 2024-03-26T08:17:39.425 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-26T08:17:39.425 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-26T08:17:39.425 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-26T08:17:39.425 INFO:teuthology.orchestra.run.smithi183.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-26T08:17:39.426 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-26T08:17:39.426 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-26T08:17:39.426 INFO:teuthology.orchestra.run.smithi183.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-26T08:17:39.426 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-26T08:17:39.426 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-26T08:17:39.426 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 163 k 2024-03-26T08:17:39.426 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-26T08:17:39.426 INFO:teuthology.orchestra.run.smithi183.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-26T08:17:39.426 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-26T08:17:39.426 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-26T08:17:39.426 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-26T08:17:39.426 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 501 k 2024-03-26T08:17:39.427 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-26T08:17:39.427 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.1 M 2024-03-26T08:17:39.427 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-26T08:17:39.427 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-26T08:17:39.427 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-26T08:17:39.427 INFO:teuthology.orchestra.run.smithi183.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-26T08:17:39.427 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-26T08:17:39.427 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-26T08:17:39.427 INFO:teuthology.orchestra.run.smithi183.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-26T08:17:39.427 INFO:teuthology.orchestra.run.smithi183.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-26T08:17:39.427 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-26T08:17:39.427 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-26T08:17:39.427 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-26T08:17:39.428 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-26T08:17:39.428 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-26T08:17:39.428 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-26T08:17:39.428 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-26T08:17:39.428 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-26T08:17:39.428 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-26T08:17:39.428 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-26T08:17:39.428 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 45 k 2024-03-26T08:17:39.428 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 131 k 2024-03-26T08:17:39.428 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-26T08:17:39.428 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-26T08:17:39.428 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-26T08:17:39.428 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-26T08:17:39.429 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-26T08:17:39.429 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-26T08:17:39.429 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-26T08:17:39.429 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-26T08:17:39.429 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-26T08:17:39.429 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-26T08:17:39.429 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-26T08:17:39.429 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-26T08:17:39.429 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-26T08:17:39.429 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-26T08:17:39.429 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-26T08:17:39.429 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-26T08:17:39.429 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-26T08:17:39.429 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-26T08:17:39.430 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-26T08:17:39.430 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-26T08:17:39.430 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-26T08:17:39.430 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-26T08:17:39.430 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-26T08:17:39.430 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-26T08:17:39.430 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-26T08:17:39.430 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-26T08:17:39.430 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-26T08:17:39.430 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-26T08:17:39.430 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-26T08:17:39.430 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-26T08:17:39.430 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-26T08:17:39.431 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-26T08:17:39.431 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-26T08:17:39.431 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-26T08:17:39.431 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-26T08:17:39.431 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-26T08:17:39.431 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-26T08:17:39.431 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-26T08:17:39.431 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-26T08:17:39.431 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-26T08:17:39.432 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-26T08:17:39.432 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-26T08:17:39.432 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-26T08:17:39.432 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-26T08:17:39.432 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-26T08:17:39.432 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-26T08:17:39.432 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-26T08:17:39.432 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-26T08:17:39.432 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-26T08:17:39.432 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-26T08:17:39.432 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-26T08:17:39.433 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-26T08:17:39.433 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-26T08:17:39.433 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-26T08:17:39.433 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-26T08:17:39.433 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-26T08:17:39.433 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-26T08:17:39.433 INFO:teuthology.orchestra.run.smithi183.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-26T08:17:39.433 INFO:teuthology.orchestra.run.smithi183.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-26T08:17:39.433 INFO:teuthology.orchestra.run.smithi183.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-26T08:17:39.433 INFO:teuthology.orchestra.run.smithi183.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-26T08:17:39.433 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-26T08:17:39.433 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-26T08:17:39.433 INFO:teuthology.orchestra.run.smithi183.stdout:Installing weak dependencies: 2024-03-26T08:17:39.434 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-26T08:17:39.434 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-26T08:17:39.434 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-03-26T08:17:39.434 INFO:teuthology.orchestra.run.smithi183.stdout:============================================================================================ 2024-03-26T08:17:39.434 INFO:teuthology.orchestra.run.smithi183.stdout:Install 150 Packages 2024-03-26T08:17:39.434 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrade 24 Packages 2024-03-26T08:17:39.434 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-26T08:17:39.435 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 225 M 2024-03-26T08:17:39.435 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-03-26T08:17:39.436 INFO:teuthology.orchestra.run.smithi052.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-03-26T08:17:39.561 INFO:teuthology.orchestra.run.smithi052.stdout:(99/174): qatlib-24.02.0-1.el9.x86_64.rpm 1.7 MB/s | 221 kB 00:00 2024-03-26T08:17:39.670 INFO:teuthology.orchestra.run.smithi052.stdout:(100/174): qatlib-service-24.02.0-1.el9.x86_64. 334 kB/s | 36 kB 00:00 2024-03-26T08:17:39.779 INFO:teuthology.orchestra.run.smithi052.stdout:(101/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 442 kB/s | 47 kB 00:00 2024-03-26T08:17:39.896 INFO:teuthology.orchestra.run.smithi052.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 811 kB/s | 94 kB 00:00 2024-03-26T08:17:40.004 INFO:teuthology.orchestra.run.smithi052.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 208 kB/s | 22 kB 00:00 2024-03-26T08:17:40.146 INFO:teuthology.orchestra.run.smithi052.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 6.0 MB/s | 863 kB 00:00 2024-03-26T08:17:40.280 INFO:teuthology.orchestra.run.smithi052.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 4.0 MB/s | 548 kB 00:00 2024-03-26T08:17:40.322 INFO:teuthology.orchestra.run.smithi052.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 7.4 MB/s | 309 kB 00:00 2024-03-26T08:17:40.388 INFO:teuthology.orchestra.run.smithi052.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 298 kB/s | 19 kB 00:00 2024-03-26T08:17:40.572 INFO:teuthology.orchestra.run.smithi052.stdout:(108/174): libarrow-9.0.0-10.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-03-26T08:17:40.606 INFO:teuthology.orchestra.run.smithi052.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 807 kB/s | 26 kB 00:00 2024-03-26T08:17:40.616 INFO:teuthology.orchestra.run.smithi183.stdout:(1/174): ceph-19.0.0-2145.g3be9a5c9.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-03-26T08:17:40.640 INFO:teuthology.orchestra.run.smithi052.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-03-26T08:17:40.673 INFO:teuthology.orchestra.run.smithi052.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-03-26T08:17:40.707 INFO:teuthology.orchestra.run.smithi052.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 4.5 MB/s | 151 kB 00:00 2024-03-26T08:17:40.782 INFO:teuthology.orchestra.run.smithi052.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 11 MB/s | 838 kB 00:00 2024-03-26T08:17:40.832 INFO:teuthology.orchestra.run.smithi052.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-03-26T08:17:40.866 INFO:teuthology.orchestra.run.smithi052.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-03-26T08:17:40.899 INFO:teuthology.orchestra.run.smithi052.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 986 kB/s | 32 kB 00:00 2024-03-26T08:17:40.925 INFO:teuthology.orchestra.run.smithi052.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 614 kB/s | 15 kB 00:00 2024-03-26T08:17:40.967 INFO:teuthology.orchestra.run.smithi052.stdout:(118/174): python3-cheroot-8.6.0-4.el9.noarch.r 4.1 MB/s | 172 kB 00:00 2024-03-26T08:17:41.017 INFO:teuthology.orchestra.run.smithi052.stdout:(119/174): python3-cherrypy-18.6.1-2.el9.noarch 7.1 MB/s | 358 kB 00:00 2024-03-26T08:17:41.067 INFO:teuthology.orchestra.run.smithi052.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 6.0 MB/s | 303 kB 00:00 2024-03-26T08:17:41.092 INFO:teuthology.orchestra.run.smithi183.stdout:(2/174): ceph-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.8 MB/s | 890 kB 00:00 2024-03-26T08:17:41.101 INFO:teuthology.orchestra.run.smithi052.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 1.1 MB/s | 35 kB 00:00 2024-03-26T08:17:41.143 INFO:teuthology.orchestra.run.smithi052.stdout:(122/174): python3-google-auth-2.28.2-1.el9.noa 4.9 MB/s | 203 kB 00:00 2024-03-26T08:17:41.200 INFO:teuthology.orchestra.run.smithi183.stdout:(3/174): ceph-immutable-object-cache-19.0.0-214 1.3 MB/s | 145 kB 00:00 2024-03-26T08:17:41.253 INFO:teuthology.orchestra.run.smithi052.stdout:(123/174): python3-grpcio-1.46.7-10.el9.x86_64. 19 MB/s | 2.0 MB 00:00 2024-03-26T08:17:41.294 INFO:teuthology.orchestra.run.smithi052.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 3.4 MB/s | 144 kB 00:00 2024-03-26T08:17:41.319 INFO:teuthology.orchestra.run.smithi052.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 438 kB/s | 11 kB 00:00 2024-03-26T08:17:41.345 INFO:teuthology.orchestra.run.smithi052.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 727 kB/s | 18 kB 00:00 2024-03-26T08:17:41.378 INFO:teuthology.orchestra.run.smithi052.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 712 kB/s | 23 kB 00:00 2024-03-26T08:17:41.384 INFO:teuthology.orchestra.run.smithi183.stdout:(4/174): ceph-base-19.0.0-2145.g3be9a5c9.el9.x8 4.9 MB/s | 5.4 MB 00:01 2024-03-26T08:17:41.403 INFO:teuthology.orchestra.run.smithi052.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 799 kB/s | 19 kB 00:00 2024-03-26T08:17:41.429 INFO:teuthology.orchestra.run.smithi052.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 820 kB/s | 20 kB 00:00 2024-03-26T08:17:41.496 INFO:teuthology.orchestra.run.smithi052.stdout:(130/174): python3-kubernetes-26.1.0-2.el9.noar 15 MB/s | 1.0 MB 00:00 2024-03-26T08:17:41.538 INFO:teuthology.orchestra.run.smithi052.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-03-26T08:17:41.580 INFO:teuthology.orchestra.run.smithi052.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 1.9 MB/s | 79 kB 00:00 2024-03-26T08:17:41.592 INFO:teuthology.orchestra.run.smithi183.stdout:(5/174): ceph-mgr-19.0.0-2145.g3be9a5c9.el9.x86 7.2 MB/s | 1.5 MB 00:00 2024-03-26T08:17:41.622 INFO:teuthology.orchestra.run.smithi052.stdout:(133/174): python3-natsort-7.1.1-5.el9.noarch.r 1.4 MB/s | 58 kB 00:00 2024-03-26T08:17:41.665 INFO:teuthology.orchestra.run.smithi052.stdout:(134/174): python3-pecan-1.4.2-3.el9.noarch.rpm 6.5 MB/s | 272 kB 00:00 2024-03-26T08:17:41.697 INFO:teuthology.orchestra.run.smithi052.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 505 kB/s | 16 kB 00:00 2024-03-26T08:17:41.731 INFO:teuthology.orchestra.run.smithi052.stdout:(136/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.7 MB/s | 90 kB 00:00 2024-03-26T08:17:41.773 INFO:teuthology.orchestra.run.smithi052.stdout:(137/174): python3-pyperclip-1.8.2-2.el9.noarch 686 kB/s | 28 kB 00:00 2024-03-26T08:17:41.815 INFO:teuthology.orchestra.run.smithi052.stdout:(138/174): python3-repoze-lru-0.7-16.el9.noarch 755 kB/s | 31 kB 00:00 2024-03-26T08:17:41.857 INFO:teuthology.orchestra.run.smithi052.stdout:(139/174): python3-routes-2.5.1-5.el9.noarch.rp 4.5 MB/s | 188 kB 00:00 2024-03-26T08:17:41.905 INFO:teuthology.orchestra.run.smithi052.stdout:(140/174): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-03-26T08:17:41.939 INFO:teuthology.orchestra.run.smithi052.stdout:(141/174): python3-tempora-5.0.0-2.el9.noarch.r 1.1 MB/s | 36 kB 00:00 2024-03-26T08:17:41.981 INFO:teuthology.orchestra.run.smithi052.stdout:(142/174): python3-typing-extensions-4.4.0-2.el 1.2 MB/s | 51 kB 00:00 2024-03-26T08:17:42.043 INFO:teuthology.orchestra.run.smithi052.stdout:(143/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.7 MB/s | 230 kB 00:00 2024-03-26T08:17:42.090 INFO:teuthology.orchestra.run.smithi052.stdout:(144/174): python3-websocket-client-1.2.3-2.el9 1.9 MB/s | 90 kB 00:00 2024-03-26T08:17:42.141 INFO:teuthology.orchestra.run.smithi052.stdout:(145/174): python3-werkzeug-2.0.3-3.el9.1.noarc 8.5 MB/s | 427 kB 00:00 2024-03-26T08:17:42.159 INFO:teuthology.orchestra.run.smithi183.stdout:(6/174): ceph-mon-19.0.0-2145.g3be9a5c9.el9.x86 8.3 MB/s | 4.7 MB 00:00 2024-03-26T08:17:42.182 INFO:teuthology.orchestra.run.smithi052.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 489 kB/s | 20 kB 00:00 2024-03-26T08:17:42.218 INFO:teuthology.orchestra.run.smithi183.stdout:(7/174): ceph-mds-19.0.0-2145.g3be9a5c9.el9.x86 2.3 MB/s | 2.3 MB 00:01 2024-03-26T08:17:42.224 INFO:teuthology.orchestra.run.smithi052.stdout:(147/174): re2-20211101-3.el9.x86_64.rpm 4.6 MB/s | 192 kB 00:00 2024-03-26T08:17:42.317 INFO:teuthology.orchestra.run.smithi052.stdout:(148/174): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-03-26T08:17:42.743 INFO:teuthology.orchestra.run.smithi183.stdout:(8/174): ceph-common-19.0.0-2145.g3be9a5c9.el9. 8.4 MB/s | 20 MB 00:02 2024-03-26T08:17:42.802 INFO:teuthology.orchestra.run.smithi183.stdout:(9/174): ceph-selinux-19.0.0-2145.g3be9a5c9.el9 441 kB/s | 25 kB 00:00 2024-03-26T08:17:43.050 INFO:teuthology.orchestra.run.smithi052.stdout:(149/174): ceph-test-19.0.0-2145.g3be9a5c9.el9. 4.6 MB/s | 48 MB 00:10 2024-03-26T08:17:43.376 INFO:teuthology.orchestra.run.smithi052.stdout:(150/174): librados2-19.0.0-2145.g3be9a5c9.el9. 3.2 MB/s | 3.4 MB 00:01 2024-03-26T08:17:43.426 INFO:teuthology.orchestra.run.smithi052.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.4 MB/s | 321 kB 00:00 2024-03-26T08:17:43.743 INFO:teuthology.orchestra.run.smithi183.stdout:(10/174): ceph-osd-19.0.0-2145.g3be9a5c9.el9.x8 11 MB/s | 17 MB 00:01 2024-03-26T08:17:43.819 INFO:teuthology.orchestra.run.smithi183.stdout:(11/174): cephfs-mirror-19.0.0-2145.g3be9a5c9.e 2.9 MB/s | 220 kB 00:00 2024-03-26T08:17:43.869 INFO:teuthology.orchestra.run.smithi183.stdout:(12/174): cephfs-shell-19.0.0-2145.g3be9a5c9.el 549 kB/s | 27 kB 00:00 2024-03-26T08:17:43.876 INFO:teuthology.orchestra.run.smithi052.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 5.1 MB/s | 2.3 MB 00:00 2024-03-26T08:17:43.952 INFO:teuthology.orchestra.run.smithi183.stdout:(13/174): libcephfs-devel-19.0.0-2145.g3be9a5c9 380 kB/s | 31 kB 00:00 2024-03-26T08:17:43.959 INFO:teuthology.orchestra.run.smithi052.stdout:(153/174): librbd1-19.0.0-2145.g3be9a5c9.el9.x8 3.5 MB/s | 3.1 MB 00:00 2024-03-26T08:17:44.060 INFO:teuthology.orchestra.run.smithi052.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 425 kB/s | 42 kB 00:00 2024-03-26T08:17:44.086 INFO:teuthology.orchestra.run.smithi183.stdout:(14/174): libcephfs2-19.0.0-2145.g3be9a5c9.el9. 5.5 MB/s | 746 kB 00:00 2024-03-26T08:17:44.168 INFO:teuthology.orchestra.run.smithi052.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 2.3 MB/s | 248 kB 00:00 2024-03-26T08:17:44.218 INFO:teuthology.orchestra.run.smithi052.stdout:(156/174): NetworkManager-libnm-1.47.2-1.el9.x8 5.4 MB/s | 1.8 MB 00:00 2024-03-26T08:17:44.236 INFO:teuthology.orchestra.run.smithi183.stdout:(15/174): ceph-radosgw-19.0.0-2145.g3be9a5c9.el 4.9 MB/s | 9.9 MB 00:02 2024-03-26T08:17:44.244 INFO:teuthology.orchestra.run.smithi052.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 958 kB/s | 71 kB 00:00 2024-03-26T08:17:44.270 INFO:teuthology.orchestra.run.smithi183.stdout:(16/174): libcephsqlite-19.0.0-2145.g3be9a5c9.e 892 kB/s | 163 kB 00:00 2024-03-26T08:17:44.278 INFO:teuthology.orchestra.run.smithi052.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 625 kB/s | 36 kB 00:00 2024-03-26T08:17:44.303 INFO:teuthology.orchestra.run.smithi052.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.2 MB/s | 192 kB 00:00 2024-03-26T08:17:44.311 INFO:teuthology.orchestra.run.smithi183.stdout:(17/174): librados-devel-19.0.0-2145.g3be9a5c9. 1.7 MB/s | 127 kB 00:00 2024-03-26T08:17:44.345 INFO:teuthology.orchestra.run.smithi052.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.8 MB/s | 256 kB 00:00 2024-03-26T08:17:44.354 INFO:teuthology.orchestra.run.smithi183.stdout:(18/174): libradosstriper1-19.0.0-2145.g3be9a5c 5.9 MB/s | 501 kB 00:00 2024-03-26T08:17:44.370 INFO:teuthology.orchestra.run.smithi052.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 347 kB/s | 23 kB 00:00 2024-03-26T08:17:44.412 INFO:teuthology.orchestra.run.smithi183.stdout:(19/174): python3-ceph-argparse-19.0.0-2145.g3b 786 kB/s | 45 kB 00:00 2024-03-26T08:17:44.437 INFO:teuthology.orchestra.run.smithi052.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 155 kB 00:00 2024-03-26T08:17:44.479 INFO:teuthology.orchestra.run.smithi183.stdout:(20/174): python3-ceph-common-19.0.0-2145.g3be9 2.0 MB/s | 131 kB 00:00 2024-03-26T08:17:44.504 INFO:teuthology.orchestra.run.smithi052.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 136 kB 00:00 2024-03-26T08:17:44.546 INFO:teuthology.orchestra.run.smithi183.stdout:(21/174): python3-cephfs-19.0.0-2145.g3be9a5c9. 2.4 MB/s | 162 kB 00:00 2024-03-26T08:17:44.554 INFO:teuthology.orchestra.run.smithi052.stdout:(164/174): libblkid-2.37.4-18.el9.x86_64.rpm 520 kB/s | 108 kB 00:00 2024-03-26T08:17:44.579 INFO:teuthology.orchestra.run.smithi052.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 855 kB/s | 64 kB 00:00 2024-03-26T08:17:44.613 INFO:teuthology.orchestra.run.smithi052.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 496 kB/s | 29 kB 00:00 2024-03-26T08:17:44.629 INFO:teuthology.orchestra.run.smithi183.stdout:(22/174): python3-rados-19.0.0-2145.g3be9a5c9.e 3.8 MB/s | 321 kB 00:00 2024-03-26T08:17:44.713 INFO:teuthology.orchestra.run.smithi183.stdout:(23/174): python3-rbd-19.0.0-2145.g3be9a5c9.el9 3.5 MB/s | 298 kB 00:00 2024-03-26T08:17:44.814 INFO:teuthology.orchestra.run.smithi052.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 5.2 MB/s | 1.2 MB 00:00 2024-03-26T08:17:44.830 INFO:teuthology.orchestra.run.smithi183.stdout:(24/174): librgw2-19.0.0-2145.g3be9a5c9.el9.x86 9.9 MB/s | 5.1 MB 00:00 2024-03-26T08:17:44.864 INFO:teuthology.orchestra.run.smithi183.stdout:(25/174): python3-rgw-19.0.0-2145.g3be9a5c9.el9 664 kB/s | 99 kB 00:00 2024-03-26T08:17:44.906 INFO:teuthology.orchestra.run.smithi183.stdout:(26/174): rbd-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.1 MB/s | 85 kB 00:00 2024-03-26T08:17:45.006 INFO:teuthology.orchestra.run.smithi183.stdout:(27/174): rbd-nbd-19.0.0-2145.g3be9a5c9.el9.x86 1.7 MB/s | 170 kB 00:00 2024-03-26T08:17:45.014 INFO:teuthology.orchestra.run.smithi052.stdout:(168/174): postgresql-server-13.14-1.el9.x86_64 1.0 MB/s | 5.8 MB 00:05 2024-03-26T08:17:45.081 INFO:teuthology.orchestra.run.smithi052.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 4.6 MB/s | 2.2 MB 00:00 2024-03-26T08:17:45.135 INFO:teuthology.orchestra.run.smithi183.stdout:(28/174): rbd-mirror-19.0.0-2145.g3be9a5c9.el9. 12 MB/s | 3.1 MB 00:00 2024-03-26T08:17:45.148 INFO:teuthology.orchestra.run.smithi183.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2145.g 240 kB/s | 34 kB 00:00 2024-03-26T08:17:45.181 INFO:teuthology.orchestra.run.smithi052.stdout:(170/174): util-linux-2.37.4-18.el9.x86_64.rpm 6.2 MB/s | 2.3 MB 00:00 2024-03-26T08:17:45.190 INFO:teuthology.orchestra.run.smithi183.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2145.g3be9a5c 2.4 MB/s | 139 kB 00:00 2024-03-26T08:17:45.198 INFO:teuthology.orchestra.run.smithi052.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 267 kB/s | 31 kB 00:00 2024-03-26T08:17:45.315 INFO:teuthology.orchestra.run.smithi052.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 641 kB/s | 74 kB 00:00 2024-03-26T08:17:45.357 INFO:teuthology.orchestra.run.smithi183.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2145.g3be9a 8.6 MB/s | 1.8 MB 00:00 2024-03-26T08:17:45.365 INFO:teuthology.orchestra.run.smithi052.stdout:(173/174): util-linux-core-2.37.4-18.el9.x86_64 1.3 MB/s | 465 kB 00:00 2024-03-26T08:17:45.457 INFO:teuthology.orchestra.run.smithi183.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2145.g3b 2.4 MB/s | 248 kB 00:00 2024-03-26T08:17:45.465 INFO:teuthology.orchestra.run.smithi052.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 3.8 MB/s | 1.1 MB 00:00 2024-03-26T08:17:45.474 INFO:teuthology.orchestra.run.smithi052.stdout:-------------------------------------------------------------------------------- 2024-03-26T08:17:45.474 INFO:teuthology.orchestra.run.smithi052.stdout:Total 14 MB/s | 225 MB 00:16 2024-03-26T08:17:45.558 INFO:teuthology.orchestra.run.smithi183.stdout:(33/174): ceph-mgr-rook-19.0.0-2145.g3be9a5c9.e 497 kB/s | 49 kB 00:00 2024-03-26T08:17:45.604 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-03-26T08:17:45.650 INFO:teuthology.orchestra.run.smithi183.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2145.g3 181 kB/s | 16 kB 00:00 2024-03-26T08:17:45.714 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-26T08:17:45.752 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-26T08:17:45.775 INFO:teuthology.orchestra.run.smithi183.stdout:(35/174): ceph-mgr-diskprediction-local-19.0.0- 13 MB/s | 7.4 MB 00:00 2024-03-26T08:17:45.817 INFO:teuthology.orchestra.run.smithi183.stdout:(36/174): ceph-volume-19.0.0-2145.g3be9a5c9.el9 1.5 MB/s | 264 kB 00:00 2024-03-26T08:17:45.840 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T08:17:45.875 INFO:teuthology.orchestra.run.smithi183.stdout:(37/174): cephadm-19.0.0-2145.g3be9a5c9.el9.noa 7.4 MB/s | 748 kB 00:00 2024-03-26T08:17:45.901 INFO:teuthology.orchestra.run.smithi183.stdout:(38/174): cephfs-top-19.0.0-2145.g3be9a5c9.el9. 223 kB/s | 18 kB 00:00 2024-03-26T08:17:45.972 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T08:17:46.100 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-26T08:17:46.134 INFO:teuthology.orchestra.run.smithi183.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 662 kB 00:00 2024-03-26T08:17:46.201 INFO:teuthology.orchestra.run.smithi183.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 3.0 MB/s | 920 kB 00:00 2024-03-26T08:17:46.213 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-26T08:17:46.218 INFO:teuthology.orchestra.run.smithi183.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 106 kB 00:00 2024-03-26T08:17:46.251 INFO:teuthology.orchestra.run.smithi183.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.2 MB/s | 307 kB 00:00 2024-03-26T08:17:46.269 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-26T08:17:46.277 INFO:teuthology.orchestra.run.smithi183.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 550 kB/s | 32 kB 00:00 2024-03-26T08:17:46.306 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-26T08:17:46.403 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-26T08:17:46.419 INFO:teuthology.orchestra.run.smithi183.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2024-03-26T08:17:46.444 INFO:teuthology.orchestra.run.smithi183.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 106 kB/s | 17 kB 00:00 2024-03-26T08:17:46.486 INFO:teuthology.orchestra.run.smithi183.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 249 kB/s | 10 kB 00:00 2024-03-26T08:17:46.520 INFO:teuthology.orchestra.run.smithi183.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2024-03-26T08:17:46.545 INFO:teuthology.orchestra.run.smithi183.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 790 kB/s | 44 kB 00:00 2024-03-26T08:17:46.600 INFO:teuthology.orchestra.run.smithi183.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 661 kB 00:00 2024-03-26T08:17:46.638 INFO:teuthology.orchestra.run.smithi183.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 6.5 MB/s | 578 kB 00:00 2024-03-26T08:17:46.663 INFO:teuthology.orchestra.run.smithi183.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 756 kB/s | 46 kB 00:00 2024-03-26T08:17:46.696 INFO:teuthology.orchestra.run.smithi183.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2024-03-26T08:17:46.730 INFO:teuthology.orchestra.run.smithi183.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.9 MB/s | 187 kB 00:00 2024-03-26T08:17:46.763 INFO:teuthology.orchestra.run.smithi183.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2024-03-26T08:17:46.789 INFO:teuthology.orchestra.run.smithi183.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 610 kB/s | 34 kB 00:00 2024-03-26T08:17:46.842 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-26T08:17:46.888 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-26T08:17:46.955 INFO:teuthology.orchestra.run.smithi183.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2024-03-26T08:17:47.006 INFO:teuthology.orchestra.run.smithi183.stdout:(57/174): protobuf-3.14.0-13.el9.x86_64.rpm 4.9 MB/s | 1.0 MB 00:00 2024-03-26T08:17:47.006 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction check 2024-03-26T08:17:47.031 INFO:teuthology.orchestra.run.smithi183.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 78 kB 00:00 2024-03-26T08:17:47.040 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-26T08:17:47.064 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-26T08:17:47.220 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-26T08:17:47.255 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-26T08:17:47.324 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-26T08:17:47.403 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction check succeeded. 2024-03-26T08:17:47.403 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction test 2024-03-26T08:17:47.565 INFO:teuthology.orchestra.run.smithi183.stdout:(59/174): ceph-test-19.0.0-2145.g3be9a5c9.el9.x 10 MB/s | 48 MB 00:04 2024-03-26T08:17:47.631 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-26T08:17:47.686 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-26T08:17:47.690 INFO:teuthology.orchestra.run.smithi183.stdout:(60/174): python3-babel-2.9.1-2.el9.noarch.rpm 8.5 MB/s | 5.8 MB 00:00 2024-03-26T08:17:47.711 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-26T08:17:47.715 INFO:teuthology.orchestra.run.smithi183.stdout:(61/174): python3-jsonpatch-1.21-16.el9.noarch. 36 kB/s | 24 kB 00:00 2024-03-26T08:17:47.740 INFO:teuthology.orchestra.run.smithi183.stdout:(62/174): python3-jsonpointer-2.0-4.el9.noarch. 112 kB/s | 18 kB 00:00 2024-03-26T08:17:47.765 INFO:teuthology.orchestra.run.smithi183.stdout:(63/174): python3-markupsafe-1.1.1-12.el9.x86_6 649 kB/s | 32 kB 00:00 2024-03-26T08:17:47.799 INFO:teuthology.orchestra.run.smithi183.stdout:(64/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 144 kB 00:00 2024-03-26T08:17:47.866 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-26T08:17:47.928 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-26T08:17:48.016 INFO:teuthology.orchestra.run.smithi183.stdout:(65/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 4.8 MB 00:00 2024-03-26T08:17:48.028 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-26T08:17:48.041 INFO:teuthology.orchestra.run.smithi183.stdout:(66/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 805 kB/s | 217 kB 00:00 2024-03-26T08:17:48.066 INFO:teuthology.orchestra.run.smithi183.stdout:(67/174): python3-oauthlib-3.1.1-5.el9.noarch.r 686 kB/s | 174 kB 00:00 2024-03-26T08:17:48.082 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-26T08:17:48.109 INFO:teuthology.orchestra.run.smithi183.stdout:(68/174): python3-packaging-20.9-5.el9.noarch.r 1.7 MB/s | 69 kB 00:00 2024-03-26T08:17:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T08:17:48.125 INFO:teuthology.orchestra.run.smithi183.stdout:(69/174): python3-prettytable-0.7.2-27.el9.noar 722 kB/s | 41 kB 00:00 2024-03-26T08:17:48.150 INFO:teuthology.orchestra.run.smithi183.stdout:(70/174): python3-protobuf-3.14.0-13.el9.noarch 2.8 MB/s | 238 kB 00:00 2024-03-26T08:17:48.175 INFO:teuthology.orchestra.run.smithi183.stdout:(71/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.0 MB/s | 132 kB 00:00 2024-03-26T08:17:48.187 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T08:17:48.209 INFO:teuthology.orchestra.run.smithi183.stdout:(72/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.7 MB/s | 211 kB 00:00 2024-03-26T08:17:48.234 INFO:teuthology.orchestra.run.smithi183.stdout:(73/174): python3-pytz-2021.1-5.el9.noarch.rpm 582 kB/s | 47 kB 00:00 2024-03-26T08:17:48.260 INFO:teuthology.orchestra.run.smithi183.stdout:(74/174): python3-requests-oauthlib-1.3.0-12.el 517 kB/s | 43 kB 00:00 2024-03-26T08:17:48.285 INFO:teuthology.orchestra.run.smithi183.stdout:(75/174): python3-toml-0.10.2-6.el9.noarch.rpm 794 kB/s | 39 kB 00:00 2024-03-26T08:17:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T08:17:48.538 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T08:17:48.591 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-26T08:17:48.593 INFO:teuthology.orchestra.run.smithi183.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 123 kB/s | 41 kB 00:00 2024-03-26T08:17:48.635 INFO:teuthology.orchestra.run.smithi183.stdout:(77/174): socat-1.7.4.1-5.el9.x86_64.rpm 891 kB/s | 300 kB 00:00 2024-03-26T08:17:48.844 INFO:teuthology.orchestra.run.smithi183.stdout:(78/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-03-26T08:17:48.869 INFO:teuthology.orchestra.run.smithi183.stdout:(79/174): uuid-1.6.2-55.el9.x86_64.rpm 207 kB/s | 56 kB 00:00 2024-03-26T08:17:48.894 INFO:teuthology.orchestra.run.smithi183.stdout:(80/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 255 kB/s | 63 kB 00:00 2024-03-26T08:17:48.919 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-26T08:17:48.928 INFO:teuthology.orchestra.run.smithi183.stdout:(81/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 771 kB/s | 44 kB 00:00 2024-03-26T08:17:48.953 INFO:teuthology.orchestra.run.smithi183.stdout:(82/174): fuse-2.9.9-15.el9.x86_64.rpm 959 kB/s | 79 kB 00:00 2024-03-26T08:17:48.987 INFO:teuthology.orchestra.run.smithi183.stdout:(83/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.6 MB/s | 241 kB 00:00 2024-03-26T08:17:49.012 INFO:teuthology.orchestra.run.smithi183.stdout:(84/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-03-26T08:17:49.062 INFO:teuthology.orchestra.run.smithi183.stdout:(85/174): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-03-26T08:17:49.088 INFO:teuthology.orchestra.run.smithi183.stdout:(86/174): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 124 kB 00:00 2024-03-26T08:17:49.119 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T08:17:49.121 INFO:teuthology.orchestra.run.smithi183.stdout:(87/174): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 187 kB 00:00 2024-03-26T08:17:49.212 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T08:17:49.415 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T08:17:49.484 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T08:17:49.630 INFO:teuthology.orchestra.run.smithi183.stdout:(88/174): libquadmath-11.4.1-3.el9.x86_64.rpm 385 kB/s | 195 kB 00:00 2024-03-26T08:17:49.671 INFO:teuthology.orchestra.run.smithi183.stdout:(89/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 803 kB 00:00 2024-03-26T08:17:49.705 INFO:teuthology.orchestra.run.smithi183.stdout:(90/174): jq-1.6-16.el9.x86_64.rpm 304 kB/s | 187 kB 00:00 2024-03-26T08:17:49.830 INFO:teuthology.orchestra.run.smithi183.stdout:(91/174): elfutils-libelf-devel-0.190-2.el9.x86 136 kB/s | 21 kB 00:00 2024-03-26T08:17:49.855 INFO:teuthology.orchestra.run.smithi183.stdout:(92/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 310 kB/s | 25 kB 00:00 2024-03-26T08:17:49.866 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T08:17:49.871 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction test succeeded. 2024-03-26T08:17:49.871 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction 2024-03-26T08:17:50.031 INFO:teuthology.orchestra.run.smithi183.stdout:(93/174): postgresql-13.14-1.el9.x86_64.rpm 9.1 MB/s | 1.6 MB 00:00 2024-03-26T08:17:50.064 INFO:teuthology.orchestra.run.smithi183.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 288 kB/s | 125 kB 00:00 2024-03-26T08:17:50.083 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-26T08:17:50.156 INFO:teuthology.orchestra.run.smithi183.stdout:(95/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 13 MB/s | 4.1 MB 00:00 2024-03-26T08:17:50.198 INFO:teuthology.orchestra.run.smithi183.stdout:(96/174): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-03-26T08:17:50.210 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-26T08:17:50.248 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-26T08:17:50.273 INFO:teuthology.orchestra.run.smithi183.stdout:(97/174): postgresql-contrib-13.14-1.el9.x86_64 3.7 MB/s | 911 kB 00:00 2024-03-26T08:17:50.315 INFO:teuthology.orchestra.run.smithi183.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.1 MB/s | 245 kB 00:00 2024-03-26T08:17:50.317 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-26T08:17:50.374 INFO:teuthology.orchestra.run.smithi183.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-03-26T08:17:50.384 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-26T08:17:50.432 INFO:teuthology.orchestra.run.smithi183.stdout:(100/174): qatlib-24.02.0-1.el9.x86_64.rpm 1.9 MB/s | 221 kB 00:00 2024-03-26T08:17:50.458 INFO:teuthology.orchestra.run.smithi183.stdout:(101/174): qatlib-service-24.02.0-1.el9.x86_64. 434 kB/s | 36 kB 00:00 2024-03-26T08:17:50.509 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-26T08:17:50.575 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-26T08:17:50.591 INFO:teuthology.orchestra.run.smithi183.stdout:(102/174): postgresql-server-13.14-1.el9.x86_64 13 MB/s | 5.8 MB 00:00 2024-03-26T08:17:50.617 INFO:teuthology.orchestra.run.smithi183.stdout:(103/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 259 kB/s | 47 kB 00:00 2024-03-26T08:17:50.625 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-26T08:17:50.650 INFO:teuthology.orchestra.run.smithi183.stdout:(104/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 490 kB/s | 94 kB 00:00 2024-03-26T08:17:50.753 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-26T08:17:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-26T08:17:50.872 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-26T08:17:50.884 INFO:teuthology.orchestra.run.smithi183.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 2.3 MB/s | 548 kB 00:00 2024-03-26T08:17:50.918 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-26T08:17:50.958 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-26T08:17:50.984 INFO:teuthology.orchestra.run.smithi183.stdout:(106/174): lua-devel-5.4.4-4.el9.x86_64.rpm 57 kB/s | 22 kB 00:00 2024-03-26T08:17:51.079 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-26T08:17:51.121 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 39/198 2024-03-26T08:17:51.134 INFO:teuthology.orchestra.run.smithi183.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 132 kB/s | 19 kB 00:00 2024-03-26T08:17:51.168 INFO:teuthology.orchestra.run.smithi183.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.1 MB/s | 309 kB 00:00 2024-03-26T08:17:51.187 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 40/198 2024-03-26T08:17:51.219 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 41/198 2024-03-26T08:17:51.345 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-26T08:17:51.393 INFO:teuthology.orchestra.run.smithi183.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 165 kB/s | 26 kB 00:00 2024-03-26T08:17:51.413 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-26T08:17:51.536 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-26T08:17:51.552 INFO:teuthology.orchestra.run.smithi183.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 310 kB/s | 49 kB 00:00 2024-03-26T08:17:51.621 INFO:teuthology.orchestra.run.smithi183.stdout:(111/174): protobuf-compiler-3.14.0-13.el9.x86_ 861 kB/s | 863 kB 00:01 2024-03-26T08:17:51.724 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-26T08:17:51.791 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-26T08:17:51.810 INFO:teuthology.orchestra.run.smithi183.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 262 kB/s | 67 kB 00:00 2024-03-26T08:17:51.953 INFO:teuthology.orchestra.run.smithi183.stdout:(113/174): libarrow-9.0.0-10.el9.x86_64.rpm 5.4 MB/s | 4.4 MB 00:00 2024-03-26T08:17:51.986 INFO:teuthology.orchestra.run.smithi183.stdout:(114/174): luarocks-3.9.2-1.el9.noarch.rpm 511 kB/s | 151 kB 00:00 2024-03-26T08:17:52.186 INFO:teuthology.orchestra.run.smithi183.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.2 MB/s | 838 kB 00:00 2024-03-26T08:17:52.212 INFO:teuthology.orchestra.run.smithi183.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 265 kB/s | 43 kB 00:00 2024-03-26T08:17:52.387 INFO:teuthology.orchestra.run.smithi183.stdout:(117/174): python3-asyncssh-2.13.2-5.el9.noarch 1.2 MB/s | 548 kB 00:00 2024-03-26T08:17:52.412 INFO:teuthology.orchestra.run.smithi183.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 143 kB/s | 32 kB 00:00 2024-03-26T08:17:52.496 INFO:teuthology.orchestra.run.smithi183.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 68 kB/s | 15 kB 00:00 2024-03-26T08:17:52.504 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-26T08:17:52.610 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-26T08:17:52.654 INFO:teuthology.orchestra.run.smithi183.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 649 kB/s | 172 kB 00:00 2024-03-26T08:17:52.726 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-26T08:17:52.763 INFO:teuthology.orchestra.run.smithi183.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 1.2 MB/s | 358 kB 00:00 2024-03-26T08:17:52.780 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T08:17:52.809 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T08:17:52.852 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-26T08:17:52.863 INFO:teuthology.orchestra.run.smithi183.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 962 kB/s | 303 kB 00:00 2024-03-26T08:17:52.912 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-26T08:17:52.955 INFO:teuthology.orchestra.run.smithi183.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 118 kB/s | 35 kB 00:00 2024-03-26T08:17:52.964 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 53/198 2024-03-26T08:17:53.038 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-26T08:17:53.047 INFO:teuthology.orchestra.run.smithi183.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 721 kB/s | 203 kB 00:00 2024-03-26T08:17:53.164 INFO:teuthology.orchestra.run.smithi183.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 699 kB/s | 144 kB 00:00 2024-03-26T08:17:53.189 INFO:teuthology.orchestra.run.smithi183.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 76 kB/s | 11 kB 00:00 2024-03-26T08:17:53.364 INFO:teuthology.orchestra.run.smithi183.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 4.1 MB/s | 2.0 MB 00:00 2024-03-26T08:17:53.389 INFO:teuthology.orchestra.run.smithi183.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 79 kB/s | 18 kB 00:00 2024-03-26T08:17:53.456 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-26T08:17:53.465 INFO:teuthology.orchestra.run.smithi183.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 108 kB/s | 23 kB 00:00 2024-03-26T08:17:53.598 INFO:teuthology.orchestra.run.smithi183.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 84 kB/s | 19 kB 00:00 2024-03-26T08:17:53.631 INFO:teuthology.orchestra.run.smithi183.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 104 kB/s | 20 kB 00:00 2024-03-26T08:17:53.694 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-26T08:17:53.816 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-26T08:17:53.822 INFO:teuthology.orchestra.run.smithi183.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 210 kB/s | 46 kB 00:00 2024-03-26T08:17:53.856 INFO:teuthology.orchestra.run.smithi183.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 459 kB/s | 79 kB 00:00 2024-03-26T08:17:53.892 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-26T08:17:53.964 INFO:teuthology.orchestra.run.smithi183.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 2.3 MB/s | 1.0 MB 00:00 2024-03-26T08:17:54.115 INFO:teuthology.orchestra.run.smithi183.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 198 kB/s | 58 kB 00:00 2024-03-26T08:17:54.116 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-26T08:17:54.165 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-26T08:17:54.215 INFO:teuthology.orchestra.run.smithi183.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 84 kB/s | 16 kB 00:00 2024-03-26T08:17:54.262 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-26T08:17:54.315 INFO:teuthology.orchestra.run.smithi183.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 692 kB/s | 272 kB 00:00 2024-03-26T08:17:54.341 INFO:teuthology.orchestra.run.smithi183.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 400 kB/s | 90 kB 00:00 2024-03-26T08:17:54.375 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-26T08:17:54.415 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-26T08:17:54.424 INFO:teuthology.orchestra.run.smithi183.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 137 kB/s | 28 kB 00:00 2024-03-26T08:17:54.455 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-26T08:17:54.526 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-26T08:17:54.566 INFO:teuthology.orchestra.run.smithi183.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 124 kB/s | 31 kB 00:00 2024-03-26T08:17:54.600 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-26T08:17:54.658 INFO:teuthology.orchestra.run.smithi183.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 325 kB/s | 59 kB 00:00 2024-03-26T08:17:54.672 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-26T08:17:54.692 INFO:teuthology.orchestra.run.smithi183.stdout:(142/174): python3-routes-2.5.1-5.el9.noarch.rp 643 kB/s | 188 kB 00:00 2024-03-26T08:17:54.740 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-26T08:17:54.783 INFO:teuthology.orchestra.run.smithi183.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 166 kB/s | 36 kB 00:00 2024-03-26T08:17:54.926 INFO:teuthology.orchestra.run.smithi183.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 192 kB/s | 51 kB 00:00 2024-03-26T08:17:54.942 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-26T08:17:55.018 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-26T08:17:55.034 INFO:teuthology.orchestra.run.smithi183.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 465 kB/s | 90 kB 00:00 2024-03-26T08:17:55.058 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-26T08:17:55.068 INFO:teuthology.orchestra.run.smithi183.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 715 kB/s | 230 kB 00:00 2024-03-26T08:17:55.119 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-26T08:17:55.263 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-26T08:17:55.293 INFO:teuthology.orchestra.run.smithi183.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 1.1 MB/s | 427 kB 00:00 2024-03-26T08:17:55.304 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-26T08:17:55.318 INFO:teuthology.orchestra.run.smithi183.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 71 kB/s | 20 kB 00:00 2024-03-26T08:17:55.354 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-26T08:17:55.396 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-26T08:17:55.418 INFO:teuthology.orchestra.run.smithi183.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 663 kB/s | 192 kB 00:00 2024-03-26T08:17:55.429 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-26T08:17:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-26T08:17:55.710 INFO:teuthology.orchestra.run.smithi183.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 3.8 MB/s | 1.6 MB 00:00 2024-03-26T08:17:55.778 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-26T08:17:55.785 INFO:teuthology.orchestra.run.smithi183.stdout:(151/174): librbd1-19.0.0-2145.g3be9a5c9.el9.x8 8.6 MB/s | 3.1 MB 00:00 2024-03-26T08:17:55.819 INFO:teuthology.orchestra.run.smithi183.stdout:(152/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.9 MB/s | 321 kB 00:00 2024-03-26T08:17:55.924 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-26T08:17:55.973 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-26T08:17:56.011 INFO:teuthology.orchestra.run.smithi183.stdout:(153/174): librados2-19.0.0-2145.g3be9a5c9.el9. 5.4 MB/s | 3.4 MB 00:00 2024-03-26T08:17:56.045 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-26T08:17:56.120 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-26T08:17:56.336 INFO:teuthology.orchestra.run.smithi183.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 130 kB/s | 42 kB 00:00 2024-03-26T08:17:56.376 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-26T08:17:56.411 INFO:teuthology.orchestra.run.smithi183.stdout:(155/174): NetworkManager-libnm-1.47.2-1.el9.x8 3.1 MB/s | 1.8 MB 00:00 2024-03-26T08:17:56.435 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-26T08:17:56.553 INFO:teuthology.orchestra.run.smithi183.stdout:(156/174): NetworkManager-1.47.2-1.el9.x86_64.r 3.0 MB/s | 2.3 MB 00:00 2024-03-26T08:17:56.610 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-26T08:17:56.657 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-26T08:17:56.697 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-26T08:17:56.744 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-26T08:17:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-26T08:17:56.813 INFO:teuthology.orchestra.run.smithi183.stdout:(157/174): NetworkManager-tui-1.47.2-1.el9.x86_ 533 kB/s | 248 kB 00:00 2024-03-26T08:17:56.831 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-26T08:17:56.845 INFO:teuthology.orchestra.run.smithi183.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 154 kB/s | 36 kB 00:00 2024-03-26T08:17:56.900 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-26T08:17:56.945 INFO:teuthology.orchestra.run.smithi183.stdout:(159/174): acl-2.3.1-4.el9.x86_64.rpm 153 kB/s | 71 kB 00:00 2024-03-26T08:17:56.970 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-26T08:17:57.027 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-26T08:17:57.171 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-26T08:17:57.212 INFO:teuthology.orchestra.run.smithi183.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 471 kB/s | 192 kB 00:00 2024-03-26T08:17:57.220 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-26T08:17:57.237 INFO:teuthology.orchestra.run.smithi183.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 79 kB/s | 23 kB 00:00 2024-03-26T08:17:57.246 INFO:teuthology.orchestra.run.smithi052.stdout: Preparing : 1/1 2024-03-26T08:17:57.264 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-26T08:17:57.340 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-26T08:17:57.347 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-26T08:17:57.379 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-26T08:17:57.392 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-26T08:17:57.426 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T08:17:57.446 INFO:teuthology.orchestra.run.smithi183.stdout:(162/174): elfutils-libs-0.190-2.el9.x86_64.rpm 481 kB/s | 256 kB 00:00 2024-03-26T08:17:57.471 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T08:17:57.544 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-26T08:17:57.558 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-26T08:17:57.580 INFO:teuthology.orchestra.run.smithi183.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 296 kB/s | 108 kB 00:00 2024-03-26T08:17:57.622 INFO:teuthology.orchestra.run.smithi183.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 404 kB/s | 155 kB 00:00 2024-03-26T08:17:57.669 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-26T08:17:57.670 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-26T08:17:57.731 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-26T08:17:57.737 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-26T08:17:57.769 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-26T08:17:57.782 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-26T08:17:57.844 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-26T08:17:57.847 INFO:teuthology.orchestra.run.smithi183.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 339 kB/s | 136 kB 00:00 2024-03-26T08:17:57.871 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-26T08:17:57.881 INFO:teuthology.orchestra.run.smithi183.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 213 kB/s | 64 kB 00:00 2024-03-26T08:17:57.996 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-26T08:17:58.031 INFO:teuthology.orchestra.run.smithi183.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 84 kB/s | 29 kB 00:00 2024-03-26T08:17:58.088 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-26T08:17:58.305 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-26T08:17:58.333 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-26T08:17:58.364 INFO:teuthology.orchestra.run.smithi183.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 2.4 MB/s | 1.2 MB 00:00 2024-03-26T08:17:58.475 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-26T08:17:58.501 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-26T08:17:58.531 INFO:teuthology.orchestra.run.smithi183.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 3.3 MB/s | 2.2 MB 00:00 2024-03-26T08:17:58.615 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-26T08:17:58.649 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-26T08:17:58.698 INFO:teuthology.orchestra.run.smithi183.stdout:(170/174): util-linux-2.37.4-18.el9.x86_64.rpm 3.4 MB/s | 2.3 MB 00:00 2024-03-26T08:17:58.717 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-26T08:17:58.832 INFO:teuthology.orchestra.run.smithi183.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 104 kB/s | 31 kB 00:00 2024-03-26T08:17:58.882 INFO:teuthology.orchestra.run.smithi183.stdout:(172/174): util-linux-core-2.37.4-18.el9.x86_64 901 kB/s | 465 kB 00:00 2024-03-26T08:17:58.907 INFO:teuthology.orchestra.run.smithi183.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.0 MB/s | 74 kB 00:00 2024-03-26T08:17:59.070 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-26T08:17:59.115 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-26T08:17:59.131 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-26T08:17:59.156 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-26T08:17:59.157 INFO:teuthology.orchestra.run.smithi183.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 2.4 MB/s | 1.1 MB 00:00 2024-03-26T08:17:59.170 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-03-26T08:17:59.170 INFO:teuthology.orchestra.run.smithi183.stdout:Total 11 MB/s | 225 MB 00:19 2024-03-26T08:17:59.171 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-26T08:17:59.232 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-26T08:17:59.306 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-26T08:17:59.325 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-26T08:17:59.357 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-26T08:17:59.422 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-26T08:17:59.457 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-26T08:17:59.464 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-26T08:17:59.502 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-26T08:17:59.534 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T08:17:59.597 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T08:17:59.709 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-26T08:17:59.743 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-26T08:17:59.838 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-26T08:17:59.869 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T08:17:59.922 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T08:17:59.967 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-26T08:18:00.323 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-26T08:18:00.485 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T08:18:00.575 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T08:18:00.672 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-26T08:18:00.780 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T08:18:00.803 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-26T08:18:00.835 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-03-26T08:18:00.837 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T08:18:01.210 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-03-26T08:18:01.211 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-03-26T08:18:01.225 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T08:18:01.414 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-26T08:18:01.538 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-26T08:18:01.576 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-26T08:18:01.612 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-26T08:18:01.674 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-26T08:18:01.791 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-26T08:18:01.864 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-26T08:18:01.912 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-26T08:18:02.035 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-26T08:18:02.083 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-26T08:18:02.172 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-26T08:18:02.231 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-26T08:18:02.270 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-26T08:18:02.397 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-26T08:18:02.403 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-26T08:18:02.448 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 39/198 2024-03-26T08:18:02.489 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-26T08:18:02.521 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 40/198 2024-03-26T08:18:02.526 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-26T08:18:02.556 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 41/198 2024-03-26T08:18:02.557 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-26T08:18:02.597 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-26T08:18:02.683 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-26T08:18:02.741 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-26T08:18:02.844 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-26T08:18:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-26T08:18:03.036 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-26T08:18:03.102 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-26T08:18:03.330 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-26T08:18:03.494 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-26T08:18:03.531 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 125/198 2024-03-26T08:18:03.628 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-26T08:18:03.633 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-03-26T08:18:03.633 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-03-26T08:18:03.822 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-26T08:18:03.922 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-26T08:18:04.029 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-26T08:18:04.074 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T08:18:04.102 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T08:18:04.144 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-26T08:18:04.210 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-26T08:18:04.218 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-26T08:18:04.249 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-26T08:18:04.273 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 53/198 2024-03-26T08:18:04.315 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-26T08:18:04.366 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-26T08:18:04.393 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 129/198 2024-03-26T08:18:04.727 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-26T08:18:04.973 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-26T08:18:05.087 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-26T08:18:05.144 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-26T08:18:05.365 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-26T08:18:05.416 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-26T08:18:05.527 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-26T08:18:05.634 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-26T08:18:05.674 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-26T08:18:05.714 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-26T08:18:05.784 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-26T08:18:05.831 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-26T08:18:05.897 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-26T08:18:05.974 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-26T08:18:06.178 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-26T08:18:06.228 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-26T08:18:06.266 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-26T08:18:06.341 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-26T08:18:06.488 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-26T08:18:06.538 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-26T08:18:06.912 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-26T08:18:06.959 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-26T08:18:06.990 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-26T08:18:07.154 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-26T08:18:07.189 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-26T08:18:07.198 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-26T08:18:07.375 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-26T08:18:07.506 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-26T08:18:07.565 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-26T08:18:07.628 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-26T08:18:07.703 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-26T08:18:07.862 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-26T08:18:07.909 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-26T08:18:07.957 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-26T08:18:08.027 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-26T08:18:08.080 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-26T08:18:08.080 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-26T08:18:08.080 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-26T08:18:08.080 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:18:08.135 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-26T08:18:08.207 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-26T08:18:08.257 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-26T08:18:08.297 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-26T08:18:08.342 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-26T08:18:08.391 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-26T08:18:08.440 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-26T08:18:08.509 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-26T08:18:08.567 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-26T08:18:08.608 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-26T08:18:08.745 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-26T08:18:08.806 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-26T08:18:08.851 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-26T08:18:08.915 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-26T08:18:08.955 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-26T08:18:09.106 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-26T08:18:09.234 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-26T08:18:09.292 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-26T08:18:09.328 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-26T08:18:09.389 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-26T08:18:09.543 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-26T08:18:09.611 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-26T08:18:10.631 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-26T08:18:10.676 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-26T08:18:10.709 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-26T08:18:10.765 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-26T08:18:10.879 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-26T08:18:10.906 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-03-26T08:18:10.926 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-26T08:18:11.056 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-26T08:18:11.104 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-26T08:18:11.160 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T08:18:11.170 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-26T08:18:11.204 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T08:18:11.215 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-26T08:18:11.299 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-26T08:18:11.322 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-26T08:18:11.422 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-26T08:18:11.476 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-26T08:18:11.508 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-26T08:18:11.590 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-26T08:18:12.009 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-26T08:18:12.048 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-26T08:18:12.197 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-26T08:18:12.216 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-26T08:18:12.232 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-26T08:18:12.364 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-26T08:18:12.379 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-26T08:18:12.419 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-26T08:18:12.500 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-26T08:18:12.801 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-26T08:18:12.854 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-26T08:18:12.879 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-26T08:18:13.028 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-26T08:18:13.063 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-26T08:18:13.137 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-26T08:18:13.173 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-26T08:18:13.199 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T08:18:13.263 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T08:18:13.592 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T08:18:13.640 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T08:18:13.690 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-26T08:18:13.977 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-26T08:18:14.049 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-26T08:18:14.068 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-26T08:18:14.106 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-26T08:18:14.136 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-26T08:18:14.176 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-26T08:18:14.205 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T08:18:14.278 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T08:18:14.471 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T08:18:14.542 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T08:18:14.574 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-26T08:18:14.639 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-26T08:18:14.759 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-26T08:18:14.790 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 125/198 2024-03-26T08:18:14.848 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-26T08:18:14.875 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T08:18:15.067 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-26T08:18:15.195 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-26T08:18:15.248 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-26T08:18:15.295 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-26T08:18:15.357 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-26T08:18:15.414 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-26T08:18:15.476 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-26T08:18:15.478 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-26T08:18:15.547 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-26T08:18:15.580 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-26T08:18:15.605 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 129/198 2024-03-26T08:18:15.606 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-26T08:18:15.698 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-26T08:18:15.745 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-26T08:18:15.807 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-26T08:18:15.851 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-26T08:18:15.890 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-26T08:18:16.003 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-26T08:18:16.069 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 39/198 2024-03-26T08:18:16.131 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 40/198 2024-03-26T08:18:16.168 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 41/198 2024-03-26T08:18:16.318 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-26T08:18:16.379 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-26T08:18:16.478 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-26T08:18:16.657 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-26T08:18:16.723 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-26T08:18:17.419 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-26T08:18:17.519 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-26T08:18:17.625 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-26T08:18:17.671 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T08:18:17.697 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T08:18:17.767 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-26T08:18:17.828 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-26T08:18:17.880 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 53/198 2024-03-26T08:18:17.933 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-26T08:18:18.248 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-26T08:18:18.294 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-26T08:18:18.340 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-26T08:18:18.562 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-26T08:18:18.651 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-26T08:18:18.709 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-26T08:18:18.926 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-26T08:18:18.963 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-26T08:18:18.980 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-26T08:18:19.005 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-26T08:18:19.057 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-26T08:18:19.145 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-26T08:18:19.146 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-26T08:18:19.146 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-26T08:18:19.146 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-26T08:18:19.151 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-26T08:18:19.181 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-26T08:18:19.197 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-26T08:18:19.236 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-26T08:18:19.313 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-26T08:18:19.362 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-26T08:18:19.425 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-26T08:18:19.494 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-26T08:18:19.691 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-26T08:18:19.751 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-26T08:18:19.789 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-26T08:18:19.846 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-26T08:18:19.978 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-26T08:18:20.020 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-26T08:18:20.066 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-26T08:18:20.114 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-26T08:18:20.144 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-26T08:18:20.301 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-26T08:18:20.478 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-26T08:18:20.622 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-26T08:18:20.671 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-26T08:18:20.734 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-26T08:18:20.808 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-26T08:18:21.057 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-26T08:18:21.118 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-26T08:18:21.289 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-26T08:18:21.307 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-26T08:18:21.307 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /sys 2024-03-26T08:18:21.307 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /proc 2024-03-26T08:18:21.307 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /mnt 2024-03-26T08:18:21.307 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /var/tmp 2024-03-26T08:18:21.307 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /home 2024-03-26T08:18:21.307 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /root 2024-03-26T08:18:21.307 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /tmp 2024-03-26T08:18:21.308 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:18:21.333 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-26T08:18:21.373 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-26T08:18:21.411 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-26T08:18:21.452 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-26T08:18:21.497 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-26T08:18:21.548 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-26T08:18:21.555 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-26T08:18:21.596 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-26T08:18:21.597 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:18:21.597 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-26T08:18:21.597 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:18:21.607 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-26T08:18:21.648 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-26T08:18:21.772 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-26T08:18:21.811 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-26T08:18:21.846 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-26T08:18:21.906 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-26T08:18:21.950 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-26T08:18:22.095 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-26T08:18:22.212 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-26T08:18:22.262 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-26T08:18:22.298 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-26T08:18:22.353 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-26T08:18:22.491 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-26T08:18:22.564 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-26T08:18:23.259 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-26T08:18:23.336 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-26T08:18:23.336 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:18:23.336 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-26T08:18:23.336 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T08:18:23.337 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T08:18:23.337 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:18:23.523 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-26T08:18:23.584 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-26T08:18:23.656 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-26T08:18:23.656 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:18:23.656 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-26T08:18:23.656 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T08:18:23.657 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T08:18:23.657 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:18:23.722 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-26T08:18:23.761 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-26T08:18:23.838 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-26T08:18:23.964 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-26T08:18:24.006 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-26T08:18:24.168 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-26T08:18:24.239 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-26T08:18:24.239 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:18:24.239 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-26T08:18:24.239 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T08:18:24.239 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T08:18:24.239 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:18:24.240 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-26T08:18:24.286 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-26T08:18:24.362 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-26T08:18:24.392 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-26T08:18:24.939 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-26T08:18:25.013 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-26T08:18:25.084 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 139/198 2024-03-26T08:18:25.202 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 140/198 2024-03-26T08:18:25.241 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-26T08:18:25.283 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-26T08:18:25.442 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-26T08:18:25.588 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-26T08:18:25.625 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-26T08:18:26.796 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-26T08:18:26.826 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-26T08:18:26.958 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-26T08:18:26.975 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-26T08:18:27.049 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-26T08:18:27.086 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-26T08:18:27.117 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-26T08:18:27.156 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 144/198 2024-03-26T08:18:27.157 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-26T08:18:27.186 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-26T08:18:27.269 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-26T08:18:27.269 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:18:27.269 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-26T08:18:27.269 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T08:18:27.269 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T08:18:27.269 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:18:27.329 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-26T08:18:27.369 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-26T08:18:27.436 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 147/198 2024-03-26T08:18:27.532 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-26T08:18:27.564 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-26T08:18:27.684 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-26T08:18:27.720 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 125/198 2024-03-26T08:18:27.774 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-26T08:18:28.285 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-26T08:18:28.317 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-26T08:18:28.417 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-26T08:18:28.438 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-26T08:18:28.444 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 129/198 2024-03-26T08:18:28.472 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-26T08:18:28.552 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-26T08:18:28.552 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:18:28.552 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-26T08:18:28.552 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T08:18:28.552 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T08:18:28.553 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:18:28.606 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-26T08:18:28.703 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-26T08:18:28.703 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:18:28.703 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-26T08:18:28.703 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:18:28.783 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-26T08:18:28.889 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-26T08:18:28.889 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:18:28.889 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-26T08:18:28.889 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T08:18:28.889 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T08:18:28.889 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:18:29.311 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-26T08:18:29.403 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-26T08:18:29.403 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:18:29.403 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-26T08:18:29.403 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T08:18:29.404 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T08:18:29.404 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:18:30.998 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-26T08:18:31.043 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-26T08:18:31.676 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-26T08:18:31.720 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-26T08:18:31.920 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-26T08:18:31.920 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-26T08:18:31.920 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-26T08:18:31.920 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-26T08:18:31.996 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-26T08:18:32.884 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-26T08:18:32.884 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /sys 2024-03-26T08:18:32.884 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /proc 2024-03-26T08:18:32.884 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /mnt 2024-03-26T08:18:32.884 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /var/tmp 2024-03-26T08:18:32.884 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /home 2024-03-26T08:18:32.885 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /root 2024-03-26T08:18:32.885 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /tmp 2024-03-26T08:18:32.885 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-26T08:18:33.195 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-26T08:18:33.248 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-26T08:18:33.248 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:18:33.248 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-26T08:18:33.248 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-26T08:18:34.532 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-26T08:18:34.596 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 154/198 2024-03-26T08:18:34.630 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 155/198 2024-03-26T08:18:34.668 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-26T08:18:34.928 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-26T08:18:34.963 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 157/198 2024-03-26T08:18:35.014 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-26T08:18:35.014 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:18:35.014 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-26T08:18:35.014 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T08:18:35.014 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T08:18:35.014 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-26T08:18:35.149 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-26T08:18:35.240 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 159/198 2024-03-26T08:18:35.270 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-26T08:18:35.307 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-26T08:18:35.342 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-26T08:18:35.342 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:18:35.342 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-26T08:18:35.342 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T08:18:35.342 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T08:18:35.342 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-26T08:18:35.374 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-26T08:18:35.421 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-26T08:18:35.463 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 163/198 2024-03-26T08:18:35.514 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-26T08:18:35.566 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-26T08:18:35.704 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-26T08:18:35.816 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-26T08:18:35.854 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-26T08:18:35.858 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-26T08:18:35.927 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-26T08:18:35.927 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:18:35.927 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-26T08:18:35.927 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T08:18:35.927 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T08:18:35.927 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-26T08:18:36.047 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-26T08:18:36.667 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-26T08:18:36.698 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-26T08:18:36.750 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 139/198 2024-03-26T08:18:36.845 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T08:18:36.852 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 140/198 2024-03-26T08:18:36.885 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-26T08:18:36.896 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T08:18:36.957 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T08:18:37.230 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-26T08:18:37.261 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-26T08:18:38.376 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 170/198 2024-03-26T08:18:38.433 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-26T08:18:38.461 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-26T08:18:38.476 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-26T08:18:38.526 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-26T08:18:38.610 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-26T08:18:38.662 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-26T08:18:38.729 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-26T08:18:38.770 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-26T08:18:38.791 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 144/198 2024-03-26T08:18:38.822 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-26T08:18:38.830 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-26T08:18:38.865 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-26T08:18:38.865 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T08:18:38.908 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-26T08:18:38.908 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:18:38.908 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-26T08:18:38.908 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T08:18:38.908 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T08:18:38.908 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-26T08:18:38.972 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-26T08:18:39.008 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T08:18:39.013 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-26T08:18:39.050 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-26T08:18:39.050 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T08:18:39.077 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 147/198 2024-03-26T08:18:39.197 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T08:18:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-26T08:18:39.295 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-26T08:18:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-26T08:18:39.369 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-26T08:18:39.439 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-26T08:18:39.509 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-26T08:18:39.557 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-26T08:18:39.598 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-26T08:18:39.641 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-26T08:18:39.674 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-26T08:18:39.707 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-26T08:18:39.746 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T08:18:39.746 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T08:18:40.058 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T08:18:40.058 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T08:18:40.082 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-26T08:18:40.116 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-26T08:18:40.187 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-26T08:18:40.187 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:18:40.187 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-26T08:18:40.187 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T08:18:40.187 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T08:18:40.187 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-26T08:18:40.205 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T08:18:40.241 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-26T08:18:40.260 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-26T08:18:40.309 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-26T08:18:40.309 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:18:40.309 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-26T08:18:40.309 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-26T08:18:40.315 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-26T08:18:40.355 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-26T08:18:40.375 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-26T08:18:40.403 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-26T08:18:40.451 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-26T08:18:40.452 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:18:40.452 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-26T08:18:40.452 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T08:18:40.452 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T08:18:40.452 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-26T08:18:40.528 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T08:18:40.816 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T08:18:40.816 INFO:teuthology.orchestra.run.smithi043.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-26T08:18:40.816 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:18:40.835 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-26T08:18:40.907 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-26T08:18:40.907 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:18:40.907 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-26T08:18:40.907 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T08:18:40.907 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T08:18:40.908 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-26T08:18:45.292 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-26T08:18:45.292 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /sys 2024-03-26T08:18:45.292 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /proc 2024-03-26T08:18:45.292 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /mnt 2024-03-26T08:18:45.292 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /var/tmp 2024-03-26T08:18:45.292 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /home 2024-03-26T08:18:45.292 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /root 2024-03-26T08:18:45.292 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /tmp 2024-03-26T08:18:45.292 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-26T08:18:45.519 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-26T08:18:45.562 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-26T08:18:45.563 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:18:45.563 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-26T08:18:45.563 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-26T08:18:45.747 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 1/198 2024-03-26T08:18:45.747 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2/198 2024-03-26T08:18:45.747 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 3/198 2024-03-26T08:18:45.748 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 4/198 2024-03-26T08:18:45.748 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 5/198 2024-03-26T08:18:45.748 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 6/198 2024-03-26T08:18:45.748 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 7/198 2024-03-26T08:18:45.748 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 8/198 2024-03-26T08:18:45.748 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 9/198 2024-03-26T08:18:45.748 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-26T08:18:45.748 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-26T08:18:45.748 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-26T08:18:45.748 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 13/198 2024-03-26T08:18:45.748 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 14/198 2024-03-26T08:18:45.748 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 15/198 2024-03-26T08:18:45.748 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-26T08:18:45.748 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 17/198 2024-03-26T08:18:45.749 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 18/198 2024-03-26T08:18:45.749 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 19/198 2024-03-26T08:18:45.749 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 20/198 2024-03-26T08:18:45.749 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 21/198 2024-03-26T08:18:45.749 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 22/198 2024-03-26T08:18:45.749 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 23/198 2024-03-26T08:18:45.749 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 24/198 2024-03-26T08:18:45.749 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 25/198 2024-03-26T08:18:45.749 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 26/198 2024-03-26T08:18:45.749 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 27/198 2024-03-26T08:18:45.749 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 28/198 2024-03-26T08:18:45.749 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 29/198 2024-03-26T08:18:45.764 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 30/198 2024-03-26T08:18:45.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 31/198 2024-03-26T08:18:45.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 32/198 2024-03-26T08:18:45.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 33/198 2024-03-26T08:18:45.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 34/198 2024-03-26T08:18:45.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 35/198 2024-03-26T08:18:45.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 36/198 2024-03-26T08:18:45.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 37/198 2024-03-26T08:18:45.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 38/198 2024-03-26T08:18:45.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 39/198 2024-03-26T08:18:45.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-26T08:18:45.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-26T08:18:45.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-26T08:18:45.766 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-26T08:18:45.766 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-26T08:18:45.766 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-26T08:18:45.766 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-26T08:18:45.766 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-26T08:18:45.766 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-26T08:18:45.766 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-26T08:18:45.766 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-26T08:18:45.766 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-26T08:18:45.766 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-26T08:18:45.766 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-26T08:18:45.766 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-26T08:18:45.766 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-26T08:18:45.766 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-26T08:18:45.767 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-26T08:18:45.767 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-26T08:18:45.767 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-26T08:18:45.767 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-26T08:18:45.767 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-26T08:18:45.767 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-26T08:18:45.767 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-26T08:18:45.767 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-26T08:18:45.767 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-26T08:18:45.767 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-26T08:18:45.767 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-26T08:18:45.767 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-26T08:18:45.767 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-26T08:18:45.768 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-26T08:18:45.768 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-26T08:18:45.768 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-26T08:18:45.768 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-26T08:18:45.768 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-26T08:18:45.768 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-26T08:18:45.768 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-26T08:18:45.768 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-26T08:18:45.768 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-26T08:18:45.768 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-26T08:18:45.768 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-26T08:18:45.768 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-26T08:18:45.768 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-26T08:18:45.768 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-26T08:18:45.769 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-26T08:18:45.769 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-26T08:18:45.769 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-26T08:18:45.769 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-26T08:18:45.769 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-26T08:18:45.769 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-26T08:18:45.769 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-26T08:18:45.769 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-26T08:18:45.769 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-26T08:18:45.769 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-26T08:18:45.769 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-26T08:18:45.769 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-26T08:18:45.769 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-26T08:18:45.769 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-26T08:18:45.770 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-26T08:18:45.770 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-26T08:18:45.770 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-26T08:18:45.770 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-26T08:18:45.770 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-26T08:18:45.770 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-26T08:18:45.770 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-26T08:18:45.770 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-26T08:18:45.770 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-26T08:18:45.770 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-26T08:18:45.770 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-26T08:18:45.770 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-26T08:18:45.770 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-26T08:18:45.770 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-26T08:18:45.771 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-26T08:18:45.771 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-26T08:18:45.771 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-26T08:18:45.771 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-26T08:18:45.771 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-26T08:18:45.771 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-26T08:18:45.771 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-26T08:18:45.771 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-26T08:18:45.771 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-26T08:18:45.771 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-26T08:18:45.771 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-26T08:18:45.771 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-26T08:18:45.771 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-26T08:18:45.771 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-26T08:18:45.771 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-26T08:18:45.772 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-26T08:18:45.772 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-26T08:18:45.772 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-26T08:18:45.772 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-26T08:18:45.772 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-26T08:18:45.772 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-26T08:18:45.772 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-26T08:18:45.772 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-26T08:18:45.772 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-26T08:18:45.772 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-26T08:18:45.772 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-26T08:18:45.772 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-26T08:18:45.772 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-26T08:18:45.772 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-26T08:18:45.773 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-26T08:18:45.773 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-26T08:18:45.773 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-26T08:18:45.773 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-26T08:18:45.773 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-26T08:18:45.773 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-26T08:18:45.773 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-26T08:18:45.773 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-26T08:18:45.773 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-26T08:18:45.773 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-26T08:18:45.773 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-26T08:18:45.773 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-26T08:18:45.773 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-26T08:18:45.773 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-26T08:18:45.773 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-26T08:18:45.774 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-26T08:18:45.774 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-26T08:18:45.774 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-26T08:18:45.774 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-26T08:18:45.774 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-26T08:18:45.774 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-26T08:18:45.774 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-26T08:18:45.774 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-26T08:18:45.774 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-26T08:18:45.774 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-26T08:18:45.774 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-26T08:18:45.774 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-26T08:18:45.774 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-26T08:18:45.774 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-26T08:18:45.775 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-26T08:18:45.775 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-26T08:18:45.775 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-26T08:18:45.775 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-26T08:18:45.775 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-26T08:18:45.775 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-26T08:18:45.775 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-26T08:18:45.775 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-26T08:18:45.775 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-26T08:18:45.775 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-26T08:18:45.775 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-26T08:18:45.775 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-26T08:18:45.775 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-26T08:18:45.775 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-26T08:18:45.775 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-26T08:18:45.776 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-26T08:18:45.776 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-26T08:18:45.776 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-26T08:18:45.776 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-26T08:18:45.776 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-26T08:18:45.776 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-26T08:18:45.776 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-26T08:18:45.776 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-26T08:18:45.776 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-26T08:18:45.776 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-26T08:18:45.776 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-26T08:18:45.776 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-26T08:18:45.776 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-26T08:18:46.008 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-26T08:18:46.067 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 154/198 2024-03-26T08:18:46.115 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 155/198 2024-03-26T08:18:46.155 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-26T08:18:46.449 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 157/198 2024-03-26T08:18:46.571 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-26T08:18:46.616 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 159/198 2024-03-26T08:18:46.674 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-26T08:18:46.731 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-26T08:18:46.775 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-26T08:18:46.809 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 163/198 2024-03-26T08:18:46.847 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-26T08:18:46.900 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-26T08:18:46.997 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-26T08:18:47.117 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-26T08:18:47.154 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-26T08:18:47.156 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-26T08:18:47.254 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-26T08:18:47.254 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:18:47.254 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-26T08:18:47.254 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T08:18:47.255 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T08:18:47.255 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-26T08:18:47.513 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-26T08:18:47.606 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-26T08:18:47.606 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:18:47.606 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-26T08:18:47.606 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T08:18:47.606 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T08:18:47.607 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-26T08:18:48.026 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T08:18:48.071 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T08:18:48.105 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-26T08:18:48.127 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T08:18:48.199 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-26T08:18:48.199 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:18:48.199 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-26T08:18:48.199 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T08:18:48.199 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T08:18:48.199 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-26T08:18:48.316 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-26T08:18:48.374 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-26T08:18:48.374 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:18:48.374 INFO:teuthology.orchestra.run.smithi043.stdout:Upgraded: 2024-03-26T08:18:48.374 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-26T08:18:48.374 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-26T08:18:48.374 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-26T08:18:48.374 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-26T08:18:48.374 INFO:teuthology.orchestra.run.smithi043.stdout: acl-2.3.1-4.el9.x86_64 2024-03-26T08:18:48.374 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-26T08:18:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-26T08:18:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-26T08:18:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-26T08:18:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-26T08:18:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-26T08:18:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-26T08:18:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-26T08:18:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-26T08:18:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-26T08:18:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-26T08:18:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-26T08:18:48.375 INFO:teuthology.orchestra.run.smithi043.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-26T08:18:48.376 INFO:teuthology.orchestra.run.smithi043.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-26T08:18:48.376 INFO:teuthology.orchestra.run.smithi043.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-26T08:18:48.376 INFO:teuthology.orchestra.run.smithi043.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-26T08:18:48.376 INFO:teuthology.orchestra.run.smithi043.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-26T08:18:48.376 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-03-26T08:18:48.376 INFO:teuthology.orchestra.run.smithi043.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-26T08:18:48.376 INFO:teuthology.orchestra.run.smithi043.stdout: automake-1.16.2-8.el9.noarch 2024-03-26T08:18:48.376 INFO:teuthology.orchestra.run.smithi043.stdout: bison-3.7.4-5.el9.x86_64 2024-03-26T08:18:48.376 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-26T08:18:48.376 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.376 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.376 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.376 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.376 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:18:48.377 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.377 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.377 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.377 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:18:48.377 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:18:48.377 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:18:48.377 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:18:48.377 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:18:48.377 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.377 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.377 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:18:48.377 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.377 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.377 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.378 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:18:48.378 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:18:48.378 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.378 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.378 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:18:48.378 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-26T08:18:48.378 INFO:teuthology.orchestra.run.smithi043.stdout: flex-2.6.4-9.el9.x86_64 2024-03-26T08:18:48.378 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-26T08:18:48.378 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-26T08:18:48.378 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-26T08:18:48.378 INFO:teuthology.orchestra.run.smithi043.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-26T08:18:48.378 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-26T08:18:48.378 INFO:teuthology.orchestra.run.smithi043.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-26T08:18:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: jq-1.6-16.el9.x86_64 2024-03-26T08:18:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-26T08:18:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-26T08:18:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-26T08:18:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-26T08:18:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-26T08:18:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-26T08:18:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-26T08:18:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-26T08:18:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.379 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-26T08:18:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-26T08:18:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-26T08:18:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-26T08:18:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-26T08:18:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-26T08:18:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: lua-5.4.4-4.el9.x86_64 2024-03-26T08:18:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-26T08:18:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-26T08:18:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-26T08:18:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-26T08:18:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-26T08:18:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-26T08:18:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-26T08:18:48.380 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-13.14-1.el9.x86_64 2024-03-26T08:18:48.381 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-26T08:18:48.381 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-26T08:18:48.381 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-26T08:18:48.381 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-26T08:18:48.381 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-26T08:18:48.381 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-26T08:18:48.381 INFO:teuthology.orchestra.run.smithi043.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-26T08:18:48.381 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-26T08:18:48.381 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-26T08:18:48.381 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-26T08:18:48.381 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.381 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-26T08:18:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-26T08:18:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-26T08:18:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-26T08:18:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-26T08:18:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-26T08:18:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-26T08:18:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-26T08:18:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-26T08:18:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-26T08:18:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-26T08:18:48.382 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-26T08:18:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-26T08:18:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-26T08:18:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-26T08:18:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-26T08:18:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-26T08:18:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-26T08:18:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-26T08:18:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-26T08:18:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-26T08:18:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-26T08:18:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-26T08:18:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-26T08:18:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-26T08:18:48.383 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-26T08:18:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-26T08:18:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-26T08:18:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-26T08:18:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-26T08:18:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply-3.11-14.el9.noarch 2024-03-26T08:18:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-26T08:18:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-26T08:18:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-26T08:18:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-26T08:18:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-26T08:18:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-26T08:18:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-26T08:18:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-26T08:18:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-26T08:18:48.384 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-26T08:18:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-26T08:18:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-26T08:18:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-26T08:18:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-26T08:18:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-26T08:18:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-26T08:18:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-26T08:18:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-26T08:18:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-26T08:18:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-26T08:18:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-26T08:18:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-26T08:18:48.385 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-26T08:18:48.386 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-26T08:18:48.386 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-26T08:18:48.386 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-26T08:18:48.386 INFO:teuthology.orchestra.run.smithi043.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-26T08:18:48.386 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.386 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.386 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:48.386 INFO:teuthology.orchestra.run.smithi043.stdout: re2-1:20211101-3.el9.x86_64 2024-03-26T08:18:48.386 INFO:teuthology.orchestra.run.smithi043.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-26T08:18:48.386 INFO:teuthology.orchestra.run.smithi043.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-26T08:18:48.386 INFO:teuthology.orchestra.run.smithi043.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-26T08:18:48.386 INFO:teuthology.orchestra.run.smithi043.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-26T08:18:48.386 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-26T08:18:48.387 INFO:teuthology.orchestra.run.smithi043.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-26T08:18:48.387 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:18:48.387 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-26T08:18:48.827 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-26T08:18:48.867 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-26T08:18:48.917 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 139/198 2024-03-26T08:18:49.016 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 140/198 2024-03-26T08:18:49.062 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-26T08:18:49.073 DEBUG:teuthology.parallel:result is None 2024-03-26T08:18:49.378 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-26T08:18:49.413 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-26T08:18:49.539 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 170/198 2024-03-26T08:18:49.635 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-26T08:18:49.689 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-26T08:18:49.810 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-26T08:18:49.845 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-26T08:18:49.878 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-26T08:18:49.912 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-26T08:18:49.953 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-26T08:18:49.953 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T08:18:50.101 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T08:18:50.150 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-26T08:18:50.151 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T08:18:50.297 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T08:18:50.341 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-26T08:18:50.412 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-26T08:18:50.453 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-26T08:18:50.485 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-26T08:18:50.498 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-26T08:18:50.530 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-26T08:18:50.558 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-26T08:18:50.627 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-26T08:18:50.671 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-26T08:18:50.681 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-26T08:18:50.715 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-26T08:18:50.748 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-26T08:18:50.782 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-26T08:18:50.815 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-26T08:18:50.843 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 144/198 2024-03-26T08:18:50.853 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T08:18:50.854 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T08:18:50.884 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-26T08:18:50.953 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-26T08:18:50.953 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:18:50.953 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-26T08:18:50.954 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T08:18:50.954 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T08:18:50.954 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-26T08:18:51.026 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-26T08:18:51.064 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-26T08:18:51.164 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 147/198 2024-03-26T08:18:51.183 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T08:18:51.183 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T08:18:51.313 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T08:18:51.343 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-26T08:18:51.398 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-26T08:18:51.437 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-26T08:18:51.478 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-26T08:18:51.602 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T08:18:51.852 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T08:18:51.852 INFO:teuthology.orchestra.run.smithi052.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-26T08:18:51.852 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-26T08:18:52.152 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-26T08:18:52.186 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-26T08:18:52.252 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-26T08:18:52.252 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:18:52.252 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-26T08:18:52.252 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T08:18:52.252 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T08:18:52.253 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-26T08:18:52.329 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-26T08:18:52.399 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-26T08:18:52.399 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:18:52.399 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-26T08:18:52.399 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-26T08:18:52.488 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-26T08:18:52.568 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-26T08:18:52.569 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:18:52.569 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-26T08:18:52.569 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T08:18:52.569 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T08:18:52.569 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-26T08:18:52.922 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-26T08:18:52.992 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-26T08:18:52.992 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:18:52.992 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-26T08:18:52.992 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T08:18:52.992 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T08:18:52.992 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-26T08:18:56.580 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 1/198 2024-03-26T08:18:56.580 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2/198 2024-03-26T08:18:56.581 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 3/198 2024-03-26T08:18:56.581 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 4/198 2024-03-26T08:18:56.581 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 5/198 2024-03-26T08:18:56.581 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 6/198 2024-03-26T08:18:56.581 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 7/198 2024-03-26T08:18:56.581 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 8/198 2024-03-26T08:18:56.581 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 9/198 2024-03-26T08:18:56.581 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-26T08:18:56.581 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-26T08:18:56.581 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-26T08:18:56.581 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 13/198 2024-03-26T08:18:56.581 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 14/198 2024-03-26T08:18:56.581 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 15/198 2024-03-26T08:18:56.581 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-26T08:18:56.582 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 17/198 2024-03-26T08:18:56.582 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 18/198 2024-03-26T08:18:56.582 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 19/198 2024-03-26T08:18:56.582 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 20/198 2024-03-26T08:18:56.582 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 21/198 2024-03-26T08:18:56.582 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 22/198 2024-03-26T08:18:56.582 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 23/198 2024-03-26T08:18:56.582 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 24/198 2024-03-26T08:18:56.582 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 25/198 2024-03-26T08:18:56.599 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 26/198 2024-03-26T08:18:56.599 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 27/198 2024-03-26T08:18:56.599 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 28/198 2024-03-26T08:18:56.599 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 29/198 2024-03-26T08:18:56.599 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 30/198 2024-03-26T08:18:56.599 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 31/198 2024-03-26T08:18:56.599 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 32/198 2024-03-26T08:18:56.599 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 33/198 2024-03-26T08:18:56.600 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 34/198 2024-03-26T08:18:56.600 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 35/198 2024-03-26T08:18:56.600 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 36/198 2024-03-26T08:18:56.600 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 37/198 2024-03-26T08:18:56.600 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 38/198 2024-03-26T08:18:56.600 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 39/198 2024-03-26T08:18:56.600 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-26T08:18:56.600 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-26T08:18:56.600 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-26T08:18:56.600 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-26T08:18:56.600 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-26T08:18:56.600 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-26T08:18:56.600 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-26T08:18:56.600 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-26T08:18:56.601 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-26T08:18:56.601 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-26T08:18:56.601 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-26T08:18:56.601 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-26T08:18:56.601 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-26T08:18:56.601 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-26T08:18:56.601 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-26T08:18:56.601 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-26T08:18:56.601 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-26T08:18:56.601 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-26T08:18:56.601 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-26T08:18:56.601 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-26T08:18:56.601 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-26T08:18:56.601 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-26T08:18:56.602 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-26T08:18:56.602 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-26T08:18:56.602 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-26T08:18:56.602 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-26T08:18:56.602 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-26T08:18:56.602 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-26T08:18:56.602 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-26T08:18:56.602 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-26T08:18:56.602 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-26T08:18:56.602 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-26T08:18:56.602 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-26T08:18:56.602 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-26T08:18:56.602 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-26T08:18:56.602 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-26T08:18:56.603 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-26T08:18:56.603 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-26T08:18:56.603 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-26T08:18:56.603 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-26T08:18:56.603 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-26T08:18:56.603 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-26T08:18:56.603 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-26T08:18:56.603 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-26T08:18:56.603 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-26T08:18:56.603 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-26T08:18:56.603 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-26T08:18:56.603 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-26T08:18:56.603 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-26T08:18:56.604 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-26T08:18:56.604 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-26T08:18:56.604 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-26T08:18:56.604 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-26T08:18:56.604 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-26T08:18:56.604 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-26T08:18:56.604 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-26T08:18:56.604 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-26T08:18:56.604 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-26T08:18:56.604 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-26T08:18:56.604 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-26T08:18:56.604 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-26T08:18:56.604 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-26T08:18:56.605 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-26T08:18:56.605 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-26T08:18:56.605 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-26T08:18:56.605 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-26T08:18:56.605 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-26T08:18:56.605 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-26T08:18:56.605 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-26T08:18:56.605 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-26T08:18:56.605 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-26T08:18:56.605 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-26T08:18:56.605 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-26T08:18:56.605 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-26T08:18:56.605 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-26T08:18:56.605 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-26T08:18:56.606 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-26T08:18:56.606 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-26T08:18:56.606 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-26T08:18:56.606 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-26T08:18:56.606 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-26T08:18:56.606 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-26T08:18:56.606 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-26T08:18:56.606 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-26T08:18:56.606 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-26T08:18:56.606 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-26T08:18:56.606 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-26T08:18:56.606 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-26T08:18:56.606 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-26T08:18:56.606 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-26T08:18:56.606 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-26T08:18:56.607 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-26T08:18:56.607 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-26T08:18:56.607 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-26T08:18:56.607 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-26T08:18:56.607 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-26T08:18:56.607 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-26T08:18:56.607 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-26T08:18:56.607 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-26T08:18:56.607 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-26T08:18:56.607 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-26T08:18:56.607 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-26T08:18:56.607 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-26T08:18:56.607 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-26T08:18:56.607 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-26T08:18:56.607 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-26T08:18:56.607 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-26T08:18:56.608 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-26T08:18:56.608 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-26T08:18:56.608 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-26T08:18:56.608 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-26T08:18:56.608 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-26T08:18:56.608 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-26T08:18:56.608 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-26T08:18:56.608 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-26T08:18:56.608 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-26T08:18:56.608 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-26T08:18:56.608 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-26T08:18:56.608 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-26T08:18:56.608 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-26T08:18:56.608 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-26T08:18:56.608 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-26T08:18:56.609 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-26T08:18:56.609 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-26T08:18:56.609 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-26T08:18:56.609 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-26T08:18:56.609 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-26T08:18:56.609 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-26T08:18:56.609 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-26T08:18:56.609 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-26T08:18:56.609 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-26T08:18:56.609 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-26T08:18:56.609 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-26T08:18:56.609 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-26T08:18:56.609 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-26T08:18:56.609 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-26T08:18:56.609 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-26T08:18:56.610 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-26T08:18:56.610 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-26T08:18:56.610 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-26T08:18:56.610 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-26T08:18:56.610 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-26T08:18:56.610 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-26T08:18:56.610 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-26T08:18:56.610 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-26T08:18:56.610 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-26T08:18:56.610 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-26T08:18:56.610 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-26T08:18:56.610 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-26T08:18:56.610 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-26T08:18:56.611 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-26T08:18:56.611 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-26T08:18:56.611 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-26T08:18:56.611 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-26T08:18:56.611 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-26T08:18:56.611 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-26T08:18:56.611 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-26T08:18:56.611 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-26T08:18:57.814 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-26T08:18:57.870 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 154/198 2024-03-26T08:18:57.911 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 155/198 2024-03-26T08:18:57.949 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-26T08:18:58.227 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 157/198 2024-03-26T08:18:58.354 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-26T08:18:58.410 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 159/198 2024-03-26T08:18:58.464 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-26T08:18:58.535 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-26T08:18:58.589 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-26T08:18:58.623 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 163/198 2024-03-26T08:18:58.660 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-26T08:18:58.711 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-26T08:18:58.798 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-26T08:18:58.940 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-26T08:18:58.978 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-26T08:18:59.113 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-26T08:18:59.113 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-26T08:18:59.113 INFO:teuthology.orchestra.run.smithi052.stdout:Upgraded: 2024-03-26T08:18:59.113 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-26T08:18:59.113 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-26T08:18:59.113 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-26T08:18:59.113 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-26T08:18:59.113 INFO:teuthology.orchestra.run.smithi052.stdout: acl-2.3.1-4.el9.x86_64 2024-03-26T08:18:59.113 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-26T08:18:59.113 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-26T08:18:59.113 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-26T08:18:59.114 INFO:teuthology.orchestra.run.smithi052.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-26T08:18:59.114 INFO:teuthology.orchestra.run.smithi052.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-26T08:18:59.114 INFO:teuthology.orchestra.run.smithi052.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-26T08:18:59.114 INFO:teuthology.orchestra.run.smithi052.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-26T08:18:59.114 INFO:teuthology.orchestra.run.smithi052.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-26T08:18:59.114 INFO:teuthology.orchestra.run.smithi052.stdout: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.114 INFO:teuthology.orchestra.run.smithi052.stdout: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.114 INFO:teuthology.orchestra.run.smithi052.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-26T08:18:59.114 INFO:teuthology.orchestra.run.smithi052.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-26T08:18:59.114 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-26T08:18:59.114 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-26T08:18:59.114 INFO:teuthology.orchestra.run.smithi052.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-26T08:18:59.114 INFO:teuthology.orchestra.run.smithi052.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-26T08:18:59.115 INFO:teuthology.orchestra.run.smithi052.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-26T08:18:59.115 INFO:teuthology.orchestra.run.smithi052.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-26T08:18:59.115 INFO:teuthology.orchestra.run.smithi052.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-26T08:18:59.115 INFO:teuthology.orchestra.run.smithi052.stdout:Installed: 2024-03-26T08:18:59.115 INFO:teuthology.orchestra.run.smithi052.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-26T08:18:59.115 INFO:teuthology.orchestra.run.smithi052.stdout: automake-1.16.2-8.el9.noarch 2024-03-26T08:18:59.115 INFO:teuthology.orchestra.run.smithi052.stdout: bison-3.7.4-5.el9.x86_64 2024-03-26T08:18:59.115 INFO:teuthology.orchestra.run.smithi052.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-26T08:18:59.115 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.115 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.115 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.115 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.115 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:18:59.115 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.115 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.116 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.116 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:18:59.116 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:18:59.116 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:18:59.116 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:18:59.116 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:18:59.116 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.116 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.116 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:18:59.116 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.116 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.116 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.116 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:18:59.117 INFO:teuthology.orchestra.run.smithi052.stdout: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:18:59.117 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.117 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.117 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:18:59.117 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-26T08:18:59.117 INFO:teuthology.orchestra.run.smithi052.stdout: flex-2.6.4-9.el9.x86_64 2024-03-26T08:18:59.117 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-26T08:18:59.117 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-26T08:18:59.117 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-26T08:18:59.117 INFO:teuthology.orchestra.run.smithi052.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-26T08:18:59.117 INFO:teuthology.orchestra.run.smithi052.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-26T08:18:59.117 INFO:teuthology.orchestra.run.smithi052.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-26T08:18:59.117 INFO:teuthology.orchestra.run.smithi052.stdout: jq-1.6-16.el9.x86_64 2024-03-26T08:18:59.117 INFO:teuthology.orchestra.run.smithi052.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-26T08:18:59.117 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-26T08:18:59.118 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-26T08:18:59.118 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.118 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.118 INFO:teuthology.orchestra.run.smithi052.stdout: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.118 INFO:teuthology.orchestra.run.smithi052.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-26T08:18:59.118 INFO:teuthology.orchestra.run.smithi052.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-26T08:18:59.118 INFO:teuthology.orchestra.run.smithi052.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-26T08:18:59.118 INFO:teuthology.orchestra.run.smithi052.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-26T08:18:59.118 INFO:teuthology.orchestra.run.smithi052.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-26T08:18:59.118 INFO:teuthology.orchestra.run.smithi052.stdout: librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.118 INFO:teuthology.orchestra.run.smithi052.stdout: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.118 INFO:teuthology.orchestra.run.smithi052.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-26T08:18:59.118 INFO:teuthology.orchestra.run.smithi052.stdout: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.119 INFO:teuthology.orchestra.run.smithi052.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-26T08:18:59.119 INFO:teuthology.orchestra.run.smithi052.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-26T08:18:59.119 INFO:teuthology.orchestra.run.smithi052.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-26T08:18:59.119 INFO:teuthology.orchestra.run.smithi052.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-26T08:18:59.119 INFO:teuthology.orchestra.run.smithi052.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-26T08:18:59.119 INFO:teuthology.orchestra.run.smithi052.stdout: lua-5.4.4-4.el9.x86_64 2024-03-26T08:18:59.119 INFO:teuthology.orchestra.run.smithi052.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-26T08:18:59.119 INFO:teuthology.orchestra.run.smithi052.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-26T08:18:59.119 INFO:teuthology.orchestra.run.smithi052.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-26T08:18:59.119 INFO:teuthology.orchestra.run.smithi052.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-26T08:18:59.119 INFO:teuthology.orchestra.run.smithi052.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-26T08:18:59.119 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-26T08:18:59.119 INFO:teuthology.orchestra.run.smithi052.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-26T08:18:59.119 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-13.14-1.el9.x86_64 2024-03-26T08:18:59.119 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-26T08:18:59.119 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-26T08:18:59.120 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-26T08:18:59.120 INFO:teuthology.orchestra.run.smithi052.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-26T08:18:59.120 INFO:teuthology.orchestra.run.smithi052.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-26T08:18:59.120 INFO:teuthology.orchestra.run.smithi052.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-26T08:18:59.120 INFO:teuthology.orchestra.run.smithi052.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-26T08:18:59.120 INFO:teuthology.orchestra.run.smithi052.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-26T08:18:59.120 INFO:teuthology.orchestra.run.smithi052.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-26T08:18:59.120 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-26T08:18:59.120 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.120 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.120 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.120 INFO:teuthology.orchestra.run.smithi052.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-26T08:18:59.121 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-26T08:18:59.121 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-26T08:18:59.121 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-26T08:18:59.121 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-26T08:18:59.121 INFO:teuthology.orchestra.run.smithi052.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-26T08:18:59.121 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-26T08:18:59.121 INFO:teuthology.orchestra.run.smithi052.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-26T08:18:59.121 INFO:teuthology.orchestra.run.smithi052.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-26T08:18:59.121 INFO:teuthology.orchestra.run.smithi052.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-26T08:18:59.121 INFO:teuthology.orchestra.run.smithi052.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-26T08:18:59.121 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-26T08:18:59.121 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-26T08:18:59.121 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-26T08:18:59.121 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-26T08:18:59.122 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-26T08:18:59.122 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-26T08:18:59.122 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-26T08:18:59.122 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-26T08:18:59.122 INFO:teuthology.orchestra.run.smithi052.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-26T08:18:59.122 INFO:teuthology.orchestra.run.smithi052.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-26T08:18:59.122 INFO:teuthology.orchestra.run.smithi052.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-26T08:18:59.122 INFO:teuthology.orchestra.run.smithi052.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-26T08:18:59.122 INFO:teuthology.orchestra.run.smithi052.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-26T08:18:59.122 INFO:teuthology.orchestra.run.smithi052.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-26T08:18:59.122 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-26T08:18:59.122 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-26T08:18:59.123 INFO:teuthology.orchestra.run.smithi052.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-26T08:18:59.123 INFO:teuthology.orchestra.run.smithi052.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-26T08:18:59.123 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-26T08:18:59.123 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ply-3.11-14.el9.noarch 2024-03-26T08:18:59.123 INFO:teuthology.orchestra.run.smithi052.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-26T08:18:59.123 INFO:teuthology.orchestra.run.smithi052.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-26T08:18:59.123 INFO:teuthology.orchestra.run.smithi052.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-26T08:18:59.123 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-26T08:18:59.123 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-26T08:18:59.123 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-26T08:18:59.123 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-26T08:18:59.123 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-26T08:18:59.123 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-26T08:18:59.123 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.124 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.124 INFO:teuthology.orchestra.run.smithi052.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-26T08:18:59.124 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-26T08:18:59.124 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-26T08:18:59.124 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.124 INFO:teuthology.orchestra.run.smithi052.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-26T08:18:59.124 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-26T08:18:59.124 INFO:teuthology.orchestra.run.smithi052.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-26T08:18:59.124 INFO:teuthology.orchestra.run.smithi052.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-26T08:18:59.124 INFO:teuthology.orchestra.run.smithi052.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-26T08:18:59.124 INFO:teuthology.orchestra.run.smithi052.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-26T08:18:59.124 INFO:teuthology.orchestra.run.smithi052.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-26T08:18:59.124 INFO:teuthology.orchestra.run.smithi052.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-26T08:18:59.124 INFO:teuthology.orchestra.run.smithi052.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-26T08:18:59.124 INFO:teuthology.orchestra.run.smithi052.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-26T08:18:59.125 INFO:teuthology.orchestra.run.smithi052.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-26T08:18:59.125 INFO:teuthology.orchestra.run.smithi052.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-26T08:18:59.125 INFO:teuthology.orchestra.run.smithi052.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-26T08:18:59.125 INFO:teuthology.orchestra.run.smithi052.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-26T08:18:59.125 INFO:teuthology.orchestra.run.smithi052.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-26T08:18:59.125 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.125 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.125 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:18:59.125 INFO:teuthology.orchestra.run.smithi052.stdout: re2-1:20211101-3.el9.x86_64 2024-03-26T08:18:59.125 INFO:teuthology.orchestra.run.smithi052.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-26T08:18:59.125 INFO:teuthology.orchestra.run.smithi052.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-26T08:18:59.125 INFO:teuthology.orchestra.run.smithi052.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-26T08:18:59.125 INFO:teuthology.orchestra.run.smithi052.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-26T08:18:59.125 INFO:teuthology.orchestra.run.smithi052.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-26T08:18:59.126 INFO:teuthology.orchestra.run.smithi052.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-26T08:18:59.126 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-26T08:18:59.126 INFO:teuthology.orchestra.run.smithi052.stdout:Complete! 2024-03-26T08:18:59.800 DEBUG:teuthology.parallel:result is None 2024-03-26T08:18:59.886 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T08:18:59.933 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T08:18:59.988 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T08:19:01.265 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 170/198 2024-03-26T08:19:01.355 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-26T08:19:01.399 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-26T08:19:01.520 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-26T08:19:01.557 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-26T08:19:01.590 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-26T08:19:01.625 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-26T08:19:01.667 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-26T08:19:01.667 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T08:19:01.795 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T08:19:01.838 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-26T08:19:01.838 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T08:19:01.967 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T08:19:02.011 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-26T08:19:02.058 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-26T08:19:02.116 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-26T08:19:02.148 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-26T08:19:02.216 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-26T08:19:02.289 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-26T08:19:02.344 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-26T08:19:02.378 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-26T08:19:02.411 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-26T08:19:02.445 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-26T08:19:02.477 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-26T08:19:02.513 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T08:19:02.514 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T08:19:02.837 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T08:19:02.837 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T08:19:02.976 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T08:19:03.006 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-26T08:19:03.060 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-26T08:19:03.093 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-26T08:19:03.133 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-26T08:19:03.279 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T08:19:03.591 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T08:19:03.591 INFO:teuthology.orchestra.run.smithi183.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-26T08:19:03.591 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-26T08:19:08.443 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 1/198 2024-03-26T08:19:08.443 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2/198 2024-03-26T08:19:08.443 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 3/198 2024-03-26T08:19:08.443 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 4/198 2024-03-26T08:19:08.443 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 5/198 2024-03-26T08:19:08.443 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 6/198 2024-03-26T08:19:08.443 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 7/198 2024-03-26T08:19:08.444 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 8/198 2024-03-26T08:19:08.444 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 9/198 2024-03-26T08:19:08.444 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-26T08:19:08.444 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-26T08:19:08.444 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-26T08:19:08.444 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 13/198 2024-03-26T08:19:08.444 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 14/198 2024-03-26T08:19:08.444 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 15/198 2024-03-26T08:19:08.444 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-26T08:19:08.444 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 17/198 2024-03-26T08:19:08.444 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 18/198 2024-03-26T08:19:08.444 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 19/198 2024-03-26T08:19:08.444 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 20/198 2024-03-26T08:19:08.444 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 21/198 2024-03-26T08:19:08.445 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 22/198 2024-03-26T08:19:08.445 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 23/198 2024-03-26T08:19:08.445 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 24/198 2024-03-26T08:19:08.445 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 25/198 2024-03-26T08:19:08.461 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 26/198 2024-03-26T08:19:08.461 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 27/198 2024-03-26T08:19:08.461 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 28/198 2024-03-26T08:19:08.462 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 29/198 2024-03-26T08:19:08.462 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 30/198 2024-03-26T08:19:08.462 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 31/198 2024-03-26T08:19:08.462 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 32/198 2024-03-26T08:19:08.462 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 33/198 2024-03-26T08:19:08.462 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 34/198 2024-03-26T08:19:08.462 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 35/198 2024-03-26T08:19:08.462 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 36/198 2024-03-26T08:19:08.462 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 37/198 2024-03-26T08:19:08.462 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 38/198 2024-03-26T08:19:08.462 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 39/198 2024-03-26T08:19:08.463 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-26T08:19:08.463 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-26T08:19:08.463 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-26T08:19:08.463 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-26T08:19:08.463 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-26T08:19:08.463 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-26T08:19:08.463 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-26T08:19:08.463 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-26T08:19:08.463 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-26T08:19:08.463 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-26T08:19:08.463 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-26T08:19:08.464 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-26T08:19:08.464 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-26T08:19:08.464 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-26T08:19:08.464 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-26T08:19:08.464 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-26T08:19:08.464 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-26T08:19:08.464 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-26T08:19:08.464 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-26T08:19:08.464 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-26T08:19:08.464 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-26T08:19:08.464 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-26T08:19:08.464 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-26T08:19:08.464 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-26T08:19:08.465 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-26T08:19:08.465 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-26T08:19:08.465 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-26T08:19:08.465 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-26T08:19:08.465 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-26T08:19:08.465 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-26T08:19:08.465 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-26T08:19:08.465 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-26T08:19:08.465 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-26T08:19:08.465 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-26T08:19:08.465 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-26T08:19:08.465 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-26T08:19:08.466 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-26T08:19:08.466 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-26T08:19:08.466 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-26T08:19:08.466 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-26T08:19:08.466 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-26T08:19:08.466 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-26T08:19:08.466 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-26T08:19:08.466 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-26T08:19:08.466 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-26T08:19:08.466 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-26T08:19:08.466 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-26T08:19:08.466 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-26T08:19:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-26T08:19:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-26T08:19:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-26T08:19:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-26T08:19:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-26T08:19:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-26T08:19:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-26T08:19:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-26T08:19:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-26T08:19:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-26T08:19:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-26T08:19:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-26T08:19:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-26T08:19:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-26T08:19:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-26T08:19:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-26T08:19:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-26T08:19:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-26T08:19:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-26T08:19:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-26T08:19:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-26T08:19:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-26T08:19:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-26T08:19:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-26T08:19:08.469 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-26T08:19:08.469 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-26T08:19:08.469 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-26T08:19:08.469 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-26T08:19:08.469 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-26T08:19:08.469 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-26T08:19:08.469 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-26T08:19:08.469 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-26T08:19:08.469 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-26T08:19:08.469 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-26T08:19:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-26T08:19:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-26T08:19:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-26T08:19:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-26T08:19:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-26T08:19:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-26T08:19:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-26T08:19:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-26T08:19:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-26T08:19:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-26T08:19:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-26T08:19:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-26T08:19:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-26T08:19:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-26T08:19:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-26T08:19:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-26T08:19:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-26T08:19:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-26T08:19:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-26T08:19:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-26T08:19:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-26T08:19:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-26T08:19:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-26T08:19:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-26T08:19:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-26T08:19:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-26T08:19:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-26T08:19:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-26T08:19:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-26T08:19:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-26T08:19:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-26T08:19:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-26T08:19:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-26T08:19:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-26T08:19:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-26T08:19:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-26T08:19:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-26T08:19:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-26T08:19:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-26T08:19:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-26T08:19:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-26T08:19:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-26T08:19:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-26T08:19:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-26T08:19:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-26T08:19:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-26T08:19:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-26T08:19:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-26T08:19:08.474 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-26T08:19:08.474 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-26T08:19:08.474 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-26T08:19:08.474 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-26T08:19:08.474 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-26T08:19:08.474 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-26T08:19:08.474 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-26T08:19:08.474 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-26T08:19:08.474 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-26T08:19:08.474 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-26T08:19:08.474 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-26T08:19:08.474 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-26T08:19:08.474 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-26T08:19:08.474 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-26T08:19:08.475 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-26T08:19:08.475 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-26T08:19:08.475 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-26T08:19:08.475 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-26T08:19:08.475 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-26T08:19:08.475 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-26T08:19:08.475 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-26T08:19:08.475 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-26T08:19:08.475 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-26T08:19:08.475 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-26T08:19:08.475 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-26T08:19:08.475 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-26T08:19:08.475 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-26T08:19:08.476 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-26T08:19:10.946 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-26T08:19:10.946 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-26T08:19:10.946 INFO:teuthology.orchestra.run.smithi183.stdout:Upgraded: 2024-03-26T08:19:10.946 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-26T08:19:10.946 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-26T08:19:10.946 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-26T08:19:10.946 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-26T08:19:10.946 INFO:teuthology.orchestra.run.smithi183.stdout: acl-2.3.1-4.el9.x86_64 2024-03-26T08:19:10.947 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-26T08:19:10.947 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-26T08:19:10.947 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-26T08:19:10.947 INFO:teuthology.orchestra.run.smithi183.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-26T08:19:10.947 INFO:teuthology.orchestra.run.smithi183.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-26T08:19:10.947 INFO:teuthology.orchestra.run.smithi183.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-26T08:19:10.947 INFO:teuthology.orchestra.run.smithi183.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-26T08:19:10.947 INFO:teuthology.orchestra.run.smithi183.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-26T08:19:10.947 INFO:teuthology.orchestra.run.smithi183.stdout: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.947 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.947 INFO:teuthology.orchestra.run.smithi183.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-26T08:19:10.947 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-26T08:19:10.947 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-26T08:19:10.948 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-26T08:19:10.948 INFO:teuthology.orchestra.run.smithi183.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-26T08:19:10.948 INFO:teuthology.orchestra.run.smithi183.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-26T08:19:10.948 INFO:teuthology.orchestra.run.smithi183.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-26T08:19:10.948 INFO:teuthology.orchestra.run.smithi183.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-26T08:19:10.948 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-26T08:19:10.948 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-03-26T08:19:10.948 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-26T08:19:10.948 INFO:teuthology.orchestra.run.smithi183.stdout: automake-1.16.2-8.el9.noarch 2024-03-26T08:19:10.948 INFO:teuthology.orchestra.run.smithi183.stdout: bison-3.7.4-5.el9.x86_64 2024-03-26T08:19:10.948 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-26T08:19:10.948 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.948 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.948 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.949 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.949 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:19:10.949 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.949 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.949 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.949 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:19:10.949 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:19:10.949 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:19:10.949 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:19:10.949 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:19:10.949 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.949 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.949 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:19:10.949 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.949 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.950 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.950 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:19:10.950 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:19:10.950 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.950 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.950 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:19:10.950 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-26T08:19:10.950 INFO:teuthology.orchestra.run.smithi183.stdout: flex-2.6.4-9.el9.x86_64 2024-03-26T08:19:10.950 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-26T08:19:10.950 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-26T08:19:10.950 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-26T08:19:10.950 INFO:teuthology.orchestra.run.smithi183.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-26T08:19:10.950 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-26T08:19:10.950 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-26T08:19:10.951 INFO:teuthology.orchestra.run.smithi183.stdout: jq-1.6-16.el9.x86_64 2024-03-26T08:19:10.951 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-26T08:19:10.951 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-26T08:19:10.951 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-26T08:19:10.951 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.951 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.951 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.951 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-26T08:19:10.951 INFO:teuthology.orchestra.run.smithi183.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-26T08:19:10.951 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-26T08:19:10.951 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-26T08:19:10.951 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-26T08:19:10.951 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.951 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.951 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-26T08:19:10.951 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.951 INFO:teuthology.orchestra.run.smithi183.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-26T08:19:10.952 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-26T08:19:10.952 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-26T08:19:10.952 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-26T08:19:10.952 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-26T08:19:10.952 INFO:teuthology.orchestra.run.smithi183.stdout: lua-5.4.4-4.el9.x86_64 2024-03-26T08:19:10.952 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-26T08:19:10.952 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-26T08:19:10.952 INFO:teuthology.orchestra.run.smithi183.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-26T08:19:10.952 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-26T08:19:10.952 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-26T08:19:10.952 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-26T08:19:10.952 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-26T08:19:10.952 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-13.14-1.el9.x86_64 2024-03-26T08:19:10.952 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-26T08:19:10.952 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-26T08:19:10.953 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-26T08:19:10.953 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-26T08:19:10.953 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-26T08:19:10.953 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-26T08:19:10.953 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-26T08:19:10.953 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-26T08:19:10.953 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-26T08:19:10.953 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-26T08:19:10.953 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.953 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.953 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.953 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-26T08:19:10.953 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-26T08:19:10.953 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-26T08:19:10.953 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-26T08:19:10.954 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-26T08:19:10.954 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-26T08:19:10.954 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-26T08:19:10.954 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-26T08:19:10.954 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-26T08:19:10.954 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-26T08:19:10.954 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-26T08:19:10.955 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-26T08:19:10.955 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-26T08:19:10.955 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-26T08:19:10.955 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-26T08:19:10.955 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-26T08:19:10.956 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-26T08:19:10.956 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-26T08:19:10.956 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-26T08:19:10.956 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-26T08:19:10.956 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-26T08:19:10.956 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-26T08:19:10.956 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-26T08:19:10.956 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-26T08:19:10.957 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-26T08:19:10.957 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-26T08:19:10.957 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-26T08:19:10.957 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-26T08:19:10.957 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-26T08:19:10.957 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-26T08:19:10.957 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply-3.11-14.el9.noarch 2024-03-26T08:19:10.957 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-26T08:19:10.957 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-26T08:19:10.958 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-26T08:19:10.958 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-26T08:19:10.958 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-26T08:19:10.958 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-26T08:19:10.958 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-26T08:19:10.958 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-26T08:19:10.958 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-26T08:19:10.958 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.959 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.959 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-26T08:19:10.959 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-26T08:19:10.959 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-26T08:19:10.959 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.959 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-26T08:19:10.959 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-26T08:19:10.960 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-26T08:19:10.960 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-26T08:19:10.960 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-26T08:19:10.960 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-26T08:19:10.960 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-26T08:19:10.960 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-26T08:19:10.960 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-26T08:19:10.960 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-26T08:19:10.960 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-26T08:19:10.961 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-26T08:19:10.961 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-26T08:19:10.961 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-26T08:19:10.961 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-26T08:19:10.961 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.961 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.961 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:19:10.961 INFO:teuthology.orchestra.run.smithi183.stdout: re2-1:20211101-3.el9.x86_64 2024-03-26T08:19:10.961 INFO:teuthology.orchestra.run.smithi183.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-26T08:19:10.961 INFO:teuthology.orchestra.run.smithi183.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-26T08:19:10.961 INFO:teuthology.orchestra.run.smithi183.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-26T08:19:10.961 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-26T08:19:10.961 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-26T08:19:10.961 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-26T08:19:10.962 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-26T08:19:10.962 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-03-26T08:19:11.584 DEBUG:teuthology.parallel:result is None 2024-03-26T08:19:11.585 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T08:19:11.731 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-26T08:19:11.809 INFO:teuthology.orchestra.run.smithi043.stdout:19.0.0-2145.g3be9a5c9.el9 2024-03-26T08:19:11.810 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2145.g3be9a5c9.el9 2024-03-26T08:19:11.810 INFO:teuthology.task.install:The correct ceph version 19.0.0-2145.g3be9a5c9 is installed. 2024-03-26T08:19:11.812 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T08:19:12.034 DEBUG:teuthology.orchestra.run.smithi052:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-26T08:19:12.110 INFO:teuthology.orchestra.run.smithi052.stdout:19.0.0-2145.g3be9a5c9.el9 2024-03-26T08:19:12.111 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2145.g3be9a5c9.el9 2024-03-26T08:19:12.111 INFO:teuthology.task.install:The correct ceph version 19.0.0-2145.g3be9a5c9 is installed. 2024-03-26T08:19:12.113 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T08:19:12.333 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-26T08:19:12.391 INFO:teuthology.orchestra.run.smithi183.stdout:19.0.0-2145.g3be9a5c9.el9 2024-03-26T08:19:12.392 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2145.g3be9a5c9.el9 2024-03-26T08:19:12.392 INFO:teuthology.task.install:The correct ceph version 19.0.0-2145.g3be9a5c9 is installed. 2024-03-26T08:19:12.394 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-26T08:19:12.394 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-26T08:19:12.394 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-26T08:19:12.504 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-26T08:19:12.504 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-26T08:19:12.610 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-03-26T08:19:12.610 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-26T08:19:12.705 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-26T08:19:12.705 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-26T08:19:12.705 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/daemon-helper 2024-03-26T08:19:12.812 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-26T08:19:12.959 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-26T08:19:12.959 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/daemon-helper 2024-03-26T08:19:13.064 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-26T08:19:13.207 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-03-26T08:19:13.207 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/daemon-helper 2024-03-26T08:19:13.288 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-26T08:19:13.408 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-26T08:19:13.408 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-26T08:19:13.408 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-26T08:19:13.516 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-26T08:19:13.663 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-26T08:19:13.663 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-26T08:19:13.768 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-26T08:19:13.913 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-03-26T08:19:13.913 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-26T08:19:13.988 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-26T08:19:14.094 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-26T08:19:14.094 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-26T08:19:14.094 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/stdin-killer 2024-03-26T08:19:14.199 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-26T08:19:14.300 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-26T08:19:14.300 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/stdin-killer 2024-03-26T08:19:14.406 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-26T08:19:14.503 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-03-26T08:19:14.503 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/stdin-killer 2024-03-26T08:19:14.578 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-26T08:19:14.685 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-26T08:19:14.960 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 1, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-03-26T08:19:14.960 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T08:19:14.961 INFO:tasks.cephadm:Cluster fsid is 88759420-eb49-11ee-95d0-87774f69a715 2024-03-26T08:19:14.961 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-26T08:19:14.961 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.43', 'mon.b': '172.21.15.52', 'mon.c': '172.21.15.183'} 2024-03-26T08:19:14.961 INFO:tasks.cephadm:First mon is mon.a on smithi043 2024-03-26T08:19:14.961 INFO:tasks.cephadm:First mgr is x 2024-03-26T08:19:14.961 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-26T08:19:14.961 DEBUG:teuthology.orchestra.run.smithi043:> sudo hostname $(hostname -s) 2024-03-26T08:19:15.070 DEBUG:teuthology.orchestra.run.smithi052:> sudo hostname $(hostname -s) 2024-03-26T08:19:15.175 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname $(hostname -s) 2024-03-26T08:19:15.252 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-26T08:19:15.252 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T08:19:15.532 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77877/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2145-g3be9a5c9', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2145.g3be9a5c9'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-22 05:51:42.267384', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-mchangir-testing-main-20240318.032620', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-26T08:19:15.672 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-mchangir-testing-main-20240318.032620, sha1 3be9a5c9bc793e11e2800a8c0c696e8b46742033 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T08:19:15.674 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/x86_64/flavors/default/cephadm 2024-03-26T08:19:15.674 DEBUG:teuthology.orchestra.run.smithi043:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-26T08:19:16.421 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r--. 1 ubuntu ubuntu 767371 Mar 26 08:19 /home/ubuntu/cephtest/cephadm 2024-03-26T08:19:16.421 DEBUG:teuthology.orchestra.run.smithi052:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-26T08:19:16.974 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r--. 1 ubuntu ubuntu 767371 Mar 26 08:19 /home/ubuntu/cephtest/cephadm 2024-03-26T08:19:16.974 DEBUG:teuthology.orchestra.run.smithi183:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-26T08:19:17.518 INFO:teuthology.orchestra.run.smithi183.stdout:-rw-r--r--. 1 ubuntu ubuntu 767371 Mar 26 08:19 /home/ubuntu/cephtest/cephadm 2024-03-26T08:19:17.518 DEBUG:teuthology.orchestra.run.smithi043:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-26T08:19:17.582 DEBUG:teuthology.orchestra.run.smithi052:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-26T08:19:17.645 DEBUG:teuthology.orchestra.run.smithi183:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-26T08:19:17.719 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 on all hosts... 2024-03-26T08:19:17.720 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 pull 2024-03-26T08:19:17.727 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 pull 2024-03-26T08:19:17.734 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 pull 2024-03-26T08:19:18.900 INFO:teuthology.orchestra.run.smithi043.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-26T08:19:18.965 INFO:teuthology.orchestra.run.smithi052.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-26T08:19:19.168 INFO:teuthology.orchestra.run.smithi183.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-26T08:19:52.262 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-03-26T08:19:52.263 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-26T08:19:52.263 INFO:teuthology.orchestra.run.smithi183.stdout: "image_id": "c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2", 2024-03-26T08:19:52.263 INFO:teuthology.orchestra.run.smithi183.stdout: "repo_digests": [ 2024-03-26T08:19:52.263 INFO:teuthology.orchestra.run.smithi183.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc" 2024-03-26T08:19:52.263 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-03-26T08:19:52.263 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-03-26T08:19:58.806 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-03-26T08:19:58.806 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-26T08:19:58.806 INFO:teuthology.orchestra.run.smithi052.stdout: "image_id": "c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2", 2024-03-26T08:19:58.807 INFO:teuthology.orchestra.run.smithi052.stdout: "repo_digests": [ 2024-03-26T08:19:58.807 INFO:teuthology.orchestra.run.smithi052.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc" 2024-03-26T08:19:58.807 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-03-26T08:19:58.807 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-03-26T08:20:05.750 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-26T08:20:05.751 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-26T08:20:05.751 INFO:teuthology.orchestra.run.smithi043.stdout: "image_id": "c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2", 2024-03-26T08:20:05.751 INFO:teuthology.orchestra.run.smithi043.stdout: "repo_digests": [ 2024-03-26T08:20:05.751 INFO:teuthology.orchestra.run.smithi043.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc" 2024-03-26T08:20:05.751 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-03-26T08:20:05.751 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-26T08:20:05.802 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /etc/ceph 2024-03-26T08:20:05.912 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /etc/ceph 2024-03-26T08:20:06.020 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /etc/ceph 2024-03-26T08:20:06.124 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 777 /etc/ceph 2024-03-26T08:20:06.231 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 777 /etc/ceph 2024-03-26T08:20:06.320 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 777 /etc/ceph 2024-03-26T08:20:06.413 INFO:tasks.cephadm:Writing seed config... 2024-03-26T08:20:06.416 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-26T08:20:06.416 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-26T08:20:06.416 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-26T08:20:06.416 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-26T08:20:06.416 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-26T08:20:06.416 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-26T08:20:06.417 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-26T08:20:06.417 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-26T08:20:06.417 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-26T08:20:06.417 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-26T08:20:06.417 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-26T08:20:06.417 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-26T08:20:06.417 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-26T08:20:06.417 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-26T08:20:06.417 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-26T08:20:06.417 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-26T08:20:06.417 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-26T08:20:06.417 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-26T08:20:06.417 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-26T08:20:06.417 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-26T08:20:06.417 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-26T08:20:06.418 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-26T08:20:06.418 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-26T08:20:06.418 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-26T08:20:06.418 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-26T08:20:06.418 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-26T08:20:06.418 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-26T08:20:06.418 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-03-26T08:20:06.418 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-26T08:20:06.418 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-26T08:20:06.418 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-26T08:20:06.418 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-26T08:20:06.418 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-26T08:20:06.418 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-26T08:20:06.419 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-26T08:20:06.419 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-26T08:20:06.419 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-26T08:20:06.419 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-26T08:20:06.419 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-26T08:20:06.419 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-26T08:20:06.419 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-26T08:20:06.419 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-03-26T08:20:06.420 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-26T08:20:06.420 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-26T08:20:06.480 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 88759420-eb49-11ee-95d0-87774f69a715 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-26T08:20:06.481 DEBUG:teuthology.orchestra.run.smithi043:mon.a> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mon.a.service 2024-03-26T08:20:06.528 DEBUG:teuthology.orchestra.run.smithi043:mgr.x> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mgr.x.service 2024-03-26T08:20:06.574 INFO:tasks.cephadm:Bootstrapping... 2024-03-26T08:20:06.574 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 -v bootstrap --fsid 88759420-eb49-11ee-95d0-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.43 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-26T08:20:06.862 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-03-26T08:20:06.862 INFO:teuthology.orchestra.run.smithi043.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033', '-v', 'bootstrap', '--fsid', '88759420-eb49-11ee-95d0-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.43', '--skip-admin-label'] 2024-03-26T08:20:06.925 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-26T08:20:06.925 INFO:teuthology.orchestra.run.smithi043.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-26T08:20:06.926 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying podman|docker is present... 2024-03-26T08:20:06.984 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-26T08:20:06.984 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying lvm2 is present... 2024-03-26T08:20:06.984 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying time synchronization is in place... 2024-03-26T08:20:07.004 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-26T08:20:07.004 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-26T08:20:07.025 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-26T08:20:07.026 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-03-26T08:20:07.049 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout enabled 2024-03-26T08:20:07.071 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout active 2024-03-26T08:20:07.072 INFO:teuthology.orchestra.run.smithi043.stdout:Unit chronyd.service is enabled and running 2024-03-26T08:20:07.072 INFO:teuthology.orchestra.run.smithi043.stdout:Repeating the final host check... 2024-03-26T08:20:07.129 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-26T08:20:07.129 INFO:teuthology.orchestra.run.smithi043.stdout:podman (/bin/podman) version 4.9.4 is present 2024-03-26T08:20:07.129 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl is present 2024-03-26T08:20:07.129 INFO:teuthology.orchestra.run.smithi043.stdout:lvcreate is present 2024-03-26T08:20:07.143 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-26T08:20:07.143 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-26T08:20:07.158 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-26T08:20:07.158 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-03-26T08:20:07.173 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout enabled 2024-03-26T08:20:07.190 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout active 2024-03-26T08:20:07.190 INFO:teuthology.orchestra.run.smithi043.stdout:Unit chronyd.service is enabled and running 2024-03-26T08:20:07.190 INFO:teuthology.orchestra.run.smithi043.stdout:Host looks OK 2024-03-26T08:20:07.190 INFO:teuthology.orchestra.run.smithi043.stdout:Cluster fsid: 88759420-eb49-11ee-95d0-87774f69a715 2024-03-26T08:20:07.190 INFO:teuthology.orchestra.run.smithi043.stdout:Acquiring lock 140374368583104 on /run/cephadm/88759420-eb49-11ee-95d0-87774f69a715.lock 2024-03-26T08:20:07.190 INFO:teuthology.orchestra.run.smithi043.stdout:Lock 140374368583104 acquired on /run/cephadm/88759420-eb49-11ee-95d0-87774f69a715.lock 2024-03-26T08:20:07.191 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying IP 172.21.15.43 port 3300 ... 2024-03-26T08:20:07.191 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying IP 172.21.15.43 port 6789 ... 2024-03-26T08:20:07.191 INFO:teuthology.orchestra.run.smithi043.stdout:Base mon IP(s) is [172.21.15.43:3300, 172.21.15.43:6789], mon addrv is [v2:172.21.15.43:3300,v1:172.21.15.43:6789] 2024-03-26T08:20:07.200 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.43 metric 100 2024-03-26T08:20:07.200 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.43 metric 100 2024-03-26T08:20:07.208 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-26T08:20:07.208 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-03-26T08:20:07.208 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-03-26T08:20:07.216 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-26T08:20:07.216 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-26T08:20:07.216 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-26T08:20:07.216 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-26T08:20:07.216 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1358/64 scope link noprefixroute 2024-03-26T08:20:07.216 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-26T08:20:07.217 INFO:teuthology.orchestra.run.smithi043.stdout:Mon IP `172.21.15.43` is in CIDR network `172.21.0.0/20` 2024-03-26T08:20:07.217 INFO:teuthology.orchestra.run.smithi043.stdout:Mon IP `172.21.15.43` is in CIDR network `172.21.0.0/20` 2024-03-26T08:20:07.217 INFO:teuthology.orchestra.run.smithi043.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-26T08:20:07.218 INFO:teuthology.orchestra.run.smithi043.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-26T08:20:07.219 INFO:teuthology.orchestra.run.smithi043.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-26T08:20:07.852 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 2024-03-26T08:20:07.853 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-26T08:20:07.853 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Getting image source signatures 2024-03-26T08:20:07.853 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying blob sha256:4dd83c6ee8921fe17396d77c31e205985ae94c87d3f752465b5f04aaf9ae340a 2024-03-26T08:20:07.853 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-26T08:20:07.853 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying config sha256:c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 2024-03-26T08:20:07.853 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-26T08:20:09.196 INFO:teuthology.orchestra.run.smithi043.stdout:ceph: stdout ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-26T08:20:09.196 INFO:teuthology.orchestra.run.smithi043.stdout:Ceph version: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-26T08:20:09.196 INFO:teuthology.orchestra.run.smithi043.stdout:Extracting ceph user uid/gid from container image... 2024-03-26T08:20:18.386 INFO:teuthology.orchestra.run.smithi043.stdout:stat: stdout 167 167 2024-03-26T08:20:18.386 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial keys... 2024-03-26T08:20:19.674 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQBDhQJmqUCXBRAA2qcnzqvkE3D8l1jBKfLXjw== 2024-03-26T08:20:20.752 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQBEhQJmM5PaCBAAdY+aQTp0WJzBH1CaNP+gSg== 2024-03-26T08:20:21.952 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQBFhQJmaj0BEBAAH/WGdrZ9WyERCk8hmb1dBQ== 2024-03-26T08:20:21.955 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial monmap... 2024-03-26T08:20:24.225 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-26T08:20:24.246 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-26T08:20:24.246 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 88759420-eb49-11ee-95d0-87774f69a715 2024-03-26T08:20:24.246 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-26T08:20:24.246 INFO:teuthology.orchestra.run.smithi043.stdout:monmaptool for a [v2:172.21.15.43:3300,v1:172.21.15.43:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-26T08:20:24.247 INFO:teuthology.orchestra.run.smithi043.stdout:setting min_mon_release = quincy 2024-03-26T08:20:24.247 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: set fsid to 88759420-eb49-11ee-95d0-87774f69a715 2024-03-26T08:20:24.247 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-26T08:20:24.247 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:20:24.247 INFO:teuthology.orchestra.run.smithi043.stdout:Creating mon... 2024-03-26T08:20:25.417 INFO:teuthology.orchestra.run.smithi043.stdout:create mon.a on 2024-03-26T08:20:25.788 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-26T08:20:26.059 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-26T08:20:26.354 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-88759420-eb49-11ee-95d0-87774f69a715.target → /etc/systemd/system/ceph-88759420-eb49-11ee-95d0-87774f69a715.target. 2024-03-26T08:20:26.354 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-88759420-eb49-11ee-95d0-87774f69a715.target → /etc/systemd/system/ceph-88759420-eb49-11ee-95d0-87774f69a715.target. 2024-03-26T08:20:26.752 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-88759420-eb49-11ee-95d0-87774f69a715@mon.a 2024-03-26T08:20:26.752 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to reset failed state of unit ceph-88759420-eb49-11ee-95d0-87774f69a715@mon.a.service: Unit ceph-88759420-eb49-11ee-95d0-87774f69a715@mon.a.service not loaded. 2024-03-26T08:20:27.032 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-88759420-eb49-11ee-95d0-87774f69a715.target.wants/ceph-88759420-eb49-11ee-95d0-87774f69a715@mon.a.service → /etc/systemd/system/ceph-88759420-eb49-11ee-95d0-87774f69a715@.service. 2024-03-26T08:20:27.723 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:27 smithi043 podman[31541]: 2024-03-26 08:20:27.453732327 +0000 UTC m=+0.147119604 container create df1725d042cc144dffd26c3d5715bcc47308979f0c830b46ee63949628599d2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-a, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T08:20:27.723 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:27 smithi043 podman[31541]: 2024-03-26 08:20:27.725922496 +0000 UTC m=+0.419309851 container init df1725d042cc144dffd26c3d5715bcc47308979f0c830b46ee63949628599d2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-26T08:20:27.833 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-26T08:20:27.834 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-03-26T08:20:27.852 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-26T08:20:27.852 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-03-26T08:20:27.852 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-03-26T08:20:27.852 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to enable service . firewalld.service is not available 2024-03-26T08:20:27.853 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mon to start... 2024-03-26T08:20:27.853 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mon... 2024-03-26T08:20:28.006 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:27 smithi043 podman[31541]: 2024-03-26 08:20:27.737374666 +0000 UTC m=+0.430761943 container start df1725d042cc144dffd26c3d5715bcc47308979f0c830b46ee63949628599d2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-a, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0) 2024-03-26T08:20:28.006 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:27 smithi043 bash[31541]: df1725d042cc144dffd26c3d5715bcc47308979f0c830b46ee63949628599d2a 2024-03-26T08:20:28.006 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:27 smithi043 systemd[1]: Started Ceph mon.a for 88759420-eb49-11ee-95d0-87774f69a715. 2024-03-26T08:20:28.006 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:27 smithi043 ceph-mon[31560]: mkfs 88759420-eb49-11ee-95d0-87774f69a715 2024-03-26T08:20:28.006 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:27 smithi043 ceph-mon[31560]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-26T08:20:28.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:28 smithi043 ceph-mon[31560]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-26T08:20:28.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:28 smithi043 ceph-mon[31560]: monmap e1: 1 mons at {a=[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T08:20:28.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:28 smithi043 ceph-mon[31560]: fsmap 2024-03-26T08:20:28.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:28 smithi043 ceph-mon[31560]: osdmap e1: 0 total, 0 up, 0 in 2024-03-26T08:20:28.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:28 smithi043 ceph-mon[31560]: mgrmap e1: no daemons active 2024-03-26T08:20:28.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:28 smithi043 ceph-mon[31560]: from='client.? 172.21.15.43:0/1191209014' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-26T08:20:29.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout cluster: 2024-03-26T08:20:29.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout id: 88759420-eb49-11ee-95d0-87774f69a715 2024-03-26T08:20:29.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-26T08:20:29.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-03-26T08:20:29.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout services: 2024-03-26T08:20:29.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.776738s) 2024-03-26T08:20:29.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-26T08:20:29.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-26T08:20:29.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-03-26T08:20:29.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout data: 2024-03-26T08:20:29.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-26T08:20:29.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-26T08:20:29.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-26T08:20:29.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout pgs: 2024-03-26T08:20:29.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-03-26T08:20:29.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.496+0000 7fbd16e00700 1 Processor -- start 2024-03-26T08:20:29.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.496+0000 7fbd16e00700 1 -- start start 2024-03-26T08:20:29.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.497+0000 7fbd16e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd1007d870 0x7fbd1007dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:20:29.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.497+0000 7fbd16e00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1007e260 con 0x7fbd1007d870 2024-03-26T08:20:29.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.498+0000 7fbd0fe00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd1007d870 0x7fbd1007dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:20:29.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.498+0000 7fbd0fe00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd1007d870 0x7fbd1007dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43348/0 (socket says 172.21.15.43:43348) 2024-03-26T08:20:29.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.498+0000 7fbd0fe00700 1 -- 172.21.15.43:0/669046579 learned_addr learned my addr 172.21.15.43:0/669046579 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:20:29.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.499+0000 7fbd0fe00700 1 -- 172.21.15.43:0/669046579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd1007ea70 con 0x7fbd1007d870 2024-03-26T08:20:29.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.499+0000 7fbd0fe00700 1 --2- 172.21.15.43:0/669046579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd1007d870 0x7fbd1007dc90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbd000098a0 tx=0x7fbd00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559d0d091d12be04 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:20:29.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.499+0000 7fbd0f400700 1 -- 172.21.15.43:0/669046579 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd00004160 con 0x7fbd1007d870 2024-03-26T08:20:29.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.500+0000 7fbd0f400700 1 -- 172.21.15.43:0/669046579 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd00004300 con 0x7fbd1007d870 2024-03-26T08:20:29.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.500+0000 7fbd0f400700 1 -- 172.21.15.43:0/669046579 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd000036a0 con 0x7fbd1007d870 2024-03-26T08:20:29.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.500+0000 7fbd16e00700 1 -- 172.21.15.43:0/669046579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd1007d870 msgr2=0x7fbd1007dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:20:29.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.500+0000 7fbd16e00700 1 --2- 172.21.15.43:0/669046579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd1007d870 0x7fbd1007dc90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbd000098a0 tx=0x7fbd00009bb0 comp rx=0 tx=0).stop 2024-03-26T08:20:29.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.500+0000 7fbd16e00700 1 -- 172.21.15.43:0/669046579 shutdown_connections 2024-03-26T08:20:29.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.500+0000 7fbd16e00700 1 --2- 172.21.15.43:0/669046579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd1007d870 0x7fbd1007dc90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:20:29.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.500+0000 7fbd16e00700 1 -- 172.21.15.43:0/669046579 >> 172.21.15.43:0/669046579 conn(0x7fbd101054d0 msgr2=0x7fbd101078c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:20:29.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.501+0000 7fbd16e00700 1 -- 172.21.15.43:0/669046579 shutdown_connections 2024-03-26T08:20:29.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.501+0000 7fbd16e00700 1 -- 172.21.15.43:0/669046579 wait complete. 2024-03-26T08:20:29.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.502+0000 7fbd16e00700 1 Processor -- start 2024-03-26T08:20:29.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.502+0000 7fbd16e00700 1 -- start start 2024-03-26T08:20:29.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.502+0000 7fbd16e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd1007d870 0x7fbd101a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:20:29.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.502+0000 7fbd16e00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1007efa0 con 0x7fbd1007d870 2024-03-26T08:20:29.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.502+0000 7fbd0fe00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd1007d870 0x7fbd101a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:20:29.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.503+0000 7fbd0fe00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd1007d870 0x7fbd101a14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43360/0 (socket says 172.21.15.43:43360) 2024-03-26T08:20:29.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.503+0000 7fbd0fe00700 1 -- 172.21.15.43:0/1191209014 learned_addr learned my addr 172.21.15.43:0/1191209014 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:20:29.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.503+0000 7fbd0fe00700 1 -- 172.21.15.43:0/1191209014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd101a19f0 con 0x7fbd1007d870 2024-03-26T08:20:29.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.503+0000 7fbd0fe00700 1 --2- 172.21.15.43:0/1191209014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd1007d870 0x7fbd101a14b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbd00000c00 tx=0x7fbd00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:20:29.270 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.503+0000 7fbd0d600700 1 -- 172.21.15.43:0/1191209014 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd00003db0 con 0x7fbd1007d870 2024-03-26T08:20:29.270 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.504+0000 7fbd0d600700 1 -- 172.21.15.43:0/1191209014 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd00011630 con 0x7fbd1007d870 2024-03-26T08:20:29.270 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.504+0000 7fbd0d600700 1 -- 172.21.15.43:0/1191209014 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd00011950 con 0x7fbd1007d870 2024-03-26T08:20:29.270 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.504+0000 7fbd16e00700 1 -- 172.21.15.43:0/1191209014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd101a1c80 con 0x7fbd1007d870 2024-03-26T08:20:29.270 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.504+0000 7fbd16e00700 1 -- 172.21.15.43:0/1191209014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd101a4970 con 0x7fbd1007d870 2024-03-26T08:20:29.270 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.504+0000 7fbd0d600700 1 -- 172.21.15.43:0/1191209014 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fbd00011af0 con 0x7fbd1007d870 2024-03-26T08:20:29.270 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.504+0000 7fbd0d600700 1 -- 172.21.15.43:0/1191209014 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbd0001c790 con 0x7fbd1007d870 2024-03-26T08:20:29.270 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.505+0000 7fbd16e00700 1 -- 172.21.15.43:0/1191209014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd100637c0 con 0x7fbd1007d870 2024-03-26T08:20:29.270 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.508+0000 7fbd0d600700 1 -- 172.21.15.43:0/1191209014 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fbd00032020 con 0x7fbd1007d870 2024-03-26T08:20:29.270 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.591+0000 7fbd16e00700 1 -- 172.21.15.43:0/1191209014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fbd101a4e60 con 0x7fbd1007d870 2024-03-26T08:20:29.270 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.591+0000 7fbd0d600700 1 -- 172.21.15.43:0/1191209014 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fbd00004260 con 0x7fbd1007d870 2024-03-26T08:20:29.270 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.592+0000 7fbd16e00700 1 -- 172.21.15.43:0/1191209014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd1007d870 msgr2=0x7fbd101a14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:20:29.271 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.592+0000 7fbd16e00700 1 --2- 172.21.15.43:0/1191209014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd1007d870 0x7fbd101a14b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbd00000c00 tx=0x7fbd00004750 comp rx=0 tx=0).stop 2024-03-26T08:20:29.271 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.592+0000 7fbd16e00700 1 -- 172.21.15.43:0/1191209014 shutdown_connections 2024-03-26T08:20:29.271 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.592+0000 7fbd16e00700 1 --2- 172.21.15.43:0/1191209014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd1007d870 0x7fbd101a14b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:20:29.271 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.592+0000 7fbd16e00700 1 -- 172.21.15.43:0/1191209014 >> 172.21.15.43:0/1191209014 conn(0x7fbd101054d0 msgr2=0x7fbd101072d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:20:29.271 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.593+0000 7fbd16e00700 1 -- 172.21.15.43:0/1191209014 shutdown_connections 2024-03-26T08:20:29.271 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:28.593+0000 7fbd16e00700 1 -- 172.21.15.43:0/1191209014 wait complete. 2024-03-26T08:20:29.271 INFO:teuthology.orchestra.run.smithi043.stdout:mon is available 2024-03-26T08:20:29.271 INFO:teuthology.orchestra.run.smithi043.stdout:Assimilating anything we can from ceph.conf... 2024-03-26T08:20:30.155 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:30 smithi043 ceph-mon[31560]: from='client.? 172.21.15.43:0/1161157802' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-26T08:20:30.155 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:30 smithi043 ceph-mon[31560]: from='client.? 172.21.15.43:0/1161157802' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-26T08:20:30.658 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-03-26T08:20:30.658 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [global] 2024-03-26T08:20:30.658 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout fsid = 88759420-eb49-11ee-95d0-87774f69a715 2024-03-26T08:20:30.659 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.43:3300,v1:172.21.15.43:6789] 2024-03-26T08:20:30.659 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-26T08:20:30.659 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-26T08:20:30.659 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-26T08:20:30.659 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-26T08:20:30.659 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-03-26T08:20:30.659 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-26T08:20:30.659 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-26T08:20:30.660 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-03-26T08:20:30.660 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [osd] 2024-03-26T08:20:30.660 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-26T08:20:30.660 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-26T08:20:30.660 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-26T08:20:30.660 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-26T08:20:30.660 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-26T08:20:30.660 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.860+0000 7f7cfa600700 1 Processor -- start 2024-03-26T08:20:30.660 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.860+0000 7f7cfa600700 1 -- start start 2024-03-26T08:20:30.660 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.861+0000 7f7cfa600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf410e0c0 0x7f7cf410e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:20:30.661 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.861+0000 7f7cfa600700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf410eab0 con 0x7f7cf410e0c0 2024-03-26T08:20:30.661 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.861+0000 7f7cf9200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf410e0c0 0x7f7cf410e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:20:30.661 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.861+0000 7f7cf9200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf410e0c0 0x7f7cf410e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43366/0 (socket says 172.21.15.43:43366) 2024-03-26T08:20:30.661 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.861+0000 7f7cf9200700 1 -- 172.21.15.43:0/1085883764 learned_addr learned my addr 172.21.15.43:0/1085883764 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:20:30.661 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.862+0000 7f7cf9200700 1 -- 172.21.15.43:0/1085883764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf410ee80 con 0x7f7cf410e0c0 2024-03-26T08:20:30.661 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.862+0000 7f7cf9200700 1 --2- 172.21.15.43:0/1085883764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf410e0c0 0x7f7cf410e4e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc0098a0 tx=0x7f7cdc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae05da674c89908 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:20:30.661 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.862+0000 7f7cf3e00700 1 -- 172.21.15.43:0/1085883764 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cdc004160 con 0x7f7cf410e0c0 2024-03-26T08:20:30.661 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.863+0000 7f7cf3e00700 1 -- 172.21.15.43:0/1085883764 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cdc004300 con 0x7f7cf410e0c0 2024-03-26T08:20:30.661 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.863+0000 7f7cf3e00700 1 -- 172.21.15.43:0/1085883764 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cdc0036a0 con 0x7f7cf410e0c0 2024-03-26T08:20:30.661 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.863+0000 7f7cfa600700 1 -- 172.21.15.43:0/1085883764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf410e0c0 msgr2=0x7f7cf410e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:20:30.661 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.863+0000 7f7cfa600700 1 --2- 172.21.15.43:0/1085883764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf410e0c0 0x7f7cf410e4e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc0098a0 tx=0x7f7cdc009bb0 comp rx=0 tx=0).stop 2024-03-26T08:20:30.661 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.863+0000 7f7cfa600700 1 -- 172.21.15.43:0/1085883764 shutdown_connections 2024-03-26T08:20:30.662 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.863+0000 7f7cfa600700 1 --2- 172.21.15.43:0/1085883764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf410e0c0 0x7f7cf410e4e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:20:30.662 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.863+0000 7f7cfa600700 1 -- 172.21.15.43:0/1085883764 >> 172.21.15.43:0/1085883764 conn(0x7f7cf407cde0 msgr2=0x7f7cf407f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:20:30.662 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.864+0000 7f7cfa600700 1 -- 172.21.15.43:0/1085883764 shutdown_connections 2024-03-26T08:20:30.662 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.864+0000 7f7cfa600700 1 -- 172.21.15.43:0/1085883764 wait complete. 2024-03-26T08:20:30.662 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.865+0000 7f7cfa600700 1 Processor -- start 2024-03-26T08:20:30.662 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.865+0000 7f7cfa600700 1 -- start start 2024-03-26T08:20:30.662 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.865+0000 7f7cfa600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf410e0c0 0x7f7cf41a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:20:30.662 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.865+0000 7f7cfa600700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf410f3b0 con 0x7f7cf410e0c0 2024-03-26T08:20:30.662 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.865+0000 7f7cf9200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf410e0c0 0x7f7cf41a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:20:30.662 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.866+0000 7f7cf9200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf410e0c0 0x7f7cf41a1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43382/0 (socket says 172.21.15.43:43382) 2024-03-26T08:20:30.663 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.866+0000 7f7cf9200700 1 -- 172.21.15.43:0/1161157802 learned_addr learned my addr 172.21.15.43:0/1161157802 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:20:30.663 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.866+0000 7f7cf9200700 1 -- 172.21.15.43:0/1161157802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf41a18c0 con 0x7f7cf410e0c0 2024-03-26T08:20:30.663 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.866+0000 7f7cf9200700 1 --2- 172.21.15.43:0/1161157802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf410e0c0 0x7f7cf41a1380 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc000c00 tx=0x7f7cdc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:20:30.663 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.867+0000 7f7cf2000700 1 -- 172.21.15.43:0/1161157802 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cdc003db0 con 0x7f7cf410e0c0 2024-03-26T08:20:30.663 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.867+0000 7f7cf2000700 1 -- 172.21.15.43:0/1161157802 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cdc011630 con 0x7f7cf410e0c0 2024-03-26T08:20:30.663 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.867+0000 7f7cf2000700 1 -- 172.21.15.43:0/1161157802 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cdc011950 con 0x7f7cf410e0c0 2024-03-26T08:20:30.663 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.867+0000 7f7cfa600700 1 -- 172.21.15.43:0/1161157802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf41a1b50 con 0x7f7cf410e0c0 2024-03-26T08:20:30.663 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.867+0000 7f7cfa600700 1 -- 172.21.15.43:0/1161157802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf41a4840 con 0x7f7cf410e0c0 2024-03-26T08:20:30.663 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.867+0000 7f7cf2000700 1 -- 172.21.15.43:0/1161157802 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7cdc011af0 con 0x7f7cf410e0c0 2024-03-26T08:20:30.664 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.867+0000 7f7cf2000700 1 -- 172.21.15.43:0/1161157802 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7cdc01c450 con 0x7f7cf410e0c0 2024-03-26T08:20:30.664 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.868+0000 7f7cfa600700 1 -- 172.21.15.43:0/1161157802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf40637c0 con 0x7f7cf410e0c0 2024-03-26T08:20:30.664 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.871+0000 7f7cf2000700 1 -- 172.21.15.43:0/1161157802 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7cdc032020 con 0x7f7cf410e0c0 2024-03-26T08:20:30.664 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.949+0000 7f7cfa600700 1 -- 172.21.15.43:0/1161157802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f7cf402dc40 con 0x7f7cf410e0c0 2024-03-26T08:20:30.664 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.955+0000 7f7cf2000700 1 -- 172.21.15.43:0/1161157802 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f7cdc004260 con 0x7f7cf410e0c0 2024-03-26T08:20:30.664 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.956+0000 7f7cf2000700 1 -- 172.21.15.43:0/1161157802 <== mon.0 v2:172.21.15.43:3300/0 8 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f7cdc01c6a0 con 0x7f7cf410e0c0 2024-03-26T08:20:30.664 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.957+0000 7f7cfa600700 1 -- 172.21.15.43:0/1161157802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf410e0c0 msgr2=0x7f7cf41a1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:20:30.664 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.957+0000 7f7cfa600700 1 --2- 172.21.15.43:0/1161157802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf410e0c0 0x7f7cf41a1380 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc000c00 tx=0x7f7cdc004750 comp rx=0 tx=0).stop 2024-03-26T08:20:30.664 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.957+0000 7f7cfa600700 1 -- 172.21.15.43:0/1161157802 shutdown_connections 2024-03-26T08:20:30.664 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.957+0000 7f7cfa600700 1 --2- 172.21.15.43:0/1161157802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf410e0c0 0x7f7cf41a1380 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:20:30.664 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.957+0000 7f7cfa600700 1 -- 172.21.15.43:0/1161157802 >> 172.21.15.43:0/1161157802 conn(0x7f7cf407cde0 msgr2=0x7f7cf4197fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:20:30.664 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.957+0000 7f7cfa600700 1 -- 172.21.15.43:0/1161157802 shutdown_connections 2024-03-26T08:20:30.664 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:29.957+0000 7f7cfa600700 1 -- 172.21.15.43:0/1161157802 wait complete. 2024-03-26T08:20:30.665 INFO:teuthology.orchestra.run.smithi043.stdout:Generating new minimal ceph.conf... 2024-03-26T08:20:31.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:31 smithi043 ceph-mon[31560]: from='client.? 172.21.15.43:0/2675716679' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:20:32.006 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.263+0000 7f7c17000700 1 Processor -- start 2024-03-26T08:20:32.006 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.264+0000 7f7c17000700 1 -- start start 2024-03-26T08:20:32.006 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.264+0000 7f7c17000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c10109ae0 0x7f7c10109f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:20:32.006 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.264+0000 7f7c17000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1010a4d0 con 0x7f7c10109ae0 2024-03-26T08:20:32.006 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.265+0000 7f7c0fe00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c10109ae0 0x7f7c10109f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:20:32.006 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.265+0000 7f7c0fe00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c10109ae0 0x7f7c10109f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43390/0 (socket says 172.21.15.43:43390) 2024-03-26T08:20:32.006 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.265+0000 7f7c0fe00700 1 -- 172.21.15.43:0/2524813038 learned_addr learned my addr 172.21.15.43:0/2524813038 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:20:32.006 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.265+0000 7f7c0fe00700 1 -- 172.21.15.43:0/2524813038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1010ac40 con 0x7f7c10109ae0 2024-03-26T08:20:32.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.266+0000 7f7c0fe00700 1 --2- 172.21.15.43:0/2524813038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c10109ae0 0x7f7c10109f00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7c000098a0 tx=0x7f7c00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e5ddca7ef384d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:20:32.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.266+0000 7f7c0ea00700 1 -- 172.21.15.43:0/2524813038 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c00004160 con 0x7f7c10109ae0 2024-03-26T08:20:32.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.266+0000 7f7c0ea00700 1 -- 172.21.15.43:0/2524813038 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f7c00004300 con 0x7f7c10109ae0 2024-03-26T08:20:32.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.266+0000 7f7c0ea00700 1 -- 172.21.15.43:0/2524813038 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c00011a80 con 0x7f7c10109ae0 2024-03-26T08:20:32.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.267+0000 7f7c17000700 1 -- 172.21.15.43:0/2524813038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c10109ae0 msgr2=0x7f7c10109f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:20:32.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.267+0000 7f7c17000700 1 --2- 172.21.15.43:0/2524813038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c10109ae0 0x7f7c10109f00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7c000098a0 tx=0x7f7c00009bb0 comp rx=0 tx=0).stop 2024-03-26T08:20:32.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.267+0000 7f7c17000700 1 -- 172.21.15.43:0/2524813038 shutdown_connections 2024-03-26T08:20:32.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.267+0000 7f7c17000700 1 --2- 172.21.15.43:0/2524813038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c10109ae0 0x7f7c10109f00 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:20:32.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.267+0000 7f7c17000700 1 -- 172.21.15.43:0/2524813038 >> 172.21.15.43:0/2524813038 conn(0x7f7c1007ced0 msgr2=0x7f7c101074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:20:32.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.267+0000 7f7c17000700 1 -- 172.21.15.43:0/2524813038 shutdown_connections 2024-03-26T08:20:32.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.267+0000 7f7c17000700 1 -- 172.21.15.43:0/2524813038 wait complete. 2024-03-26T08:20:32.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.268+0000 7f7c17000700 1 Processor -- start 2024-03-26T08:20:32.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.269+0000 7f7c17000700 1 -- start start 2024-03-26T08:20:32.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.269+0000 7f7c17000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c10109ae0 0x7f7c1019d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:20:32.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.269+0000 7f7c17000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1010dff0 con 0x7f7c10109ae0 2024-03-26T08:20:32.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.269+0000 7f7c0fe00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c10109ae0 0x7f7c1019d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:20:32.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.269+0000 7f7c0fe00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c10109ae0 0x7f7c1019d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43404/0 (socket says 172.21.15.43:43404) 2024-03-26T08:20:32.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.270+0000 7f7c0fe00700 1 -- 172.21.15.43:0/2675716679 learned_addr learned my addr 172.21.15.43:0/2675716679 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:20:32.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.270+0000 7f7c0fe00700 1 -- 172.21.15.43:0/2675716679 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1019df00 con 0x7f7c10109ae0 2024-03-26T08:20:32.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.270+0000 7f7c0fe00700 1 --2- 172.21.15.43:0/2675716679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c10109ae0 0x7f7c1019d9c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7c00000c00 tx=0x7f7c00004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:20:32.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.271+0000 7f7c0cc00700 1 -- 172.21.15.43:0/2675716679 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c000039c0 con 0x7f7c10109ae0 2024-03-26T08:20:32.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.271+0000 7f7c0cc00700 1 -- 172.21.15.43:0/2675716679 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f7c00003b60 con 0x7f7c10109ae0 2024-03-26T08:20:32.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.271+0000 7f7c17000700 1 -- 172.21.15.43:0/2675716679 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c1019e190 con 0x7f7c10109ae0 2024-03-26T08:20:32.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.271+0000 7f7c0cc00700 1 -- 172.21.15.43:0/2675716679 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c0001a6f0 con 0x7f7c10109ae0 2024-03-26T08:20:32.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.271+0000 7f7c17000700 1 -- 172.21.15.43:0/2675716679 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c101a0e80 con 0x7f7c10109ae0 2024-03-26T08:20:32.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.271+0000 7f7c0cc00700 1 -- 172.21.15.43:0/2675716679 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7c0002d3f0 con 0x7f7c10109ae0 2024-03-26T08:20:32.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.271+0000 7f7c0cc00700 1 -- 172.21.15.43:0/2675716679 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7c00019700 con 0x7f7c10109ae0 2024-03-26T08:20:32.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.272+0000 7f7c17000700 1 -- 172.21.15.43:0/2675716679 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c100637c0 con 0x7f7c10109ae0 2024-03-26T08:20:32.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.275+0000 7f7c0cc00700 1 -- 172.21.15.43:0/2675716679 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7c00024a40 con 0x7f7c10109ae0 2024-03-26T08:20:32.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.354+0000 7f7c17000700 1 -- 172.21.15.43:0/2675716679 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f7c101a13d0 con 0x7f7c10109ae0 2024-03-26T08:20:32.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.354+0000 7f7c0cc00700 1 -- 172.21.15.43:0/2675716679 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f7c00032030 con 0x7f7c10109ae0 2024-03-26T08:20:32.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.356+0000 7f7c17000700 1 -- 172.21.15.43:0/2675716679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c10109ae0 msgr2=0x7f7c1019d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:20:32.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.356+0000 7f7c17000700 1 --2- 172.21.15.43:0/2675716679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c10109ae0 0x7f7c1019d9c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7c00000c00 tx=0x7f7c00004290 comp rx=0 tx=0).stop 2024-03-26T08:20:32.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.356+0000 7f7c17000700 1 -- 172.21.15.43:0/2675716679 shutdown_connections 2024-03-26T08:20:32.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.356+0000 7f7c17000700 1 --2- 172.21.15.43:0/2675716679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c10109ae0 0x7f7c1019d9c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:20:32.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.356+0000 7f7c17000700 1 -- 172.21.15.43:0/2675716679 >> 172.21.15.43:0/2675716679 conn(0x7f7c1007ced0 msgr2=0x7f7c10105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:20:32.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.356+0000 7f7c17000700 1 -- 172.21.15.43:0/2675716679 shutdown_connections 2024-03-26T08:20:32.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:31.356+0000 7f7c17000700 1 -- 172.21.15.43:0/2675716679 wait complete. 2024-03-26T08:20:32.009 INFO:teuthology.orchestra.run.smithi043.stdout:Restarting the monitor... 2024-03-26T08:20:32.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:32 smithi043 systemd[1]: Stopping Ceph mon.a for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:20:32.821 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:32 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-a[31556]: 2024-03-26T08:20:32.502+0000 7fa03a600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:20:32.821 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:32 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-a[31556]: 2024-03-26T08:20:32.502+0000 7fa03a600700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-26T08:20:32.821 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:32 smithi043 podman[31802]: 2024-03-26 08:20:32.532619217 +0000 UTC m=+0.176913492 container died df1725d042cc144dffd26c3d5715bcc47308979f0c830b46ee63949628599d2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-a, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-26T08:20:33.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:32 smithi043 podman[31802]: 2024-03-26 08:20:32.935334352 +0000 UTC m=+0.579628654 container cleanup df1725d042cc144dffd26c3d5715bcc47308979f0c830b46ee63949628599d2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-a, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-26T08:20:33.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:32 smithi043 bash[31802]: ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-a 2024-03-26T08:20:33.504 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:33 smithi043 podman[31816]: 2024-03-26 08:20:33.219967336 +0000 UTC m=+0.670653092 container remove df1725d042cc144dffd26c3d5715bcc47308979f0c830b46ee63949628599d2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-03-26T08:20:33.758 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:33 smithi043 systemd[1]: ceph-88759420-eb49-11ee-95d0-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-26T08:20:33.758 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:33 smithi043 systemd[1]: Stopped Ceph mon.a for 88759420-eb49-11ee-95d0-87774f69a715. 2024-03-26T08:20:33.758 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:33 smithi043 systemd[1]: ceph-88759420-eb49-11ee-95d0-87774f69a715@mon.a.service: Consumed 1.934s CPU time. 2024-03-26T08:20:34.122 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:33 smithi043 systemd[1]: Starting Ceph mon.a for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:20:34.377 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 podman[31910]: 2024-03-26 08:20:34.03555703 +0000 UTC m=+0.040326992 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T08:20:34.377 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 podman[31910]: 2024-03-26 08:20:34.13742962 +0000 UTC m=+0.142199582 container create b43fc9922941a01d411d5cd0e4a643048d766ce35443039ac012e1128202736f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-a, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-03-26T08:20:34.377 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 podman[31910]: 2024-03-26 08:20:34.381789224 +0000 UTC m=+0.386559179 container init b43fc9922941a01d411d5cd0e4a643048d766ce35443039ac012e1128202736f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-26T08:20:34.462 INFO:teuthology.orchestra.run.smithi043.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-26T08:20:34.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 podman[31910]: 2024-03-26 08:20:34.391549839 +0000 UTC m=+0.396319769 container start b43fc9922941a01d411d5cd0e4a643048d766ce35443039ac012e1128202736f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-26T08:20:34.629 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: set uid:gid to 167:167 (ceph:ceph) 2024-03-26T08:20:34.629 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev), process ceph-mon, pid 2 2024-03-26T08:20:34.629 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: pidfile_write: ignore empty --pid-file 2024-03-26T08:20:34.629 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: load: jerasure load: lrc 2024-03-26T08:20:34.629 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: RocksDB version: 7.9.2 2024-03-26T08:20:34.630 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Git sha 0 2024-03-26T08:20:34.630 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Compile date 2024-03-22 04:58:55 2024-03-26T08:20:34.630 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: DB SUMMARY 2024-03-26T08:20:34.630 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: DB Session ID: 4R5BIVBF1RLKYD1K8OXO 2024-03-26T08:20:34.630 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: CURRENT file: CURRENT 2024-03-26T08:20:34.630 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: IDENTITY file: IDENTITY 2024-03-26T08:20:34.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-26T08:20:34.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-26T08:20:34.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 99082 ; 2024-03-26T08:20:34.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.error_if_exists: 0 2024-03-26T08:20:34.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.create_if_missing: 0 2024-03-26T08:20:34.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.paranoid_checks: 1 2024-03-26T08:20:34.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-26T08:20:34.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-26T08:20:34.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-26T08:20:34.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.env: 0x55e018ae0b20 2024-03-26T08:20:34.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.fs: PosixFileSystem 2024-03-26T08:20:34.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.info_log: 0x55e01996f960 2024-03-26T08:20:34.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.max_file_opening_threads: 16 2024-03-26T08:20:34.632 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.statistics: (nil) 2024-03-26T08:20:34.632 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.use_fsync: 0 2024-03-26T08:20:34.632 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.max_log_file_size: 0 2024-03-26T08:20:34.632 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-26T08:20:34.632 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-26T08:20:34.632 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.keep_log_file_num: 1000 2024-03-26T08:20:34.632 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.recycle_log_file_num: 0 2024-03-26T08:20:34.632 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.allow_fallocate: 1 2024-03-26T08:20:34.633 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.allow_mmap_reads: 0 2024-03-26T08:20:34.633 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.allow_mmap_writes: 0 2024-03-26T08:20:34.633 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.use_direct_reads: 0 2024-03-26T08:20:34.633 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-26T08:20:34.633 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.create_missing_column_families: 0 2024-03-26T08:20:34.634 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.db_log_dir: 2024-03-26T08:20:34.634 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.wal_dir: 2024-03-26T08:20:34.634 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-26T08:20:34.634 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-26T08:20:34.634 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-26T08:20:34.634 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-26T08:20:34.634 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-26T08:20:34.634 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-26T08:20:34.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.advise_random_on_open: 1 2024-03-26T08:20:34.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.db_write_buffer_size: 0 2024-03-26T08:20:34.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.write_buffer_manager: 0x55e0199421e0 2024-03-26T08:20:34.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-26T08:20:34.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-26T08:20:34.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-26T08:20:34.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.rate_limiter: (nil) 2024-03-26T08:20:34.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-26T08:20:34.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.wal_recovery_mode: 2 2024-03-26T08:20:34.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.enable_thread_tracking: 0 2024-03-26T08:20:34.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.enable_pipelined_write: 0 2024-03-26T08:20:34.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.unordered_write: 0 2024-03-26T08:20:34.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-26T08:20:34.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-26T08:20:34.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-26T08:20:34.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-26T08:20:34.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.row_cache: None 2024-03-26T08:20:34.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.wal_filter: None 2024-03-26T08:20:34.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-26T08:20:34.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.allow_ingest_behind: 0 2024-03-26T08:20:34.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.two_write_queues: 0 2024-03-26T08:20:34.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.manual_wal_flush: 0 2024-03-26T08:20:34.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.wal_compression: 0 2024-03-26T08:20:34.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.atomic_flush: 0 2024-03-26T08:20:34.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-26T08:20:34.637 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-26T08:20:34.637 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-26T08:20:34.637 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.log_readahead_size: 0 2024-03-26T08:20:34.637 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-26T08:20:34.637 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.best_efforts_recovery: 0 2024-03-26T08:20:34.637 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-26T08:20:34.637 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-26T08:20:34.637 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.allow_data_in_errors: 0 2024-03-26T08:20:34.638 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.db_host_id: __hostname__ 2024-03-26T08:20:34.638 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-26T08:20:34.638 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.max_background_jobs: 2 2024-03-26T08:20:34.638 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.max_background_compactions: -1 2024-03-26T08:20:34.638 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.max_subcompactions: 1 2024-03-26T08:20:34.638 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-26T08:20:34.638 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-26T08:20:34.638 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-26T08:20:34.638 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.max_total_wal_size: 0 2024-03-26T08:20:34.638 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-26T08:20:34.638 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-26T08:20:34.638 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-26T08:20:34.638 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-26T08:20:34.639 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.max_open_files: -1 2024-03-26T08:20:34.639 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.bytes_per_sync: 0 2024-03-26T08:20:34.639 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-26T08:20:34.639 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-26T08:20:34.639 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.compaction_readahead_size: 0 2024-03-26T08:20:34.639 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.max_background_flushes: -1 2024-03-26T08:20:34.639 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Compression algorithms supported: 2024-03-26T08:20:34.639 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-26T08:20:34.639 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: kZSTD supported: 0 2024-03-26T08:20:34.639 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: kXpressCompression supported: 0 2024-03-26T08:20:34.639 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: kLZ4HCCompression supported: 1 2024-03-26T08:20:34.639 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: kZlibCompression supported: 1 2024-03-26T08:20:34.639 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: kSnappyCompression supported: 1 2024-03-26T08:20:34.639 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: kLZ4Compression supported: 1 2024-03-26T08:20:34.639 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: kBZip2Compression supported: 0 2024-03-26T08:20:34.640 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-26T08:20:34.640 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-26T08:20:34.640 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-26T08:20:34.640 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-26T08:20:34.640 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-26T08:20:34.640 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.merge_operator: 2024-03-26T08:20:34.640 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.compaction_filter: None 2024-03-26T08:20:34.640 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.compaction_filter_factory: None 2024-03-26T08:20:34.640 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.sst_partitioner_factory: None 2024-03-26T08:20:34.640 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-26T08:20:34.640 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-26T08:20:34.640 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e01996fa60) 2024-03-26T08:20:34.641 INFO:journalctl@ceph.mon.a.smithi043.stdout: cache_index_and_filter_blocks: 1 2024-03-26T08:20:34.641 INFO:journalctl@ceph.mon.a.smithi043.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-26T08:20:34.641 INFO:journalctl@ceph.mon.a.smithi043.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-26T08:20:34.641 INFO:journalctl@ceph.mon.a.smithi043.stdout: pin_top_level_index_and_filter: 1 2024-03-26T08:20:34.641 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_type: 0 2024-03-26T08:20:34.641 INFO:journalctl@ceph.mon.a.smithi043.stdout: data_block_index_type: 0 2024-03-26T08:20:34.641 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_shortening: 1 2024-03-26T08:20:34.641 INFO:journalctl@ceph.mon.a.smithi043.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-26T08:20:34.641 INFO:journalctl@ceph.mon.a.smithi043.stdout: checksum: 4 2024-03-26T08:20:34.641 INFO:journalctl@ceph.mon.a.smithi043.stdout: no_block_cache: 0 2024-03-26T08:20:34.641 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache: 0x55e019975090 2024-03-26T08:20:34.641 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_name: BinnedLRUCache 2024-03-26T08:20:34.641 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_options: 2024-03-26T08:20:34.641 INFO:journalctl@ceph.mon.a.smithi043.stdout: capacity : 536870912 2024-03-26T08:20:34.642 INFO:journalctl@ceph.mon.a.smithi043.stdout: num_shard_bits : 4 2024-03-26T08:20:34.642 INFO:journalctl@ceph.mon.a.smithi043.stdout: strict_capacity_limit : 0 2024-03-26T08:20:34.642 INFO:journalctl@ceph.mon.a.smithi043.stdout: high_pri_pool_ratio: 0.000 2024-03-26T08:20:34.642 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_compressed: (nil) 2024-03-26T08:20:34.642 INFO:journalctl@ceph.mon.a.smithi043.stdout: persistent_cache: (nil) 2024-03-26T08:20:34.642 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_size: 4096 2024-03-26T08:20:34.642 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_size_deviation: 10 2024-03-26T08:20:34.642 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_restart_interval: 16 2024-03-26T08:20:34.642 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_block_restart_interval: 1 2024-03-26T08:20:34.642 INFO:journalctl@ceph.mon.a.smithi043.stdout: metadata_block_size: 4096 2024-03-26T08:20:34.642 INFO:journalctl@ceph.mon.a.smithi043.stdout: partition_filters: 0 2024-03-26T08:20:34.642 INFO:journalctl@ceph.mon.a.smithi043.stdout: use_delta_encoding: 1 2024-03-26T08:20:34.642 INFO:journalctl@ceph.mon.a.smithi043.stdout: filter_policy: bloomfilter 2024-03-26T08:20:34.643 INFO:journalctl@ceph.mon.a.smithi043.stdout: whole_key_filtering: 1 2024-03-26T08:20:34.643 INFO:journalctl@ceph.mon.a.smithi043.stdout: verify_compression: 0 2024-03-26T08:20:34.643 INFO:journalctl@ceph.mon.a.smithi043.stdout: read_amp_bytes_per_bit: 0 2024-03-26T08:20:34.643 INFO:journalctl@ceph.mon.a.smithi043.stdout: format_version: 5 2024-03-26T08:20:34.643 INFO:journalctl@ceph.mon.a.smithi043.stdout: enable_index_compression: 1 2024-03-26T08:20:34.643 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_align: 0 2024-03-26T08:20:34.643 INFO:journalctl@ceph.mon.a.smithi043.stdout: max_auto_readahead_size: 262144 2024-03-26T08:20:34.643 INFO:journalctl@ceph.mon.a.smithi043.stdout: prepopulate_block_cache: 0 2024-03-26T08:20:34.643 INFO:journalctl@ceph.mon.a.smithi043.stdout: initial_auto_readahead_size: 8192 2024-03-26T08:20:34.643 INFO:journalctl@ceph.mon.a.smithi043.stdout: num_file_reads_for_auto_readahead: 2 2024-03-26T08:20:34.643 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.write_buffer_size: 33554432 2024-03-26T08:20:34.643 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.max_write_buffer_number: 2 2024-03-26T08:20:34.643 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.compression: NoCompression 2024-03-26T08:20:34.643 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.bottommost_compression: Disabled 2024-03-26T08:20:34.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.prefix_extractor: nullptr 2024-03-26T08:20:34.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-26T08:20:34.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.num_levels: 7 2024-03-26T08:20:34.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-26T08:20:34.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-26T08:20:34.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-26T08:20:34.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-26T08:20:34.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-26T08:20:34.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-26T08:20:34.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-26T08:20:34.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-26T08:20:34.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-26T08:20:34.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-26T08:20:34.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-26T08:20:34.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-26T08:20:34.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-26T08:20:34.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.compression_opts.level: 32767 2024-03-26T08:20:34.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.compression_opts.strategy: 0 2024-03-26T08:20:34.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-26T08:20:34.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-26T08:20:34.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-26T08:20:34.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-26T08:20:34.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.compression_opts.enabled: false 2024-03-26T08:20:34.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-26T08:20:34.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-26T08:20:34.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-26T08:20:34.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-26T08:20:34.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.target_file_size_base: 67108864 2024-03-26T08:20:34.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-26T08:20:34.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-26T08:20:34.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-26T08:20:34.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-26T08:20:34.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-26T08:20:34.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-26T08:20:34.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-26T08:20:34.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-26T08:20:34.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-26T08:20:34.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-26T08:20:34.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 bash[31910]: b43fc9922941a01d411d5cd0e4a643048d766ce35443039ac012e1128202736f 2024-03-26T08:20:34.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-26T08:20:34.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-26T08:20:34.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-26T08:20:34.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-26T08:20:34.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.arena_block_size: 1048576 2024-03-26T08:20:34.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-26T08:20:34.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-26T08:20:34.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.disable_auto_compactions: 0 2024-03-26T08:20:34.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-26T08:20:34.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-26T08:20:34.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-26T08:20:34.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-26T08:20:34.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-26T08:20:34.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-26T08:20:34.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-26T08:20:34.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-26T08:20:34.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-26T08:20:34.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-26T08:20:34.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.table_properties_collectors: 2024-03-26T08:20:34.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.inplace_update_support: 0 2024-03-26T08:20:34.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-26T08:20:34.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-26T08:20:34.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-26T08:20:34.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-26T08:20:34.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.bloom_locality: 0 2024-03-26T08:20:34.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.max_successive_merges: 0 2024-03-26T08:20:34.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-26T08:20:34.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.paranoid_file_checks: 0 2024-03-26T08:20:34.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.force_consistency_checks: 1 2024-03-26T08:20:34.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.report_bg_io_stats: 0 2024-03-26T08:20:34.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.ttl: 2592000 2024-03-26T08:20:34.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-26T08:20:34.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-26T08:20:34.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-26T08:20:34.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.enable_blob_files: false 2024-03-26T08:20:34.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.min_blob_size: 0 2024-03-26T08:20:34.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.blob_file_size: 268435456 2024-03-26T08:20:34.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-26T08:20:34.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-26T08:20:34.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-26T08:20:34.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-26T08:20:34.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-26T08:20:34.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.blob_file_starting_level: 0 2024-03-26T08:20:34.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-26T08:20:34.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-26T08:20:34.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-26T08:20:34.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2ef8ca09-aea7-45ef-82ea-4b4bafe54d27 2024-03-26T08:20:34.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711441234450313, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-26T08:20:34.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-26T08:20:34.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711441234453676, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94404, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 312, "table_properties": {"data_size": 92387, "index_size": 299, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13525, "raw_average_key_size": 49, "raw_value_size": 84998, "raw_average_value_size": 313, "num_data_blocks": 11, "num_entries": 271, "num_filter_entries": 271, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711441234, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2ef8ca09-aea7-45ef-82ea-4b4bafe54d27", "db_session_id": "4R5BIVBF1RLKYD1K8OXO", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-26T08:20:34.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711441234453902, "job": 1, "event": "recovery_finished"} 2024-03-26T08:20:34.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-26T08:20:34.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-26T08:20:34.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e019a68000 2024-03-26T08:20:34.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: DB pointer 0x55e019a52000 2024-03-26T08:20:34.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-26T08:20:34.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-26T08:20:34.653 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** DB Stats ** 2024-03-26T08:20:34.653 INFO:journalctl@ceph.mon.a.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T08:20:34.653 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-26T08:20:34.653 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T08:20:34.653 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T08:20:34.653 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-26T08:20:34.653 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T08:20:34.653 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T08:20:34.653 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-03-26T08:20:34.653 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** Compaction Stats [default] ** 2024-03-26T08:20:34.653 INFO:journalctl@ceph.mon.a.smithi043.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T08:20:34.653 INFO:journalctl@ceph.mon.a.smithi043.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-26T08:20:34.653 INFO:journalctl@ceph.mon.a.smithi043.stdout: L0 2/0 94.03 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 39.8 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-26T08:20:34.654 INFO:journalctl@ceph.mon.a.smithi043.stdout: Sum 2/0 94.03 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 39.8 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-26T08:20:34.654 INFO:journalctl@ceph.mon.a.smithi043.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 39.8 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-26T08:20:34.654 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-03-26T08:20:34.654 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** Compaction Stats [default] ** 2024-03-26T08:20:34.654 INFO:journalctl@ceph.mon.a.smithi043.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T08:20:34.654 INFO:journalctl@ceph.mon.a.smithi043.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-26T08:20:34.654 INFO:journalctl@ceph.mon.a.smithi043.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 39.8 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-26T08:20:34.654 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-03-26T08:20:34.654 INFO:journalctl@ceph.mon.a.smithi043.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-26T08:20:34.654 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-03-26T08:20:34.654 INFO:journalctl@ceph.mon.a.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T08:20:34.654 INFO:journalctl@ceph.mon.a.smithi043.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-26T08:20:34.654 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-26T08:20:34.655 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-26T08:20:34.655 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-26T08:20:34.655 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-26T08:20:34.655 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative compaction: 0.00 GB write, 8.90 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T08:20:34.655 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval compaction: 0.00 GB write, 8.90 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T08:20:34.655 INFO:journalctl@ceph.mon.a.smithi043.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-26T08:20:34.655 INFO:journalctl@ceph.mon.a.smithi043.stdout: Block cache BinnedLRUCache@0x55e019975090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-03-26T08:20:34.656 INFO:journalctl@ceph.mon.a.smithi043.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-26T08:20:34.656 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-03-26T08:20:34.656 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-26T08:20:34.656 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: starting mon.a rank 0 at public addrs [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] at bind addrs [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 88759420-eb49-11ee-95d0-87774f69a715 2024-03-26T08:20:34.656 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: mon.a@-1(???) e1 preinit fsid 88759420-eb49-11ee-95d0-87774f69a715 2024-03-26T08:20:34.656 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: mon.a@-1(???).mds e1 new map 2024-03-26T08:20:34.656 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: mon.a@-1(???).mds e1 print_map 2024-03-26T08:20:34.657 INFO:journalctl@ceph.mon.a.smithi043.stdout: e1 2024-03-26T08:20:34.657 INFO:journalctl@ceph.mon.a.smithi043.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-26T08:20:34.657 INFO:journalctl@ceph.mon.a.smithi043.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-26T08:20:34.657 INFO:journalctl@ceph.mon.a.smithi043.stdout: legacy client fscid: -1 2024-03-26T08:20:34.657 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-03-26T08:20:34.657 INFO:journalctl@ceph.mon.a.smithi043.stdout: No filesystems configured 2024-03-26T08:20:34.657 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 systemd[1]: Started Ceph mon.a for 88759420-eb49-11ee-95d0-87774f69a715. 2024-03-26T08:20:34.658 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-26T08:20:34.658 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T08:20:34.658 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T08:20:34.658 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T08:20:34.658 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-26T08:20:34.658 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-26T08:20:34.659 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: monmap e1: 1 mons at {a=[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T08:20:34.659 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: fsmap 2024-03-26T08:20:34.659 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: osdmap e1: 0 total, 0 up, 0 in 2024-03-26T08:20:34.659 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:34 smithi043 ceph-mon[31929]: mgrmap e1: no daemons active 2024-03-26T08:20:35.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.153+0000 7f1a34000700 1 Processor -- start 2024-03-26T08:20:35.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.153+0000 7f1a34000700 1 -- start start 2024-03-26T08:20:35.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.153+0000 7f1a34000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a2c108310 0x7f1a2c10a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:20:35.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.153+0000 7f1a34000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c07aec0 con 0x7f1a2c108310 2024-03-26T08:20:35.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.154+0000 7f1a31600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a2c108310 0x7f1a2c10a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:20:35.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.154+0000 7f1a31600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a2c108310 0x7f1a2c10a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49584/0 (socket says 172.21.15.43:49584) 2024-03-26T08:20:35.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.154+0000 7f1a31600700 1 -- 172.21.15.43:0/746656515 learned_addr learned my addr 172.21.15.43:0/746656515 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:20:35.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.155+0000 7f1a31600700 1 -- 172.21.15.43:0/746656515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2c10ac40 con 0x7f1a2c108310 2024-03-26T08:20:35.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.155+0000 7f1a31600700 1 --2- 172.21.15.43:0/746656515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a2c108310 0x7f1a2c10a700 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1a200098a0 tx=0x7f1a20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eaf6a1181fdfb76 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:20:35.879 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.155+0000 7f1a2be00700 1 -- 172.21.15.43:0/746656515 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a20004160 con 0x7f1a2c108310 2024-03-26T08:20:35.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.155+0000 7f1a2be00700 1 -- 172.21.15.43:0/746656515 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f1a20004300 con 0x7f1a2c108310 2024-03-26T08:20:35.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.155+0000 7f1a2be00700 1 -- 172.21.15.43:0/746656515 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a20011a80 con 0x7f1a2c108310 2024-03-26T08:20:35.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.156+0000 7f1a34000700 1 -- 172.21.15.43:0/746656515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a2c108310 msgr2=0x7f1a2c10a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:20:35.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.156+0000 7f1a34000700 1 --2- 172.21.15.43:0/746656515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a2c108310 0x7f1a2c10a700 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1a200098a0 tx=0x7f1a20009bb0 comp rx=0 tx=0).stop 2024-03-26T08:20:35.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.156+0000 7f1a34000700 1 -- 172.21.15.43:0/746656515 shutdown_connections 2024-03-26T08:20:35.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.156+0000 7f1a34000700 1 --2- 172.21.15.43:0/746656515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a2c108310 0x7f1a2c10a700 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:20:35.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.156+0000 7f1a34000700 1 -- 172.21.15.43:0/746656515 >> 172.21.15.43:0/746656515 conn(0x7f1a2c101e00 msgr2=0x7f1a2c104220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:20:35.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.157+0000 7f1a34000700 1 -- 172.21.15.43:0/746656515 shutdown_connections 2024-03-26T08:20:35.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.157+0000 7f1a34000700 1 -- 172.21.15.43:0/746656515 wait complete. 2024-03-26T08:20:35.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.158+0000 7f1a34000700 1 Processor -- start 2024-03-26T08:20:35.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.158+0000 7f1a34000700 1 -- start start 2024-03-26T08:20:35.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.158+0000 7f1a34000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a2c108310 0x7f1a2c19d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:20:35.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.158+0000 7f1a34000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c10bc10 con 0x7f1a2c108310 2024-03-26T08:20:35.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.158+0000 7f1a31600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a2c108310 0x7f1a2c19d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:20:35.880 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.158+0000 7f1a31600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a2c108310 0x7f1a2c19d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49590/0 (socket says 172.21.15.43:49590) 2024-03-26T08:20:35.881 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.158+0000 7f1a31600700 1 -- 172.21.15.43:0/2687609780 learned_addr learned my addr 172.21.15.43:0/2687609780 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:20:35.881 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.159+0000 7f1a31600700 1 -- 172.21.15.43:0/2687609780 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2c19df20 con 0x7f1a2c108310 2024-03-26T08:20:35.881 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.159+0000 7f1a31600700 1 --2- 172.21.15.43:0/2687609780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a2c108310 0x7f1a2c19d9e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1a20011a50 tx=0x7f1a20004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:20:35.881 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.160+0000 7f1a2a000700 1 -- 172.21.15.43:0/2687609780 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a200039f0 con 0x7f1a2c108310 2024-03-26T08:20:35.881 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.160+0000 7f1a2a000700 1 -- 172.21.15.43:0/2687609780 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f1a20003b90 con 0x7f1a2c108310 2024-03-26T08:20:35.881 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.160+0000 7f1a2a000700 1 -- 172.21.15.43:0/2687609780 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a2001a600 con 0x7f1a2c108310 2024-03-26T08:20:35.881 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.160+0000 7f1a34000700 1 -- 172.21.15.43:0/2687609780 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a2c19e1b0 con 0x7f1a2c108310 2024-03-26T08:20:35.881 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.160+0000 7f1a34000700 1 -- 172.21.15.43:0/2687609780 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a2c1a0ea0 con 0x7f1a2c108310 2024-03-26T08:20:35.881 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.160+0000 7f1a2a000700 1 -- 172.21.15.43:0/2687609780 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f1a2001ac40 con 0x7f1a2c108310 2024-03-26T08:20:35.881 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.160+0000 7f1a2a000700 1 -- 172.21.15.43:0/2687609780 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1a20019700 con 0x7f1a2c108310 2024-03-26T08:20:35.881 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.161+0000 7f1a34000700 1 -- 172.21.15.43:0/2687609780 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a2c0637c0 con 0x7f1a2c108310 2024-03-26T08:20:35.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.164+0000 7f1a2a000700 1 -- 172.21.15.43:0/2687609780 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f1a20024a40 con 0x7f1a2c108310 2024-03-26T08:20:35.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.241+0000 7f1a34000700 1 -- 172.21.15.43:0/2687609780 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f1a2c1a1070 con 0x7f1a2c108310 2024-03-26T08:20:35.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.246+0000 7f1a2a000700 1 -- 172.21.15.43:0/2687609780 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f1a20024440 con 0x7f1a2c108310 2024-03-26T08:20:35.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.248+0000 7f1a34000700 1 -- 172.21.15.43:0/2687609780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a2c108310 msgr2=0x7f1a2c19d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:20:35.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.248+0000 7f1a34000700 1 --2- 172.21.15.43:0/2687609780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a2c108310 0x7f1a2c19d9e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1a20011a50 tx=0x7f1a20004290 comp rx=0 tx=0).stop 2024-03-26T08:20:35.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.248+0000 7f1a34000700 1 -- 172.21.15.43:0/2687609780 shutdown_connections 2024-03-26T08:20:35.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.248+0000 7f1a34000700 1 --2- 172.21.15.43:0/2687609780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a2c108310 0x7f1a2c19d9e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:20:35.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.248+0000 7f1a34000700 1 -- 172.21.15.43:0/2687609780 >> 172.21.15.43:0/2687609780 conn(0x7f1a2c101e00 msgr2=0x7f1a2c102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:20:35.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.249+0000 7f1a34000700 1 -- 172.21.15.43:0/2687609780 shutdown_connections 2024-03-26T08:20:35.882 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:35.249+0000 7f1a34000700 1 -- 172.21.15.43:0/2687609780 wait complete. 2024-03-26T08:20:35.882 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-26T08:20:35.907 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-26T08:20:35.907 INFO:teuthology.orchestra.run.smithi043.stdout:Creating mgr... 2024-03-26T08:20:35.908 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-26T08:20:35.909 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-26T08:20:36.314 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-88759420-eb49-11ee-95d0-87774f69a715@mgr.x 2024-03-26T08:20:36.315 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to reset failed state of unit ceph-88759420-eb49-11ee-95d0-87774f69a715@mgr.x.service: Unit ceph-88759420-eb49-11ee-95d0-87774f69a715@mgr.x.service not loaded. 2024-03-26T08:20:36.573 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-88759420-eb49-11ee-95d0-87774f69a715.target.wants/ceph-88759420-eb49-11ee-95d0-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-88759420-eb49-11ee-95d0-87774f69a715@.service. 2024-03-26T08:20:36.604 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:36 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2687609780' entity='client.admin' 2024-03-26T08:20:36.904 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:36 smithi043 systemd[1]: Starting Ceph mgr.x for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:20:37.312 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:36 smithi043 podman[32154]: 2024-03-26 08:20:36.880029673 +0000 UTC m=+0.035046553 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T08:20:37.312 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:36 smithi043 podman[32154]: 2024-03-26 08:20:36.998468691 +0000 UTC m=+0.153485569 container create ec1554ba1ef064986344b7d812fd120cb0f19e3bee06d4d80db09202028f184b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T08:20:37.312 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:37 smithi043 podman[32154]: 2024-03-26 08:20:37.210396048 +0000 UTC m=+0.365413000 container init ec1554ba1ef064986344b7d812fd120cb0f19e3bee06d4d80db09202028f184b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-26T08:20:37.312 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:37 smithi043 podman[32154]: 2024-03-26 08:20:37.220749391 +0000 UTC m=+0.375766276 container start ec1554ba1ef064986344b7d812fd120cb0f19e3bee06d4d80db09202028f184b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=) 2024-03-26T08:20:37.312 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:37 smithi043 bash[32154]: ec1554ba1ef064986344b7d812fd120cb0f19e3bee06d4d80db09202028f184b 2024-03-26T08:20:37.347 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-26T08:20:37.347 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-03-26T08:20:37.363 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-26T08:20:37.363 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-03-26T08:20:37.363 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-03-26T08:20:37.363 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to enable service . firewalld.service is not available 2024-03-26T08:20:37.383 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-26T08:20:37.383 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-03-26T08:20:37.400 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-26T08:20:37.400 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-03-26T08:20:37.400 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-03-26T08:20:37.401 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-26T08:20:37.401 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr to start... 2024-03-26T08:20:37.401 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr... 2024-03-26T08:20:37.588 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:37 smithi043 systemd[1]: Started Ceph mgr.x for 88759420-eb49-11ee-95d0-87774f69a715. 2024-03-26T08:20:38.125 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:38 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:38.024+0000 7faf5ba91200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T08:20:38.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:38 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2974032574' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T08:20:38.404 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:38 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:38.151+0000 7faf5ba91200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T08:20:39.004 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-03-26T08:20:39.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-03-26T08:20:39.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "88759420-eb49-11ee-95d0-87774f69a715", 2024-03-26T08:20:39.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T08:20:39.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T08:20:39.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T08:20:39.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T08:20:39.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:20:39.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T08:20:39.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T08:20:39.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-03-26T08:20:39.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-03-26T08:20:39.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T08:20:39.007 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T08:20:39.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-03-26T08:20:39.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-03-26T08:20:39.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T08:20:39.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:20:39.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T08:20:39.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T08:20:39.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:20:39.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T08:20:39.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:20:39.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T08:20:39.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T08:20:39.008 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T08:20:39.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T08:20:39.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T08:20:39.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T08:20:39.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:20:39.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T08:20:39.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T08:20:39.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T08:20:39.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T08:20:39.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T08:20:39.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T08:20:39.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T08:20:39.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T08:20:39.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T08:20:39.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:20:39.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T08:20:39.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:20:39.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T08:20:39.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T08:20:39.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:20:39.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T08:20:39.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-26T08:20:39.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T08:20:39.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T08:20:39.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T08:20:39.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T08:20:39.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T08:20:39.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-03-26T08:20:39.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T08:20:39.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:20:39.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T08:20:39.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:20:39.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T08:20:27.817727+0000", 2024-03-26T08:20:39.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T08:20:39.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:20:39.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T08:20:39.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-03-26T08:20:39.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.036+0000 7fc046c00700 1 Processor -- start 2024-03-26T08:20:39.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.036+0000 7fc046c00700 1 -- start start 2024-03-26T08:20:39.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.036+0000 7fc046c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc040107930 0x7fc040107d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:20:39.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.036+0000 7fc046c00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc040108320 con 0x7fc040107930 2024-03-26T08:20:39.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.037+0000 7fc03fe00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc040107930 0x7fc040107d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:20:39.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.037+0000 7fc03fe00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc040107930 0x7fc040107d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49622/0 (socket says 172.21.15.43:49622) 2024-03-26T08:20:39.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.037+0000 7fc03fe00700 1 -- 172.21.15.43:0/1056262141 learned_addr learned my addr 172.21.15.43:0/1056262141 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:20:39.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.037+0000 7fc03fe00700 1 -- 172.21.15.43:0/1056262141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc040108ae0 con 0x7fc040107930 2024-03-26T08:20:39.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.038+0000 7fc03fe00700 1 --2- 172.21.15.43:0/1056262141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc040107930 0x7fc040107d50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc0300098a0 tx=0x7fc030009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c55e4e524f70027 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:20:39.020 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.038+0000 7fc03ea00700 1 -- 172.21.15.43:0/1056262141 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc030004160 con 0x7fc040107930 2024-03-26T08:20:39.020 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.038+0000 7fc03ea00700 1 -- 172.21.15.43:0/1056262141 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fc030004300 con 0x7fc040107930 2024-03-26T08:20:39.020 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.038+0000 7fc03ea00700 1 -- 172.21.15.43:0/1056262141 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc030011a80 con 0x7fc040107930 2024-03-26T08:20:39.020 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.039+0000 7fc046c00700 1 -- 172.21.15.43:0/1056262141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc040107930 msgr2=0x7fc040107d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:20:39.020 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.039+0000 7fc046c00700 1 --2- 172.21.15.43:0/1056262141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc040107930 0x7fc040107d50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc0300098a0 tx=0x7fc030009bb0 comp rx=0 tx=0).stop 2024-03-26T08:20:39.020 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.039+0000 7fc046c00700 1 -- 172.21.15.43:0/1056262141 shutdown_connections 2024-03-26T08:20:39.020 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.039+0000 7fc046c00700 1 --2- 172.21.15.43:0/1056262141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc040107930 0x7fc040107d50 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:20:39.020 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.039+0000 7fc046c00700 1 -- 172.21.15.43:0/1056262141 >> 172.21.15.43:0/1056262141 conn(0x7fc040102f70 msgr2=0x7fc040105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:20:39.020 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.039+0000 7fc046c00700 1 -- 172.21.15.43:0/1056262141 shutdown_connections 2024-03-26T08:20:39.020 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.039+0000 7fc046c00700 1 -- 172.21.15.43:0/1056262141 wait complete. 2024-03-26T08:20:39.020 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.040+0000 7fc046c00700 1 Processor -- start 2024-03-26T08:20:39.020 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.040+0000 7fc046c00700 1 -- start start 2024-03-26T08:20:39.020 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.041+0000 7fc046c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc040107930 0x7fc04019d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:20:39.020 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.041+0000 7fc046c00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04010bd10 con 0x7fc040107930 2024-03-26T08:20:39.021 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.041+0000 7fc03fe00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc040107930 0x7fc04019d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:20:39.021 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.041+0000 7fc03fe00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc040107930 0x7fc04019d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49624/0 (socket says 172.21.15.43:49624) 2024-03-26T08:20:39.021 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.041+0000 7fc03fe00700 1 -- 172.21.15.43:0/2974032574 learned_addr learned my addr 172.21.15.43:0/2974032574 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:20:39.021 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.041+0000 7fc03fe00700 1 -- 172.21.15.43:0/2974032574 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04019def0 con 0x7fc040107930 2024-03-26T08:20:39.021 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.042+0000 7fc03fe00700 1 --2- 172.21.15.43:0/2974032574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc040107930 0x7fc04019d9b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc030000c00 tx=0x7fc030011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:20:39.021 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.042+0000 7fc03cc00700 1 -- 172.21.15.43:0/2974032574 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc0300039f0 con 0x7fc040107930 2024-03-26T08:20:39.021 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.042+0000 7fc03cc00700 1 -- 172.21.15.43:0/2974032574 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fc030003b90 con 0x7fc040107930 2024-03-26T08:20:39.021 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.042+0000 7fc046c00700 1 -- 172.21.15.43:0/2974032574 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc04019e180 con 0x7fc040107930 2024-03-26T08:20:39.021 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.042+0000 7fc03cc00700 1 -- 172.21.15.43:0/2974032574 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc03001a740 con 0x7fc040107930 2024-03-26T08:20:39.021 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.042+0000 7fc046c00700 1 -- 172.21.15.43:0/2974032574 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0401a0e70 con 0x7fc040107930 2024-03-26T08:20:39.021 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.043+0000 7fc03cc00700 1 -- 172.21.15.43:0/2974032574 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fc03002d3f0 con 0x7fc040107930 2024-03-26T08:20:39.021 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.043+0000 7fc03cc00700 1 -- 172.21.15.43:0/2974032574 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc030019700 con 0x7fc040107930 2024-03-26T08:20:39.021 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.043+0000 7fc046c00700 1 -- 172.21.15.43:0/2974032574 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc04010be90 con 0x7fc040107930 2024-03-26T08:20:39.022 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.046+0000 7fc03cc00700 1 -- 172.21.15.43:0/2974032574 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fc030024af0 con 0x7fc040107930 2024-03-26T08:20:39.022 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.125+0000 7fc046c00700 1 -- 172.21.15.43:0/2974032574 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc0401a13a0 con 0x7fc040107930 2024-03-26T08:20:39.022 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.126+0000 7fc03cc00700 1 -- 172.21.15.43:0/2974032574 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fc030032030 con 0x7fc040107930 2024-03-26T08:20:39.022 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.127+0000 7fc046c00700 1 -- 172.21.15.43:0/2974032574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc040107930 msgr2=0x7fc04019d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:20:39.022 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.127+0000 7fc046c00700 1 --2- 172.21.15.43:0/2974032574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc040107930 0x7fc04019d9b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc030000c00 tx=0x7fc030011b00 comp rx=0 tx=0).stop 2024-03-26T08:20:39.022 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.128+0000 7fc046c00700 1 -- 172.21.15.43:0/2974032574 shutdown_connections 2024-03-26T08:20:39.022 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.128+0000 7fc046c00700 1 --2- 172.21.15.43:0/2974032574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc040107930 0x7fc04019d9b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:20:39.022 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.128+0000 7fc046c00700 1 -- 172.21.15.43:0/2974032574 >> 172.21.15.43:0/2974032574 conn(0x7fc040102f70 msgr2=0x7fc040103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:20:39.022 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.128+0000 7fc046c00700 1 -- 172.21.15.43:0/2974032574 shutdown_connections 2024-03-26T08:20:39.022 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:38.128+0000 7fc046c00700 1 -- 172.21.15.43:0/2974032574 wait complete. 2024-03-26T08:20:39.022 INFO:teuthology.orchestra.run.smithi043.stdout:mgr not available, waiting (1/15)... 2024-03-26T08:20:39.298 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:38 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:38.991+0000 7faf5ba91200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T08:20:39.298 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:39 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:39.119+0000 7faf5ba91200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T08:20:39.298 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:39 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:39.301+0000 7faf5ba91200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T08:20:39.654 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:39 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:39.429+0000 7faf5ba91200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T08:20:39.654 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:39 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:39.541+0000 7faf5ba91200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T08:20:40.137 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:39 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:39.883+0000 7faf5ba91200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T08:20:40.137 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:40 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:40.026+0000 7faf5ba91200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T08:20:40.404 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:40 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:40.141+0000 7faf5ba91200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T08:20:40.404 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:40 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:40.268+0000 7faf5ba91200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T08:20:40.904 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:40 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:40.490+0000 7faf5ba91200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T08:20:42.050 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:41 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:41.876+0000 7faf5ba91200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T08:20:42.050 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:41 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2002201104' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T08:20:42.404 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:42 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:42.054+0000 7faf5ba91200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T08:20:42.505 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-03-26T08:20:42.507 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-03-26T08:20:42.507 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "88759420-eb49-11ee-95d0-87774f69a715", 2024-03-26T08:20:42.507 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T08:20:42.507 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T08:20:42.507 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T08:20:42.507 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T08:20:42.507 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:20:42.511 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T08:20:42.511 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T08:20:42.511 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-03-26T08:20:42.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-03-26T08:20:42.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T08:20:42.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T08:20:42.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-03-26T08:20:42.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-03-26T08:20:42.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T08:20:42.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:20:42.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T08:20:42.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T08:20:42.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:20:42.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T08:20:42.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:20:42.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T08:20:42.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T08:20:42.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T08:20:42.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T08:20:42.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T08:20:42.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T08:20:42.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:20:42.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T08:20:42.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T08:20:42.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T08:20:42.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T08:20:42.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T08:20:42.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T08:20:42.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T08:20:42.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T08:20:42.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T08:20:42.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:20:42.514 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T08:20:42.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:20:42.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T08:20:42.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T08:20:42.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:20:42.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T08:20:42.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-26T08:20:42.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T08:20:42.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T08:20:42.522 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T08:20:42.522 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T08:20:42.522 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T08:20:42.522 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-03-26T08:20:42.522 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T08:20:42.522 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:20:42.522 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T08:20:42.522 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:20:42.522 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T08:20:27.817727+0000", 2024-03-26T08:20:42.522 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T08:20:42.522 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:20:42.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T08:20:42.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-03-26T08:20:42.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.676+0000 7fed79400700 1 Processor -- start 2024-03-26T08:20:42.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.676+0000 7fed79400700 1 -- start start 2024-03-26T08:20:42.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.676+0000 7fed79400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed74109ac0 0x7fed74109ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:20:42.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.676+0000 7fed79400700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7410a4b0 con 0x7fed74109ac0 2024-03-26T08:20:42.524 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.677+0000 7fed72800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed74109ac0 0x7fed74109ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:20:42.524 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.677+0000 7fed72800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed74109ac0 0x7fed74109ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49626/0 (socket says 172.21.15.43:49626) 2024-03-26T08:20:42.524 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.677+0000 7fed72800700 1 -- 172.21.15.43:0/369650106 learned_addr learned my addr 172.21.15.43:0/369650106 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:20:42.524 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.678+0000 7fed72800700 1 -- 172.21.15.43:0/369650106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7410ac70 con 0x7fed74109ac0 2024-03-26T08:20:42.524 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.678+0000 7fed72800700 1 --2- 172.21.15.43:0/369650106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed74109ac0 0x7fed74109ee0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fed640098a0 tx=0x7fed64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592a32225ddd532 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:20:42.525 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.678+0000 7fed71400700 1 -- 172.21.15.43:0/369650106 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed64004160 con 0x7fed74109ac0 2024-03-26T08:20:42.525 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.678+0000 7fed71400700 1 -- 172.21.15.43:0/369650106 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fed64004300 con 0x7fed74109ac0 2024-03-26T08:20:42.525 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.678+0000 7fed71400700 1 -- 172.21.15.43:0/369650106 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed64011a80 con 0x7fed74109ac0 2024-03-26T08:20:42.525 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.679+0000 7fed79400700 1 -- 172.21.15.43:0/369650106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed74109ac0 msgr2=0x7fed74109ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:20:42.525 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.679+0000 7fed79400700 1 --2- 172.21.15.43:0/369650106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed74109ac0 0x7fed74109ee0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fed640098a0 tx=0x7fed64009bb0 comp rx=0 tx=0).stop 2024-03-26T08:20:42.525 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.679+0000 7fed79400700 1 -- 172.21.15.43:0/369650106 shutdown_connections 2024-03-26T08:20:42.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.679+0000 7fed79400700 1 --2- 172.21.15.43:0/369650106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed74109ac0 0x7fed74109ee0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:20:42.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.679+0000 7fed79400700 1 -- 172.21.15.43:0/369650106 >> 172.21.15.43:0/369650106 conn(0x7fed741054d0 msgr2=0x7fed741078c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:20:42.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.679+0000 7fed79400700 1 -- 172.21.15.43:0/369650106 shutdown_connections 2024-03-26T08:20:42.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.679+0000 7fed79400700 1 -- 172.21.15.43:0/369650106 wait complete. 2024-03-26T08:20:42.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.680+0000 7fed79400700 1 Processor -- start 2024-03-26T08:20:42.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.680+0000 7fed79400700 1 -- start start 2024-03-26T08:20:42.527 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.681+0000 7fed79400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed74109ac0 0x7fed74080b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:20:42.527 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.681+0000 7fed79400700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7410e020 con 0x7fed74109ac0 2024-03-26T08:20:42.527 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.681+0000 7fed72800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed74109ac0 0x7fed74080b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:20:42.527 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.681+0000 7fed72800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed74109ac0 0x7fed74080b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49628/0 (socket says 172.21.15.43:49628) 2024-03-26T08:20:42.527 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.681+0000 7fed72800700 1 -- 172.21.15.43:0/2002201104 learned_addr learned my addr 172.21.15.43:0/2002201104 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:20:42.527 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.681+0000 7fed72800700 1 -- 172.21.15.43:0/2002201104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed740810d0 con 0x7fed74109ac0 2024-03-26T08:20:42.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.682+0000 7fed72800700 1 --2- 172.21.15.43:0/2002201104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed74109ac0 0x7fed74080b90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fed64000c00 tx=0x7fed64011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:20:42.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.682+0000 7fed6b400700 1 -- 172.21.15.43:0/2002201104 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed640039f0 con 0x7fed74109ac0 2024-03-26T08:20:42.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.682+0000 7fed6b400700 1 -- 172.21.15.43:0/2002201104 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fed64003b90 con 0x7fed74109ac0 2024-03-26T08:20:42.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.682+0000 7fed6b400700 1 -- 172.21.15.43:0/2002201104 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed6401a740 con 0x7fed74109ac0 2024-03-26T08:20:42.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.682+0000 7fed79400700 1 -- 172.21.15.43:0/2002201104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed74081360 con 0x7fed74109ac0 2024-03-26T08:20:42.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.682+0000 7fed79400700 1 -- 172.21.15.43:0/2002201104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7407d920 con 0x7fed74109ac0 2024-03-26T08:20:42.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.682+0000 7fed6b400700 1 -- 172.21.15.43:0/2002201104 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fed6402d3f0 con 0x7fed74109ac0 2024-03-26T08:20:42.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.682+0000 7fed6b400700 1 -- 172.21.15.43:0/2002201104 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fed64019700 con 0x7fed74109ac0 2024-03-26T08:20:42.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.683+0000 7fed79400700 1 -- 172.21.15.43:0/2002201104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed74081530 con 0x7fed74109ac0 2024-03-26T08:20:42.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.686+0000 7fed6b400700 1 -- 172.21.15.43:0/2002201104 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fed74081530 con 0x7fed74109ac0 2024-03-26T08:20:42.528 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.766+0000 7fed79400700 1 -- 172.21.15.43:0/2002201104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fed740637c0 con 0x7fed74109ac0 2024-03-26T08:20:42.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.767+0000 7fed6b400700 1 -- 172.21.15.43:0/2002201104 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fed740637c0 con 0x7fed74109ac0 2024-03-26T08:20:42.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.769+0000 7fed79400700 1 -- 172.21.15.43:0/2002201104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed74109ac0 msgr2=0x7fed74080b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:20:42.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.769+0000 7fed79400700 1 --2- 172.21.15.43:0/2002201104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed74109ac0 0x7fed74080b90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fed64000c00 tx=0x7fed64011b00 comp rx=0 tx=0).stop 2024-03-26T08:20:42.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.769+0000 7fed79400700 1 -- 172.21.15.43:0/2002201104 shutdown_connections 2024-03-26T08:20:42.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.769+0000 7fed79400700 1 --2- 172.21.15.43:0/2002201104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed74109ac0 0x7fed74080b90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:20:42.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.769+0000 7fed79400700 1 -- 172.21.15.43:0/2002201104 >> 172.21.15.43:0/2002201104 conn(0x7fed741054d0 msgr2=0x7fed74106f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:20:42.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.769+0000 7fed79400700 1 -- 172.21.15.43:0/2002201104 shutdown_connections 2024-03-26T08:20:42.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:41.769+0000 7fed79400700 1 -- 172.21.15.43:0/2002201104 wait complete. 2024-03-26T08:20:42.530 INFO:teuthology.orchestra.run.smithi043.stdout:mgr not available, waiting (2/15)... 2024-03-26T08:20:42.878 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:42 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:42.494+0000 7faf5ba91200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T08:20:42.878 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:42 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:42.618+0000 7faf5ba91200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T08:20:43.128 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:42 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:42.881+0000 7faf5ba91200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T08:20:43.128 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:43 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:43.132+0000 7faf5ba91200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T08:20:43.833 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:43 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:43.514+0000 7faf5ba91200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T08:20:44.145 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:43 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:43.837+0000 7faf5ba91200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T08:20:44.404 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:44 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:44.149+0000 7faf5ba91200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T08:20:45.604 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:45 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/136284636' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T08:20:45.605 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:45 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:45.254+0000 7faf5ba91200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T08:20:45.910 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-03-26T08:20:45.910 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-03-26T08:20:45.910 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "88759420-eb49-11ee-95d0-87774f69a715", 2024-03-26T08:20:45.910 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T08:20:45.910 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T08:20:45.910 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T08:20:45.912 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T08:20:45.912 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:20:45.912 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T08:20:45.912 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T08:20:45.912 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-03-26T08:20:45.913 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-03-26T08:20:45.913 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T08:20:45.913 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T08:20:45.913 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-03-26T08:20:45.913 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-03-26T08:20:45.913 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T08:20:45.913 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:20:45.913 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T08:20:45.913 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T08:20:45.913 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:20:45.913 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T08:20:45.913 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:20:45.913 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T08:20:45.913 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T08:20:45.914 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T08:20:45.922 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T08:20:45.923 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T08:20:45.923 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T08:20:45.923 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:20:45.923 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T08:20:45.923 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T08:20:45.923 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T08:20:45.923 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T08:20:45.923 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T08:20:45.923 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T08:20:45.923 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T08:20:45.923 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T08:20:45.923 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T08:20:45.923 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:20:45.923 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T08:20:45.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:20:45.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T08:20:45.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T08:20:45.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:20:45.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T08:20:45.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-26T08:20:45.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T08:20:45.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T08:20:45.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T08:20:45.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T08:20:45.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T08:20:45.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-03-26T08:20:45.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T08:20:45.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:20:45.924 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T08:20:45.925 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:20:45.925 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T08:20:27.817727+0000", 2024-03-26T08:20:45.925 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T08:20:45.925 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:20:45.925 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T08:20:45.925 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-03-26T08:20:45.925 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.107+0000 7f60c4a00700 1 Processor -- start 2024-03-26T08:20:45.925 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.108+0000 7f60c4a00700 1 -- start start 2024-03-26T08:20:45.925 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.108+0000 7f60c4a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60c0108300 0x7f60c010a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:20:45.925 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.108+0000 7f60c4a00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c007aec0 con 0x7f60c0108300 2024-03-26T08:20:45.925 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.108+0000 7f60bde00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60c0108300 0x7f60c010a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:20:45.925 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.108+0000 7f60bde00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60c0108300 0x7f60c010a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49050/0 (socket says 172.21.15.43:49050) 2024-03-26T08:20:45.925 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.108+0000 7f60bde00700 1 -- 172.21.15.43:0/218459088 learned_addr learned my addr 172.21.15.43:0/218459088 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:20:45.925 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.109+0000 7f60bde00700 1 -- 172.21.15.43:0/218459088 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c010ac30 con 0x7f60c0108300 2024-03-26T08:20:45.926 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.109+0000 7f60bde00700 1 --2- 172.21.15.43:0/218459088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60c0108300 0x7f60c010a6f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f60b00098a0 tx=0x7f60b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38306b2084d155d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:20:45.926 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.110+0000 7f60bca00700 1 -- 172.21.15.43:0/218459088 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60b0004160 con 0x7f60c0108300 2024-03-26T08:20:45.926 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.110+0000 7f60bca00700 1 -- 172.21.15.43:0/218459088 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f60b0004300 con 0x7f60c0108300 2024-03-26T08:20:45.926 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.110+0000 7f60bca00700 1 -- 172.21.15.43:0/218459088 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60b0011a80 con 0x7f60c0108300 2024-03-26T08:20:45.926 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.110+0000 7f60c4a00700 1 -- 172.21.15.43:0/218459088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60c0108300 msgr2=0x7f60c010a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:20:45.926 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.110+0000 7f60c4a00700 1 --2- 172.21.15.43:0/218459088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60c0108300 0x7f60c010a6f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f60b00098a0 tx=0x7f60b0009bb0 comp rx=0 tx=0).stop 2024-03-26T08:20:45.926 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.111+0000 7f60c4a00700 1 -- 172.21.15.43:0/218459088 shutdown_connections 2024-03-26T08:20:45.926 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.111+0000 7f60c4a00700 1 --2- 172.21.15.43:0/218459088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60c0108300 0x7f60c010a6f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:20:45.926 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.111+0000 7f60c4a00700 1 -- 172.21.15.43:0/218459088 >> 172.21.15.43:0/218459088 conn(0x7f60c0101e10 msgr2=0x7f60c0104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:20:45.926 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.111+0000 7f60c4a00700 1 -- 172.21.15.43:0/218459088 shutdown_connections 2024-03-26T08:20:45.926 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.111+0000 7f60c4a00700 1 -- 172.21.15.43:0/218459088 wait complete. 2024-03-26T08:20:45.926 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.112+0000 7f60c4a00700 1 Processor -- start 2024-03-26T08:20:45.927 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.112+0000 7f60c4a00700 1 -- start start 2024-03-26T08:20:45.927 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.113+0000 7f60c4a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60c0108300 0x7f60c019b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:20:45.927 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.113+0000 7f60c4a00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c010bc00 con 0x7f60c0108300 2024-03-26T08:20:45.927 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.113+0000 7f60bde00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60c0108300 0x7f60c019b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:20:45.927 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.113+0000 7f60bde00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60c0108300 0x7f60c019b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49062/0 (socket says 172.21.15.43:49062) 2024-03-26T08:20:45.927 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.113+0000 7f60bde00700 1 -- 172.21.15.43:0/136284636 learned_addr learned my addr 172.21.15.43:0/136284636 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:20:45.927 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.113+0000 7f60bde00700 1 -- 172.21.15.43:0/136284636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c019bcb0 con 0x7f60c0108300 2024-03-26T08:20:45.927 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.114+0000 7f60bde00700 1 --2- 172.21.15.43:0/136284636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60c0108300 0x7f60c019b770 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f60b0011a50 tx=0x7f60b0011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:20:45.927 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.114+0000 7f60b6a00700 1 -- 172.21.15.43:0/136284636 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60b0003a80 con 0x7f60c0108300 2024-03-26T08:20:45.927 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.114+0000 7f60b6a00700 1 -- 172.21.15.43:0/136284636 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f60b0003c20 con 0x7f60c0108300 2024-03-26T08:20:45.927 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.114+0000 7f60b6a00700 1 -- 172.21.15.43:0/136284636 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60b001a650 con 0x7f60c0108300 2024-03-26T08:20:45.927 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.114+0000 7f60c4a00700 1 -- 172.21.15.43:0/136284636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c019bf40 con 0x7f60c0108300 2024-03-26T08:20:45.927 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.114+0000 7f60c4a00700 1 -- 172.21.15.43:0/136284636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c019ec30 con 0x7f60c0108300 2024-03-26T08:20:45.927 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.115+0000 7f60b6a00700 1 -- 172.21.15.43:0/136284636 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f60b001ac90 con 0x7f60c0108300 2024-03-26T08:20:45.928 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.115+0000 7f60b6a00700 1 -- 172.21.15.43:0/136284636 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f60b0019700 con 0x7f60c0108300 2024-03-26T08:20:45.928 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.116+0000 7f60c4a00700 1 -- 172.21.15.43:0/136284636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c00637c0 con 0x7f60c0108300 2024-03-26T08:20:45.928 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.118+0000 7f60b6a00700 1 -- 172.21.15.43:0/136284636 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f60b0024ad0 con 0x7f60c0108300 2024-03-26T08:20:45.928 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.238+0000 7f60c4a00700 1 -- 172.21.15.43:0/136284636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f60c019ee00 con 0x7f60c0108300 2024-03-26T08:20:45.928 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.239+0000 7f60b6a00700 1 -- 172.21.15.43:0/136284636 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f60b0024440 con 0x7f60c0108300 2024-03-26T08:20:45.928 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.241+0000 7f60c4a00700 1 -- 172.21.15.43:0/136284636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60c0108300 msgr2=0x7f60c019b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:20:45.928 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.241+0000 7f60c4a00700 1 --2- 172.21.15.43:0/136284636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60c0108300 0x7f60c019b770 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f60b0011a50 tx=0x7f60b0011b30 comp rx=0 tx=0).stop 2024-03-26T08:20:45.928 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.241+0000 7f60c4a00700 1 -- 172.21.15.43:0/136284636 shutdown_connections 2024-03-26T08:20:45.928 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.241+0000 7f60c4a00700 1 --2- 172.21.15.43:0/136284636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60c0108300 0x7f60c019b770 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:20:45.928 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.241+0000 7f60c4a00700 1 -- 172.21.15.43:0/136284636 >> 172.21.15.43:0/136284636 conn(0x7f60c0101e10 msgr2=0x7f60c01028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:20:45.928 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.241+0000 7f60c4a00700 1 -- 172.21.15.43:0/136284636 shutdown_connections 2024-03-26T08:20:45.928 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:45.241+0000 7f60c4a00700 1 -- 172.21.15.43:0/136284636 wait complete. 2024-03-26T08:20:45.928 INFO:teuthology.orchestra.run.smithi043.stdout:mgr not available, waiting (3/15)... 2024-03-26T08:20:46.154 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:46 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:46.025+0000 7faf5ba91200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T08:20:46.475 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:46 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:46.191+0000 7faf5ba91200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T08:20:46.904 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:46 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:46.479+0000 7faf5ba91200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T08:20:46.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:46 smithi043 ceph-mon[31929]: Activating manager daemon x 2024-03-26T08:20:46.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:46 smithi043 ceph-mon[31929]: mgrmap e2: x(active, starting, since 0.0114703s) 2024-03-26T08:20:46.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:46 smithi043 ceph-mon[31929]: from='mgr.14100 172.21.15.43:0/4183139192' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T08:20:46.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:46 smithi043 ceph-mon[31929]: from='mgr.14100 172.21.15.43:0/4183139192' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T08:20:46.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:46 smithi043 ceph-mon[31929]: from='mgr.14100 172.21.15.43:0/4183139192' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T08:20:46.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:46 smithi043 ceph-mon[31929]: from='mgr.14100 172.21.15.43:0/4183139192' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:20:46.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:46 smithi043 ceph-mon[31929]: from='mgr.14100 172.21.15.43:0/4183139192' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T08:20:46.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:46 smithi043 ceph-mon[31929]: Manager daemon x is now available 2024-03-26T08:20:46.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:46 smithi043 ceph-mon[31929]: from='mgr.14100 172.21.15.43:0/4183139192' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-26T08:20:46.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:46 smithi043 ceph-mon[31929]: from='mgr.14100 172.21.15.43:0/4183139192' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-26T08:20:46.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:46 smithi043 ceph-mon[31929]: from='mgr.14100 172.21.15.43:0/4183139192' entity='mgr.x' 2024-03-26T08:20:46.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:46 smithi043 ceph-mon[31929]: from='mgr.14100 172.21.15.43:0/4183139192' entity='mgr.x' 2024-03-26T08:20:47.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:47 smithi043 ceph-mon[31929]: from='mgr.14100 172.21.15.43:0/4183139192' entity='mgr.x' 2024-03-26T08:20:47.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:47 smithi043 ceph-mon[31929]: mgrmap e3: x(active, since 1.02347s) 2024-03-26T08:20:49.576 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-03-26T08:20:49.576 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-03-26T08:20:49.576 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "88759420-eb49-11ee-95d0-87774f69a715", 2024-03-26T08:20:49.576 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T08:20:49.577 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T08:20:49.577 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T08:20:49.577 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T08:20:49.577 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:20:49.577 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T08:20:49.577 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T08:20:49.577 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-03-26T08:20:49.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-03-26T08:20:49.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T08:20:49.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T08:20:49.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-03-26T08:20:49.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-03-26T08:20:49.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T08:20:49.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:20:49.579 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T08:20:49.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T08:20:49.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:20:49.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T08:20:49.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:20:49.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T08:20:49.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T08:20:49.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T08:20:49.588 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T08:20:49.588 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T08:20:49.589 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T08:20:49.589 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:20:49.589 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T08:20:49.589 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T08:20:49.589 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T08:20:49.589 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T08:20:49.589 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T08:20:49.590 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T08:20:49.590 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T08:20:49.590 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T08:20:49.590 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T08:20:49.590 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:20:49.590 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T08:20:49.590 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:20:49.590 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T08:20:49.590 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T08:20:49.590 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:20:49.590 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T08:20:49.590 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-26T08:20:49.590 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T08:20:49.591 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T08:20:49.591 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T08:20:49.591 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T08:20:49.591 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T08:20:49.591 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-03-26T08:20:49.591 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T08:20:49.591 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:20:49.591 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T08:20:49.591 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:20:49.591 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T08:20:27.817727+0000", 2024-03-26T08:20:49.591 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T08:20:49.591 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:20:49.591 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T08:20:49.591 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-03-26T08:20:49.591 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.519+0000 7f4de8c00700 1 Processor -- start 2024-03-26T08:20:49.592 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.519+0000 7f4de8c00700 1 -- start start 2024-03-26T08:20:49.592 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.520+0000 7f4de8c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4de4109910 0x7f4de4109d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:20:49.592 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.520+0000 7f4de8c00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de410a300 con 0x7f4de4109910 2024-03-26T08:20:49.592 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.520+0000 7f4de3400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4de4109910 0x7f4de4109d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:20:49.592 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.520+0000 7f4de3400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4de4109910 0x7f4de4109d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49128/0 (socket says 172.21.15.43:49128) 2024-03-26T08:20:49.592 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.520+0000 7f4de3400700 1 -- 172.21.15.43:0/1837525859 learned_addr learned my addr 172.21.15.43:0/1837525859 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:20:49.592 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.521+0000 7f4de3400700 1 -- 172.21.15.43:0/1837525859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de410aa70 con 0x7f4de4109910 2024-03-26T08:20:49.592 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.521+0000 7f4de3400700 1 --2- 172.21.15.43:0/1837525859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4de4109910 0x7f4de4109d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4dd40098a0 tx=0x7f4dd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c519e265386bf83 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:20:49.592 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.521+0000 7f4de2000700 1 -- 172.21.15.43:0/1837525859 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4dd4004160 con 0x7f4de4109910 2024-03-26T08:20:49.592 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.521+0000 7f4de2000700 1 -- 172.21.15.43:0/1837525859 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f4dd4004300 con 0x7f4de4109910 2024-03-26T08:20:49.592 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.521+0000 7f4de2000700 1 -- 172.21.15.43:0/1837525859 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4dd4011a80 con 0x7f4de4109910 2024-03-26T08:20:49.592 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.522+0000 7f4de8c00700 1 -- 172.21.15.43:0/1837525859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4de4109910 msgr2=0x7f4de4109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:20:49.592 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.522+0000 7f4de8c00700 1 --2- 172.21.15.43:0/1837525859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4de4109910 0x7f4de4109d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4dd40098a0 tx=0x7f4dd4009bb0 comp rx=0 tx=0).stop 2024-03-26T08:20:49.592 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.522+0000 7f4de8c00700 1 -- 172.21.15.43:0/1837525859 shutdown_connections 2024-03-26T08:20:49.593 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.522+0000 7f4de8c00700 1 --2- 172.21.15.43:0/1837525859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4de4109910 0x7f4de4109d30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:20:49.593 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.522+0000 7f4de8c00700 1 -- 172.21.15.43:0/1837525859 >> 172.21.15.43:0/1837525859 conn(0x7f4de407ce20 msgr2=0x7f4de41074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:20:49.593 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.522+0000 7f4de8c00700 1 -- 172.21.15.43:0/1837525859 shutdown_connections 2024-03-26T08:20:49.593 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.522+0000 7f4de8c00700 1 -- 172.21.15.43:0/1837525859 wait complete. 2024-03-26T08:20:49.593 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.523+0000 7f4de8c00700 1 Processor -- start 2024-03-26T08:20:49.593 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.523+0000 7f4de8c00700 1 -- start start 2024-03-26T08:20:49.593 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.524+0000 7f4de8c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4de4109910 0x7f4de419d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:20:49.593 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.524+0000 7f4de8c00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de410de20 con 0x7f4de4109910 2024-03-26T08:20:49.593 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.524+0000 7f4de3400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4de4109910 0x7f4de419d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:20:49.593 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.524+0000 7f4de3400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4de4109910 0x7f4de419d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49142/0 (socket says 172.21.15.43:49142) 2024-03-26T08:20:49.593 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.524+0000 7f4de3400700 1 -- 172.21.15.43:0/901388340 learned_addr learned my addr 172.21.15.43:0/901388340 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:20:49.593 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.524+0000 7f4de3400700 1 -- 172.21.15.43:0/901388340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de419dd40 con 0x7f4de4109910 2024-03-26T08:20:49.593 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.525+0000 7f4de3400700 1 --2- 172.21.15.43:0/901388340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4de4109910 0x7f4de419d800 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4000c00 tx=0x7f4dd4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:20:49.593 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.525+0000 7f4ddbe00700 1 -- 172.21.15.43:0/901388340 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4dd40039f0 con 0x7f4de4109910 2024-03-26T08:20:49.593 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.525+0000 7f4ddbe00700 1 -- 172.21.15.43:0/901388340 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f4dd4003b90 con 0x7f4de4109910 2024-03-26T08:20:49.594 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.525+0000 7f4ddbe00700 1 -- 172.21.15.43:0/901388340 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4dd401a740 con 0x7f4de4109910 2024-03-26T08:20:49.594 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.525+0000 7f4de8c00700 1 -- 172.21.15.43:0/901388340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de419dfd0 con 0x7f4de4109910 2024-03-26T08:20:49.594 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.525+0000 7f4de8c00700 1 -- 172.21.15.43:0/901388340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de41a0cc0 con 0x7f4de4109910 2024-03-26T08:20:49.594 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.526+0000 7f4ddbe00700 1 -- 172.21.15.43:0/901388340 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 3) v1 ==== 45989+0+0 (secure 0 0 0) 0x7f4dd4024af0 con 0x7f4de4109910 2024-03-26T08:20:49.594 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.526+0000 7f4de8c00700 1 -- 172.21.15.43:0/901388340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4de40637c0 con 0x7f4de4109910 2024-03-26T08:20:49.594 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.527+0000 7f4ddbe00700 1 --2- 172.21.15.43:0/901388340 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7f4dc8038e00 0x7f4dc803b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:20:49.594 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.527+0000 7f4ddbe00700 1 -- 172.21.15.43:0/901388340 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4dd404cf50 con 0x7f4de4109910 2024-03-26T08:20:49.594 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.527+0000 7f4de2a00700 1 --2- 172.21.15.43:0/901388340 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7f4dc8038e00 0x7f4dc803b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:20:49.594 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.528+0000 7f4de2a00700 1 --2- 172.21.15.43:0/901388340 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7f4dc8038e00 0x7f4dc803b2c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0006fd0 tx=0x7f4dd0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:20:49.594 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.533+0000 7f4ddbe00700 1 -- 172.21.15.43:0/901388340 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4dd402d3f0 con 0x7f4de4109910 2024-03-26T08:20:49.594 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.547+0000 7f4ddbe00700 1 -- 172.21.15.43:0/901388340 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7f4dd4050030 con 0x7f4de4109910 2024-03-26T08:20:49.594 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.854+0000 7f4de8c00700 1 -- 172.21.15.43:0/901388340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f4de41a0e90 con 0x7f4de4109910 2024-03-26T08:20:49.595 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.855+0000 7f4ddbe00700 1 -- 172.21.15.43:0/901388340 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f4de41a0e90 con 0x7f4de4109910 2024-03-26T08:20:49.595 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.857+0000 7f4de8c00700 1 -- 172.21.15.43:0/901388340 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7f4dc8038e00 msgr2=0x7f4dc803b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:20:49.595 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.857+0000 7f4de8c00700 1 --2- 172.21.15.43:0/901388340 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7f4dc8038e00 0x7f4dc803b2c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0006fd0 tx=0x7f4dd0006e40 comp rx=0 tx=0).stop 2024-03-26T08:20:49.595 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.857+0000 7f4de8c00700 1 -- 172.21.15.43:0/901388340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4de4109910 msgr2=0x7f4de419d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:20:49.595 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.857+0000 7f4de8c00700 1 --2- 172.21.15.43:0/901388340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4de4109910 0x7f4de419d800 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4000c00 tx=0x7f4dd4011b00 comp rx=0 tx=0).stop 2024-03-26T08:20:49.595 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.858+0000 7f4de8c00700 1 -- 172.21.15.43:0/901388340 shutdown_connections 2024-03-26T08:20:49.595 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.858+0000 7f4de8c00700 1 --2- 172.21.15.43:0/901388340 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7f4dc8038e00 0x7f4dc803b2c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:20:49.595 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.858+0000 7f4de8c00700 1 --2- 172.21.15.43:0/901388340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4de4109910 0x7f4de419d800 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:20:49.595 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.858+0000 7f4de8c00700 1 -- 172.21.15.43:0/901388340 >> 172.21.15.43:0/901388340 conn(0x7f4de407ce20 msgr2=0x7f4de4106be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:20:49.595 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.858+0000 7f4de8c00700 1 -- 172.21.15.43:0/901388340 shutdown_connections 2024-03-26T08:20:49.595 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:48.858+0000 7f4de8c00700 1 -- 172.21.15.43:0/901388340 wait complete. 2024-03-26T08:20:49.595 INFO:teuthology.orchestra.run.smithi043.stdout:mgr is available 2024-03-26T08:20:49.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:49 smithi043 ceph-mon[31929]: mgrmap e4: x(active, since 2s) 2024-03-26T08:20:49.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:49 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/901388340' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T08:20:50.863 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:50 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/4226558233' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-26T08:20:51.178 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-03-26T08:20:51.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [global] 2024-03-26T08:20:51.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout fsid = 88759420-eb49-11ee-95d0-87774f69a715 2024-03-26T08:20:51.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-26T08:20:51.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-26T08:20:51.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-26T08:20:51.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-26T08:20:51.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-03-26T08:20:51.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-26T08:20:51.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-26T08:20:51.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-03-26T08:20:51.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [osd] 2024-03-26T08:20:51.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-26T08:20:51.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-26T08:20:51.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-26T08:20:51.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-26T08:20:51.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-26T08:20:51.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.216+0000 7f5951c00700 1 Processor -- start 2024-03-26T08:20:51.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.216+0000 7f5951c00700 1 -- start start 2024-03-26T08:20:51.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.216+0000 7f5951c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f594c1098f0 0x7f594c109d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:20:51.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.216+0000 7f5951c00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594c10a2e0 con 0x7f594c1098f0 2024-03-26T08:20:51.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.217+0000 7f594be00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f594c1098f0 0x7f594c109d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:20:51.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.217+0000 7f594be00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f594c1098f0 0x7f594c109d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49156/0 (socket says 172.21.15.43:49156) 2024-03-26T08:20:51.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.217+0000 7f594be00700 1 -- 172.21.15.43:0/3735199203 learned_addr learned my addr 172.21.15.43:0/3735199203 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:20:51.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.217+0000 7f594be00700 1 -- 172.21.15.43:0/3735199203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594c10aaa0 con 0x7f594c1098f0 2024-03-26T08:20:51.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.218+0000 7f594be00700 1 --2- 172.21.15.43:0/3735199203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f594c1098f0 0x7f594c109d10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f593c009e80 tx=0x7f593c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a00d6ce9262175 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:20:51.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.218+0000 7f594b400700 1 -- 172.21.15.43:0/3735199203 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f593c017070 con 0x7f594c1098f0 2024-03-26T08:20:51.181 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.218+0000 7f594b400700 1 -- 172.21.15.43:0/3735199203 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f593c00b8c0 con 0x7f594c1098f0 2024-03-26T08:20:51.181 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.218+0000 7f594b400700 1 -- 172.21.15.43:0/3735199203 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f593c004520 con 0x7f594c1098f0 2024-03-26T08:20:51.181 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.219+0000 7f5951c00700 1 -- 172.21.15.43:0/3735199203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f594c1098f0 msgr2=0x7f594c109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:20:51.181 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.219+0000 7f5951c00700 1 --2- 172.21.15.43:0/3735199203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f594c1098f0 0x7f594c109d10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f593c009e80 tx=0x7f593c00b330 comp rx=0 tx=0).stop 2024-03-26T08:20:51.181 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.219+0000 7f5951c00700 1 -- 172.21.15.43:0/3735199203 shutdown_connections 2024-03-26T08:20:51.181 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.219+0000 7f5951c00700 1 --2- 172.21.15.43:0/3735199203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f594c1098f0 0x7f594c109d10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:20:51.181 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.219+0000 7f5951c00700 1 -- 172.21.15.43:0/3735199203 >> 172.21.15.43:0/3735199203 conn(0x7f594c07ce20 msgr2=0x7f594c1074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:20:51.181 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.219+0000 7f5951c00700 1 -- 172.21.15.43:0/3735199203 shutdown_connections 2024-03-26T08:20:51.181 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.219+0000 7f5951c00700 1 -- 172.21.15.43:0/3735199203 wait complete. 2024-03-26T08:20:51.181 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.220+0000 7f5951c00700 1 Processor -- start 2024-03-26T08:20:51.181 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.220+0000 7f5951c00700 1 -- start start 2024-03-26T08:20:51.181 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.221+0000 7f5951c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f594c1098f0 0x7f594c19d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:20:51.181 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.221+0000 7f5951c00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594c10de50 con 0x7f594c1098f0 2024-03-26T08:20:51.181 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.221+0000 7f594be00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f594c1098f0 0x7f594c19d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:20:51.182 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.221+0000 7f594be00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f594c1098f0 0x7f594c19d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49160/0 (socket says 172.21.15.43:49160) 2024-03-26T08:20:51.182 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.221+0000 7f594be00700 1 -- 172.21.15.43:0/4226558233 learned_addr learned my addr 172.21.15.43:0/4226558233 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:20:51.182 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.221+0000 7f594be00700 1 -- 172.21.15.43:0/4226558233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594c19dd10 con 0x7f594c1098f0 2024-03-26T08:20:51.182 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.221+0000 7f594be00700 1 --2- 172.21.15.43:0/4226558233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f594c1098f0 0x7f594c19d7d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f593c000f80 tx=0x7f593c009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:20:51.182 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.222+0000 7f5949600700 1 -- 172.21.15.43:0/4226558233 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f593c020020 con 0x7f594c1098f0 2024-03-26T08:20:51.182 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.222+0000 7f5949600700 1 -- 172.21.15.43:0/4226558233 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f593c00bba0 con 0x7f594c1098f0 2024-03-26T08:20:51.182 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.222+0000 7f5949600700 1 -- 172.21.15.43:0/4226558233 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f593c017040 con 0x7f594c1098f0 2024-03-26T08:20:51.182 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.222+0000 7f5951c00700 1 -- 172.21.15.43:0/4226558233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f594c19dfa0 con 0x7f594c1098f0 2024-03-26T08:20:51.182 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.222+0000 7f5951c00700 1 -- 172.21.15.43:0/4226558233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f594c1a0c90 con 0x7f594c1098f0 2024-03-26T08:20:51.182 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.223+0000 7f5949600700 1 -- 172.21.15.43:0/4226558233 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7f593c023020 con 0x7f594c1098f0 2024-03-26T08:20:51.182 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.223+0000 7f5951c00700 1 -- 172.21.15.43:0/4226558233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f594c0637c0 con 0x7f594c1098f0 2024-03-26T08:20:51.182 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.224+0000 7f5949600700 1 --2- 172.21.15.43:0/4226558233 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7f5930038f20 0x7f593003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:20:51.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.224+0000 7f5949600700 1 -- 172.21.15.43:0/4226558233 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f593c00f050 con 0x7f594c1098f0 2024-03-26T08:20:51.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.224+0000 7f5943400700 1 --2- 172.21.15.43:0/4226558233 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7f5930038f20 0x7f593003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:20:51.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.225+0000 7f5943400700 1 --2- 172.21.15.43:0/4226558233 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7f5930038f20 0x7f593003b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5934006fd0 tx=0x7f5934006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:20:51.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.229+0000 7f5949600700 1 -- 172.21.15.43:0/4226558233 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f593c02a020 con 0x7f594c1098f0 2024-03-26T08:20:51.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.452+0000 7f5951c00700 1 -- 172.21.15.43:0/4226558233 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f594c1a2500 con 0x7f594c1098f0 2024-03-26T08:20:51.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.454+0000 7f5949600700 1 -- 172.21.15.43:0/4226558233 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f593c02c300 con 0x7f594c1098f0 2024-03-26T08:20:51.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.457+0000 7f5951c00700 1 -- 172.21.15.43:0/4226558233 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7f5930038f20 msgr2=0x7f593003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:20:51.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.457+0000 7f5951c00700 1 --2- 172.21.15.43:0/4226558233 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7f5930038f20 0x7f593003b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5934006fd0 tx=0x7f5934006e40 comp rx=0 tx=0).stop 2024-03-26T08:20:51.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.457+0000 7f5951c00700 1 -- 172.21.15.43:0/4226558233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f594c1098f0 msgr2=0x7f594c19d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:20:51.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.457+0000 7f5951c00700 1 --2- 172.21.15.43:0/4226558233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f594c1098f0 0x7f594c19d7d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f593c000f80 tx=0x7f593c009640 comp rx=0 tx=0).stop 2024-03-26T08:20:51.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.457+0000 7f5951c00700 1 -- 172.21.15.43:0/4226558233 shutdown_connections 2024-03-26T08:20:51.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.457+0000 7f5951c00700 1 --2- 172.21.15.43:0/4226558233 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7f5930038f20 0x7f593003b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:20:51.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.457+0000 7f5951c00700 1 --2- 172.21.15.43:0/4226558233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f594c1098f0 0x7f594c19d7d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:20:51.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.457+0000 7f5951c00700 1 -- 172.21.15.43:0/4226558233 >> 172.21.15.43:0/4226558233 conn(0x7f594c07ce20 msgr2=0x7f594c106b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:20:51.184 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.457+0000 7f5951c00700 1 -- 172.21.15.43:0/4226558233 shutdown_connections 2024-03-26T08:20:51.184 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:50.457+0000 7f5951c00700 1 -- 172.21.15.43:0/4226558233 wait complete. 2024-03-26T08:20:51.184 INFO:teuthology.orchestra.run.smithi043.stdout:Enabling cephadm module... 2024-03-26T08:20:52.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:52 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1765116995' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-26T08:20:52.404 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:52 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: ignoring --setuser ceph since I am not root 2024-03-26T08:20:52.404 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:52 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: ignoring --setgroup ceph since I am not root 2024-03-26T08:20:52.762 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.813+0000 7f8937200700 1 Processor -- start 2024-03-26T08:20:52.762 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.813+0000 7f8937200700 1 -- start start 2024-03-26T08:20:52.762 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.814+0000 7f8937200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8938108170 0x7f893810a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:20:52.762 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.814+0000 7f8937200700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893807ae60 con 0x7f8938108170 2024-03-26T08:20:52.762 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.814+0000 7f8935e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8938108170 0x7f893810a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:20:52.762 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.814+0000 7f8935e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8938108170 0x7f893810a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49170/0 (socket says 172.21.15.43:49170) 2024-03-26T08:20:52.762 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.814+0000 7f8935e00700 1 -- 172.21.15.43:0/3117195730 learned_addr learned my addr 172.21.15.43:0/3117195730 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:20:52.762 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.815+0000 7f8935e00700 1 -- 172.21.15.43:0/3117195730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893810aaa0 con 0x7f8938108170 2024-03-26T08:20:52.762 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.815+0000 7f8935e00700 1 --2- 172.21.15.43:0/3117195730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8938108170 0x7f893810a560 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f89280098a0 tx=0x7f8928009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91bed77bc9711ee3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:20:52.763 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.815+0000 7f8934a00700 1 -- 172.21.15.43:0/3117195730 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8928004160 con 0x7f8938108170 2024-03-26T08:20:52.763 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.815+0000 7f8934a00700 1 -- 172.21.15.43:0/3117195730 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f8928004300 con 0x7f8938108170 2024-03-26T08:20:52.763 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.815+0000 7f8934a00700 1 -- 172.21.15.43:0/3117195730 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8928011a80 con 0x7f8938108170 2024-03-26T08:20:52.763 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.816+0000 7f8937200700 1 -- 172.21.15.43:0/3117195730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8938108170 msgr2=0x7f893810a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:20:52.763 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.816+0000 7f8937200700 1 --2- 172.21.15.43:0/3117195730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8938108170 0x7f893810a560 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f89280098a0 tx=0x7f8928009bb0 comp rx=0 tx=0).stop 2024-03-26T08:20:52.763 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.816+0000 7f8937200700 1 -- 172.21.15.43:0/3117195730 shutdown_connections 2024-03-26T08:20:52.763 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.816+0000 7f8937200700 1 --2- 172.21.15.43:0/3117195730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8938108170 0x7f893810a560 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:20:52.763 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.816+0000 7f8937200700 1 -- 172.21.15.43:0/3117195730 >> 172.21.15.43:0/3117195730 conn(0x7f8938101ba0 msgr2=0x7f8938103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:20:52.763 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.816+0000 7f8937200700 1 -- 172.21.15.43:0/3117195730 shutdown_connections 2024-03-26T08:20:52.763 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.816+0000 7f8937200700 1 -- 172.21.15.43:0/3117195730 wait complete. 2024-03-26T08:20:52.763 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.817+0000 7f8937200700 1 Processor -- start 2024-03-26T08:20:52.763 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.817+0000 7f8937200700 1 -- start start 2024-03-26T08:20:52.763 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.818+0000 7f8937200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8938108170 0x7f89381079f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:20:52.764 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.818+0000 7f8937200700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893810ba10 con 0x7f8938108170 2024-03-26T08:20:52.764 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.818+0000 7f8935e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8938108170 0x7f89381079f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:20:52.764 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.818+0000 7f8935e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8938108170 0x7f89381079f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49184/0 (socket says 172.21.15.43:49184) 2024-03-26T08:20:52.764 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.818+0000 7f8935e00700 1 -- 172.21.15.43:0/1765116995 learned_addr learned my addr 172.21.15.43:0/1765116995 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:20:52.764 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.818+0000 7f8935e00700 1 -- 172.21.15.43:0/1765116995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8938107f30 con 0x7f8938108170 2024-03-26T08:20:52.764 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.819+0000 7f8935e00700 1 --2- 172.21.15.43:0/1765116995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8938108170 0x7f89381079f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8928006af0 tx=0x7f8928011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:20:52.764 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.819+0000 7f892ea00700 1 -- 172.21.15.43:0/1765116995 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8928003930 con 0x7f8938108170 2024-03-26T08:20:52.764 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.819+0000 7f892ea00700 1 -- 172.21.15.43:0/1765116995 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f8928003ad0 con 0x7f8938108170 2024-03-26T08:20:52.764 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.819+0000 7f8937200700 1 -- 172.21.15.43:0/1765116995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8938106120 con 0x7f8938108170 2024-03-26T08:20:52.764 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.819+0000 7f892ea00700 1 -- 172.21.15.43:0/1765116995 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f89280195e0 con 0x7f8938108170 2024-03-26T08:20:52.764 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.819+0000 7f8937200700 1 -- 172.21.15.43:0/1765116995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8938106660 con 0x7f8938108170 2024-03-26T08:20:52.764 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.820+0000 7f892ea00700 1 -- 172.21.15.43:0/1765116995 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7f8928019890 con 0x7f8938108170 2024-03-26T08:20:52.764 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.820+0000 7f8937200700 1 -- 172.21.15.43:0/1765116995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89380637c0 con 0x7f8938108170 2024-03-26T08:20:52.765 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.821+0000 7f892ea00700 1 --2- 172.21.15.43:0/1765116995 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7f891c038ed0 0x7f891c03b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:20:52.765 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.821+0000 7f892ea00700 1 -- 172.21.15.43:0/1765116995 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f892804d300 con 0x7f8938108170 2024-03-26T08:20:52.765 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.821+0000 7f8935400700 1 --2- 172.21.15.43:0/1765116995 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7f891c038ed0 0x7f891c03b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:20:52.765 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.822+0000 7f8935400700 1 --2- 172.21.15.43:0/1765116995 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7f891c038ed0 0x7f891c03b390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8924006fd0 tx=0x7f8924006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:20:52.765 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:51.827+0000 7f892ea00700 1 -- 172.21.15.43:0/1765116995 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8928023e00 con 0x7f8938108170 2024-03-26T08:20:52.765 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:52.112+0000 7f8937200700 1 -- 172.21.15.43:0/1765116995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f89381068e0 con 0x7f8938108170 2024-03-26T08:20:52.765 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:52.166+0000 7f892ea00700 1 -- 172.21.15.43:0/1765116995 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f892804cd20 con 0x7f8938108170 2024-03-26T08:20:52.765 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:52.169+0000 7f8937200700 1 -- 172.21.15.43:0/1765116995 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7f891c038ed0 msgr2=0x7f891c03b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:20:52.765 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:52.169+0000 7f8937200700 1 --2- 172.21.15.43:0/1765116995 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7f891c038ed0 0x7f891c03b390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8924006fd0 tx=0x7f8924006e40 comp rx=0 tx=0).stop 2024-03-26T08:20:52.765 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:52.169+0000 7f8937200700 1 -- 172.21.15.43:0/1765116995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8938108170 msgr2=0x7f89381079f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:20:52.765 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:52.170+0000 7f892ea00700 1 -- 172.21.15.43:0/1765116995 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f892801a430 con 0x7f8938108170 2024-03-26T08:20:52.765 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:52.170+0000 7f8937200700 1 --2- 172.21.15.43:0/1765116995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8938108170 0x7f89381079f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8928006af0 tx=0x7f8928011b70 comp rx=0 tx=0).stop 2024-03-26T08:20:52.765 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:52.170+0000 7f8937200700 1 -- 172.21.15.43:0/1765116995 shutdown_connections 2024-03-26T08:20:52.765 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:52.170+0000 7f8937200700 1 --2- 172.21.15.43:0/1765116995 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7f891c038ed0 0x7f891c03b390 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:20:52.766 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:52.170+0000 7f8937200700 1 --2- 172.21.15.43:0/1765116995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8938108170 0x7f89381079f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:20:52.766 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:52.170+0000 7f8937200700 1 -- 172.21.15.43:0/1765116995 >> 172.21.15.43:0/1765116995 conn(0x7f8938101ba0 msgr2=0x7f8938102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:20:52.766 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:52.170+0000 7f8937200700 1 -- 172.21.15.43:0/1765116995 shutdown_connections 2024-03-26T08:20:52.766 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:52.171+0000 7f8937200700 1 -- 172.21.15.43:0/1765116995 wait complete. 2024-03-26T08:20:53.116 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:52 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:52.994+0000 7faa9bb8c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T08:20:53.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:53 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1765116995' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-26T08:20:53.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:53 smithi043 ceph-mon[31929]: mgrmap e5: x(active, since 5s) 2024-03-26T08:20:53.385 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:53 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:53.125+0000 7faa9bb8c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T08:20:54.265 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:20:54 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/778366468' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-26T08:20:54.265 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:53 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:53.958+0000 7faa9bb8c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T08:20:54.265 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:54 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:54.086+0000 7faa9bb8c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T08:20:54.265 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:54 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:54.268+0000 7faa9bb8c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T08:20:54.518 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:54 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:54.397+0000 7faa9bb8c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T08:20:54.518 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:54 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:54.511+0000 7faa9bb8c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T08:20:54.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-03-26T08:20:54.522 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-26T08:20:54.522 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-26T08:20:54.522 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-26T08:20:54.522 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-26T08:20:54.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-03-26T08:20:54.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.453+0000 7f8aaaa00700 1 Processor -- start 2024-03-26T08:20:54.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.453+0000 7f8aaaa00700 1 -- start start 2024-03-26T08:20:54.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.453+0000 7f8aaaa00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aa4109ac0 0x7f8aa4109ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:20:54.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.453+0000 7f8aaaa00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa410a4b0 con 0x7f8aa4109ac0 2024-03-26T08:20:54.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.454+0000 7f8aa3e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aa4109ac0 0x7f8aa4109ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:20:54.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.454+0000 7f8aa3e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aa4109ac0 0x7f8aa4109ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49208/0 (socket says 172.21.15.43:49208) 2024-03-26T08:20:54.525 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.454+0000 7f8aa3e00700 1 -- 172.21.15.43:0/2833744050 learned_addr learned my addr 172.21.15.43:0/2833744050 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:20:54.525 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.454+0000 7f8aa3e00700 1 -- 172.21.15.43:0/2833744050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa410ac70 con 0x7f8aa4109ac0 2024-03-26T08:20:54.525 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.454+0000 7f8aa3e00700 1 --2- 172.21.15.43:0/2833744050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aa4109ac0 0x7f8aa4109ee0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8a940098a0 tx=0x7f8a94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1bf575f8b0887be server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:20:54.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.455+0000 7f8aa2a00700 1 -- 172.21.15.43:0/2833744050 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a94004160 con 0x7f8aa4109ac0 2024-03-26T08:20:54.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.455+0000 7f8aa2a00700 1 -- 172.21.15.43:0/2833744050 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f8a94004300 con 0x7f8aa4109ac0 2024-03-26T08:20:54.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.455+0000 7f8aa2a00700 1 -- 172.21.15.43:0/2833744050 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a94011a80 con 0x7f8aa4109ac0 2024-03-26T08:20:54.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.455+0000 7f8aaaa00700 1 -- 172.21.15.43:0/2833744050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aa4109ac0 msgr2=0x7f8aa4109ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:20:54.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.455+0000 7f8aaaa00700 1 --2- 172.21.15.43:0/2833744050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aa4109ac0 0x7f8aa4109ee0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8a940098a0 tx=0x7f8a94009bb0 comp rx=0 tx=0).stop 2024-03-26T08:20:54.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.455+0000 7f8aaaa00700 1 -- 172.21.15.43:0/2833744050 shutdown_connections 2024-03-26T08:20:54.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.455+0000 7f8aaaa00700 1 --2- 172.21.15.43:0/2833744050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aa4109ac0 0x7f8aa4109ee0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:20:54.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.455+0000 7f8aaaa00700 1 -- 172.21.15.43:0/2833744050 >> 172.21.15.43:0/2833744050 conn(0x7f8aa41054d0 msgr2=0x7f8aa41078c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:20:54.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.456+0000 7f8aaaa00700 1 -- 172.21.15.43:0/2833744050 shutdown_connections 2024-03-26T08:20:54.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.456+0000 7f8aaaa00700 1 -- 172.21.15.43:0/2833744050 wait complete. 2024-03-26T08:20:54.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.456+0000 7f8aaaa00700 1 Processor -- start 2024-03-26T08:20:54.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.457+0000 7f8aaaa00700 1 -- start start 2024-03-26T08:20:54.526 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.457+0000 7f8aaaa00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aa4109ac0 0x7f8aa419d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:20:54.527 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.457+0000 7f8aaaa00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa410e020 con 0x7f8aa4109ac0 2024-03-26T08:20:54.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.457+0000 7f8aa3e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aa4109ac0 0x7f8aa419d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:20:54.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.457+0000 7f8aa3e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aa4109ac0 0x7f8aa419d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49216/0 (socket says 172.21.15.43:49216) 2024-03-26T08:20:54.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.457+0000 7f8aa3e00700 1 -- 172.21.15.43:0/778366468 learned_addr learned my addr 172.21.15.43:0/778366468 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:20:54.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.458+0000 7f8aa3e00700 1 -- 172.21.15.43:0/778366468 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa419ded0 con 0x7f8aa4109ac0 2024-03-26T08:20:54.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.458+0000 7f8aa3e00700 1 --2- 172.21.15.43:0/778366468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aa4109ac0 0x7f8aa419d990 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8a94000c00 tx=0x7f8a94011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:20:54.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.458+0000 7f8aa0c00700 1 -- 172.21.15.43:0/778366468 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a940039f0 con 0x7f8aa4109ac0 2024-03-26T08:20:54.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.458+0000 7f8aa0c00700 1 -- 172.21.15.43:0/778366468 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f8a94003b90 con 0x7f8aa4109ac0 2024-03-26T08:20:54.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.458+0000 7f8aa0c00700 1 -- 172.21.15.43:0/778366468 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a9401a740 con 0x7f8aa4109ac0 2024-03-26T08:20:54.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.458+0000 7f8aaaa00700 1 -- 172.21.15.43:0/778366468 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa419e160 con 0x7f8aa4109ac0 2024-03-26T08:20:54.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.458+0000 7f8aaaa00700 1 -- 172.21.15.43:0/778366468 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa41a0e50 con 0x7f8aa4109ac0 2024-03-26T08:20:54.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.459+0000 7f8aa0c00700 1 -- 172.21.15.43:0/778366468 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f8a94024440 con 0x7f8aa4109ac0 2024-03-26T08:20:54.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.460+0000 7f8aaaa00700 1 -- 172.21.15.43:0/778366468 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aa40637c0 con 0x7f8aa4109ac0 2024-03-26T08:20:54.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.460+0000 7f8aa0c00700 1 --2- 172.21.15.43:0/778366468 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7f8a88038f70 0x7f8a8803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:20:54.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.460+0000 7f8aa0c00700 1 -- 172.21.15.43:0/778366468 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8a9404d120 con 0x7f8aa4109ac0 2024-03-26T08:20:54.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.460+0000 7f8aa3400700 1 -- 172.21.15.43:0/778366468 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7f8a88038f70 msgr2=0x7f8a8803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/4108777098 2024-03-26T08:20:54.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.460+0000 7f8aa3400700 1 --2- 172.21.15.43:0/778366468 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7f8a88038f70 0x7f8a8803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T08:20:54.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.465+0000 7f8aa0c00700 1 -- 172.21.15.43:0/778366468 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8a9404cb40 con 0x7f8aa4109ac0 2024-03-26T08:20:54.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.661+0000 7f8aa3400700 1 -- 172.21.15.43:0/778366468 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7f8a88038f70 msgr2=0x7f8a8803b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/4108777098 2024-03-26T08:20:54.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.661+0000 7f8aa3400700 1 --2- 172.21.15.43:0/778366468 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7f8a88038f70 0x7f8a8803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T08:20:54.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.757+0000 7f8aaaa00700 1 -- 172.21.15.43:0/778366468 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f8aa41a1020 con 0x7f8aa4109ac0 2024-03-26T08:20:54.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.758+0000 7f8aa0c00700 1 -- 172.21.15.43:0/778366468 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f8aa41a1020 con 0x7f8aa4109ac0 2024-03-26T08:20:54.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.761+0000 7f8aaaa00700 1 -- 172.21.15.43:0/778366468 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7f8a88038f70 msgr2=0x7f8a8803b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T08:20:54.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.761+0000 7f8aaaa00700 1 --2- 172.21.15.43:0/778366468 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7f8a88038f70 0x7f8a8803b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:20:54.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.761+0000 7f8aaaa00700 1 -- 172.21.15.43:0/778366468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aa4109ac0 msgr2=0x7f8aa419d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:20:54.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.761+0000 7f8aaaa00700 1 --2- 172.21.15.43:0/778366468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aa4109ac0 0x7f8aa419d990 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8a94000c00 tx=0x7f8a94011b00 comp rx=0 tx=0).stop 2024-03-26T08:20:54.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.761+0000 7f8aaaa00700 1 -- 172.21.15.43:0/778366468 shutdown_connections 2024-03-26T08:20:54.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.761+0000 7f8aaaa00700 1 --2- 172.21.15.43:0/778366468 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7f8a88038f70 0x7f8a8803b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:20:54.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.761+0000 7f8aaaa00700 1 --2- 172.21.15.43:0/778366468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aa4109ac0 0x7f8aa419d990 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:20:54.532 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.761+0000 7f8aaaa00700 1 -- 172.21.15.43:0/778366468 >> 172.21.15.43:0/778366468 conn(0x7f8aa41054d0 msgr2=0x7f8aa4106f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:20:54.532 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.761+0000 7f8aaaa00700 1 -- 172.21.15.43:0/778366468 shutdown_connections 2024-03-26T08:20:54.532 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:53.761+0000 7f8aaaa00700 1 -- 172.21.15.43:0/778366468 wait complete. 2024-03-26T08:20:54.532 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for the mgr to restart... 2024-03-26T08:20:54.532 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr epoch 5... 2024-03-26T08:20:55.050 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:54 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:54.865+0000 7faa9bb8c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T08:20:55.050 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:55 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:55.013+0000 7faa9bb8c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T08:20:55.327 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:55 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:55.134+0000 7faa9bb8c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T08:20:55.327 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:55 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:55.260+0000 7faa9bb8c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T08:20:55.654 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:55 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:55.483+0000 7faa9bb8c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T08:20:57.154 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:56 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:56.790+0000 7faa9bb8c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T08:20:57.154 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:56 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:56.965+0000 7faa9bb8c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T08:20:57.654 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:57 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:57.391+0000 7faa9bb8c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T08:20:57.654 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:57 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:57.512+0000 7faa9bb8c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T08:20:58.024 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:57 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:57.776+0000 7faa9bb8c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T08:20:58.404 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:58 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:58.027+0000 7faa9bb8c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T08:20:58.731 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:58 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:58.412+0000 7faa9bb8c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T08:20:58.731 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:58 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:58.735+0000 7faa9bb8c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T08:20:59.404 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:20:59 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:20:59.047+0000 7faa9bb8c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T08:21:00.405 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:00 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:00.105+0000 7faa9bb8c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T08:21:01.154 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:00 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:00.820+0000 7faa9bb8c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T08:21:01.154 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:00 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:00.956+0000 7faa9bb8c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T08:21:01.654 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:01 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:01.252+0000 7faa9bb8c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T08:21:01.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:01 smithi043 ceph-mon[31929]: Active manager daemon x restarted 2024-03-26T08:21:01.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:01 smithi043 ceph-mon[31929]: Activating manager daemon x 2024-03-26T08:21:01.655 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:01 smithi043 ceph-mon[31929]: osdmap e2: 0 total, 0 up, 0 in 2024-03-26T08:21:01.655 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:01 smithi043 ceph-mon[31929]: mgrmap e6: x(active, starting, since 0.00886632s) 2024-03-26T08:21:01.655 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:01 smithi043 ceph-mon[31929]: from='mgr.14120 172.21.15.43:0/1789936270' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:21:01.655 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:01 smithi043 ceph-mon[31929]: from='mgr.14120 172.21.15.43:0/1789936270' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T08:21:01.655 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:01 smithi043 ceph-mon[31929]: from='mgr.14120 172.21.15.43:0/1789936270' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T08:21:01.655 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:01 smithi043 ceph-mon[31929]: from='mgr.14120 172.21.15.43:0/1789936270' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T08:21:01.655 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:01 smithi043 ceph-mon[31929]: from='mgr.14120 172.21.15.43:0/1789936270' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T08:21:01.655 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:01 smithi043 ceph-mon[31929]: Manager daemon x is now available 2024-03-26T08:21:02.634 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:02 smithi043 ceph-mon[31929]: Found migration_current of "None". Setting to last migration. 2024-03-26T08:21:02.634 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:02 smithi043 ceph-mon[31929]: from='mgr.14120 172.21.15.43:0/1789936270' entity='mgr.x' 2024-03-26T08:21:02.634 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:02 smithi043 ceph-mon[31929]: from='mgr.14120 172.21.15.43:0/1789936270' entity='mgr.x' 2024-03-26T08:21:02.634 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:02 smithi043 ceph-mon[31929]: from='mgr.14120 172.21.15.43:0/1789936270' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:21:02.634 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:02 smithi043 ceph-mon[31929]: from='mgr.14120 172.21.15.43:0/1789936270' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:21:02.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:02 smithi043 ceph-mon[31929]: from='mgr.14120 172.21.15.43:0/1789936270' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:21:02.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:02 smithi043 ceph-mon[31929]: from='mgr.14120 172.21.15.43:0/1789936270' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-26T08:21:02.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:02 smithi043 ceph-mon[31929]: from='mgr.14120 172.21.15.43:0/1789936270' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-26T08:21:02.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:02 smithi043 ceph-mon[31929]: from='mgr.14120 172.21.15.43:0/1789936270' entity='mgr.x' 2024-03-26T08:21:02.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:02 smithi043 ceph-mon[31929]: from='mgr.14120 172.21.15.43:0/1789936270' entity='mgr.x' 2024-03-26T08:21:02.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:02 smithi043 ceph-mon[31929]: mgrmap e7: x(active, since 1.01674s) 2024-03-26T08:21:02.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:02 smithi043 ceph-mon[31929]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-26T08:21:02.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:02 smithi043 ceph-mon[31929]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-26T08:21:02.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-03-26T08:21:02.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-26T08:21:02.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-26T08:21:02.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-03-26T08:21:02.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.121+0000 7fc618200700 1 Processor -- start 2024-03-26T08:21:02.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.121+0000 7fc618200700 1 -- start start 2024-03-26T08:21:02.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.121+0000 7fc618200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc61007ce60 0x7fc61007b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:02.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.121+0000 7fc618200700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61007b8e0 con 0x7fc61007ce60 2024-03-26T08:21:02.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.122+0000 7fc615800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc61007ce60 0x7fc61007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:02.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.122+0000 7fc615800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc61007ce60 0x7fc61007b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33532/0 (socket says 172.21.15.43:33532) 2024-03-26T08:21:02.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.122+0000 7fc615800700 1 -- 172.21.15.43:0/3240512894 learned_addr learned my addr 172.21.15.43:0/3240512894 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:02.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.122+0000 7fc615800700 1 -- 172.21.15.43:0/3240512894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc61007ba60 con 0x7fc61007ce60 2024-03-26T08:21:02.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.123+0000 7fc615800700 1 --2- 172.21.15.43:0/3240512894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc61007ce60 0x7fc61007b310 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc6040098a0 tx=0x7fc604009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5ec92cc0589b1f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:02.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.123+0000 7fc60fe00700 1 -- 172.21.15.43:0/3240512894 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc604004160 con 0x7fc61007ce60 2024-03-26T08:21:02.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.123+0000 7fc60fe00700 1 -- 172.21.15.43:0/3240512894 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fc604004300 con 0x7fc61007ce60 2024-03-26T08:21:02.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.123+0000 7fc60fe00700 1 -- 172.21.15.43:0/3240512894 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc604011a80 con 0x7fc61007ce60 2024-03-26T08:21:02.985 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.124+0000 7fc618200700 1 -- 172.21.15.43:0/3240512894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc61007ce60 msgr2=0x7fc61007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:02.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.124+0000 7fc618200700 1 --2- 172.21.15.43:0/3240512894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc61007ce60 0x7fc61007b310 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc6040098a0 tx=0x7fc604009bb0 comp rx=0 tx=0).stop 2024-03-26T08:21:02.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.124+0000 7fc618200700 1 -- 172.21.15.43:0/3240512894 shutdown_connections 2024-03-26T08:21:02.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.124+0000 7fc618200700 1 --2- 172.21.15.43:0/3240512894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc61007ce60 0x7fc61007b310 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:02.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.124+0000 7fc618200700 1 -- 172.21.15.43:0/3240512894 >> 172.21.15.43:0/3240512894 conn(0x7fc610102f70 msgr2=0x7fc610105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:02.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.124+0000 7fc618200700 1 -- 172.21.15.43:0/3240512894 shutdown_connections 2024-03-26T08:21:02.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.124+0000 7fc618200700 1 -- 172.21.15.43:0/3240512894 wait complete. 2024-03-26T08:21:02.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.125+0000 7fc618200700 1 Processor -- start 2024-03-26T08:21:02.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.126+0000 7fc618200700 1 -- start start 2024-03-26T08:21:02.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.126+0000 7fc618200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc61007ce60 0x7fc61019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:02.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.126+0000 7fc618200700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61010de20 con 0x7fc61007ce60 2024-03-26T08:21:02.986 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.126+0000 7fc615800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc61007ce60 0x7fc61019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:02.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.126+0000 7fc615800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc61007ce60 0x7fc61019d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33542/0 (socket says 172.21.15.43:33542) 2024-03-26T08:21:02.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.126+0000 7fc615800700 1 -- 172.21.15.43:0/578444227 learned_addr learned my addr 172.21.15.43:0/578444227 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:02.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.127+0000 7fc615800700 1 -- 172.21.15.43:0/578444227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc61019deb0 con 0x7fc61007ce60 2024-03-26T08:21:02.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.127+0000 7fc615800700 1 --2- 172.21.15.43:0/578444227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc61007ce60 0x7fc61019d970 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc604011a50 tx=0x7fc604011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:02.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.127+0000 7fc60e000700 1 -- 172.21.15.43:0/578444227 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc604003a80 con 0x7fc61007ce60 2024-03-26T08:21:02.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.128+0000 7fc618200700 1 -- 172.21.15.43:0/578444227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc61019e140 con 0x7fc61007ce60 2024-03-26T08:21:02.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.128+0000 7fc60e000700 1 -- 172.21.15.43:0/578444227 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fc604003c20 con 0x7fc61007ce60 2024-03-26T08:21:02.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.128+0000 7fc60e000700 1 -- 172.21.15.43:0/578444227 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc60401a650 con 0x7fc61007ce60 2024-03-26T08:21:02.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.128+0000 7fc618200700 1 -- 172.21.15.43:0/578444227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6101a0e30 con 0x7fc61007ce60 2024-03-26T08:21:02.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.129+0000 7fc60e000700 1 -- 172.21.15.43:0/578444227 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fc604024ad0 con 0x7fc61007ce60 2024-03-26T08:21:02.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.130+0000 7fc60e000700 1 --2- 172.21.15.43:0/578444227 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7fc5f8038f20 0x7fc5f803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:02.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.130+0000 7fc60e000700 1 -- 172.21.15.43:0/578444227 --> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5f803baf0 con 0x7fc5f8038f20 2024-03-26T08:21:02.987 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.130+0000 7fc60e000700 1 -- 172.21.15.43:0/578444227 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc60404d360 con 0x7fc61007ce60 2024-03-26T08:21:02.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.130+0000 7fc614e00700 1 -- 172.21.15.43:0/578444227 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7fc5f8038f20 msgr2=0x7fc5f803b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/4108777098 2024-03-26T08:21:02.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.130+0000 7fc614e00700 1 --2- 172.21.15.43:0/578444227 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7fc5f8038f20 0x7fc5f803b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T08:21:02.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.331+0000 7fc614e00700 1 -- 172.21.15.43:0/578444227 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7fc5f8038f20 msgr2=0x7fc5f803b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/4108777098 2024-03-26T08:21:02.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.331+0000 7fc614e00700 1 --2- 172.21.15.43:0/578444227 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7fc5f8038f20 0x7fc5f803b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T08:21:02.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.732+0000 7fc614e00700 1 -- 172.21.15.43:0/578444227 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7fc5f8038f20 msgr2=0x7fc5f803b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/4108777098 2024-03-26T08:21:02.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:55.732+0000 7fc614e00700 1 --2- 172.21.15.43:0/578444227 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7fc5f8038f20 0x7fc5f803b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-26T08:21:02.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:56.533+0000 7fc614e00700 1 -- 172.21.15.43:0/578444227 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7fc5f8038f20 msgr2=0x7fc5f803b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/4108777098 2024-03-26T08:21:02.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:56.533+0000 7fc614e00700 1 --2- 172.21.15.43:0/578444227 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7fc5f8038f20 0x7fc5f803b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-26T08:21:02.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:58.135+0000 7fc614e00700 1 -- 172.21.15.43:0/578444227 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7fc5f8038f20 msgr2=0x7fc5f803b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/4108777098 2024-03-26T08:21:02.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:20:58.135+0000 7fc614e00700 1 --2- 172.21.15.43:0/578444227 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7fc5f8038f20 0x7fc5f803b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-26T08:21:02.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:01.260+0000 7fc60e000700 1 -- 172.21.15.43:0/578444227 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mgrmap(e 6) v1 ==== 45873+0+0 (secure 0 0 0) 0x7fc6040247c0 con 0x7fc61007ce60 2024-03-26T08:21:02.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:01.260+0000 7fc60e000700 1 -- 172.21.15.43:0/578444227 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7fc5f8038f20 msgr2=0x7fc5f803b3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T08:21:02.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:01.260+0000 7fc60e000700 1 --2- 172.21.15.43:0/578444227 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7fc5f8038f20 0x7fc5f803b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:02.988 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:02.266+0000 7fc60e000700 1 -- 172.21.15.43:0/578444227 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 7) v1 ==== 46000+0+0 (secure 0 0 0) 0x7fc60404f550 con 0x7fc61007ce60 2024-03-26T08:21:02.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:02.267+0000 7fc60e000700 1 --2- 172.21.15.43:0/578444227 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7fc5f803c810 0x7fc5f803ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:02.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:02.267+0000 7fc60e000700 1 -- 172.21.15.43:0/578444227 --> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5f803baf0 con 0x7fc5f803c810 2024-03-26T08:21:02.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:02.268+0000 7fc614e00700 1 --2- 172.21.15.43:0/578444227 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7fc5f803c810 0x7fc5f803ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:02.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:02.272+0000 7fc614e00700 1 --2- 172.21.15.43:0/578444227 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7fc5f803c810 0x7fc5f803ec00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc003a10 tx=0x7fc5fc009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:02.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:02.274+0000 7fc60e000700 1 -- 172.21.15.43:0/578444227 <== mgr.14120 v2:172.21.15.43:6800/2446912488 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fc5f803baf0 con 0x7fc5f803c810 2024-03-26T08:21:02.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:02.282+0000 7fc618200700 1 -- 172.21.15.43:0/578444227 --> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fc6100637c0 con 0x7fc5f803c810 2024-03-26T08:21:02.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:02.282+0000 7fc60e000700 1 -- 172.21.15.43:0/578444227 <== mgr.14120 v2:172.21.15.43:6800/2446912488 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fc6100637c0 con 0x7fc5f803c810 2024-03-26T08:21:02.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:02.283+0000 7fc618200700 1 -- 172.21.15.43:0/578444227 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7fc5f803c810 msgr2=0x7fc5f803ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:02.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:02.283+0000 7fc618200700 1 --2- 172.21.15.43:0/578444227 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7fc5f803c810 0x7fc5f803ec00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc003a10 tx=0x7fc5fc009290 comp rx=0 tx=0).stop 2024-03-26T08:21:02.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:02.283+0000 7fc618200700 1 -- 172.21.15.43:0/578444227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc61007ce60 msgr2=0x7fc61019d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:02.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:02.283+0000 7fc618200700 1 --2- 172.21.15.43:0/578444227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc61007ce60 0x7fc61019d970 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc604011a50 tx=0x7fc604011b30 comp rx=0 tx=0).stop 2024-03-26T08:21:02.989 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:02.283+0000 7fc618200700 1 -- 172.21.15.43:0/578444227 shutdown_connections 2024-03-26T08:21:02.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:02.283+0000 7fc618200700 1 --2- 172.21.15.43:0/578444227 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7fc5f803c810 0x7fc5f803ec00 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:02.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:02.283+0000 7fc618200700 1 --2- 172.21.15.43:0/578444227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc61007ce60 0x7fc61019d970 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:02.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:02.283+0000 7fc618200700 1 --2- 172.21.15.43:0/578444227 >> [v2:172.21.15.43:6800/4108777098,v1:172.21.15.43:6801/4108777098] conn(0x7fc5f8038f20 0x7fc5f803b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:02.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:02.283+0000 7fc618200700 1 -- 172.21.15.43:0/578444227 >> 172.21.15.43:0/578444227 conn(0x7fc610102f70 msgr2=0x7fc610103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:02.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:02.283+0000 7fc618200700 1 -- 172.21.15.43:0/578444227 shutdown_connections 2024-03-26T08:21:02.990 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:02.283+0000 7fc618200700 1 -- 172.21.15.43:0/578444227 wait complete. 2024-03-26T08:21:02.990 INFO:teuthology.orchestra.run.smithi043.stdout:mgr epoch 5 is available 2024-03-26T08:21:02.990 INFO:teuthology.orchestra.run.smithi043.stdout:Setting orchestrator backend to cephadm... 2024-03-26T08:21:03.595 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:03 smithi043 ceph-mon[31929]: [26/Mar/2024:08:21:02] ENGINE Bus STARTING 2024-03-26T08:21:03.596 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:03 smithi043 ceph-mon[31929]: [26/Mar/2024:08:21:02] ENGINE Serving on http://172.21.15.43:8765 2024-03-26T08:21:03.596 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:03 smithi043 ceph-mon[31929]: [26/Mar/2024:08:21:02] ENGINE Serving on https://172.21.15.43:7150 2024-03-26T08:21:03.596 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:03 smithi043 ceph-mon[31929]: [26/Mar/2024:08:21:02] ENGINE Bus STARTED 2024-03-26T08:21:03.596 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:03 smithi043 ceph-mon[31929]: from='mgr.14120 172.21.15.43:0/1789936270' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:21:04.535 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:04 smithi043 ceph-mon[31929]: mgrmap e8: x(active, since 2s) 2024-03-26T08:21:04.535 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:04 smithi043 ceph-mon[31929]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:21:04.536 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:04 smithi043 ceph-mon[31929]: from='mgr.14120 172.21.15.43:0/1789936270' entity='mgr.x' 2024-03-26T08:21:04.536 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:04 smithi043 ceph-mon[31929]: from='mgr.14120 172.21.15.43:0/1789936270' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:21:04.553 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.600+0000 7fba46a00700 1 Processor -- start 2024-03-26T08:21:04.553 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.600+0000 7fba46a00700 1 -- start start 2024-03-26T08:21:04.553 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.600+0000 7fba46a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba40106170 0x7fba40106590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:04.553 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.600+0000 7fba46a00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba4007ae60 con 0x7fba40106170 2024-03-26T08:21:04.553 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.601+0000 7fba3fe00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba40106170 0x7fba40106590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:04.553 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.601+0000 7fba3fe00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba40106170 0x7fba40106590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33614/0 (socket says 172.21.15.43:33614) 2024-03-26T08:21:04.553 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.601+0000 7fba3fe00700 1 -- 172.21.15.43:0/3677077046 learned_addr learned my addr 172.21.15.43:0/3677077046 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:04.553 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.601+0000 7fba3fe00700 1 -- 172.21.15.43:0/3677077046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba40106ad0 con 0x7fba40106170 2024-03-26T08:21:04.553 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.602+0000 7fba3fe00700 1 --2- 172.21.15.43:0/3677077046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba40106170 0x7fba40106590 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fba300098a0 tx=0x7fba30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4458db0cb9c803d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:04.553 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.602+0000 7fba3f400700 1 -- 172.21.15.43:0/3677077046 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba30004160 con 0x7fba40106170 2024-03-26T08:21:04.554 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.602+0000 7fba3f400700 1 -- 172.21.15.43:0/3677077046 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fba30004300 con 0x7fba40106170 2024-03-26T08:21:04.554 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.602+0000 7fba3f400700 1 -- 172.21.15.43:0/3677077046 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba30011a80 con 0x7fba40106170 2024-03-26T08:21:04.554 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.603+0000 7fba46a00700 1 -- 172.21.15.43:0/3677077046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba40106170 msgr2=0x7fba40106590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:04.554 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.603+0000 7fba46a00700 1 --2- 172.21.15.43:0/3677077046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba40106170 0x7fba40106590 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fba300098a0 tx=0x7fba30009bb0 comp rx=0 tx=0).stop 2024-03-26T08:21:04.554 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.603+0000 7fba46a00700 1 -- 172.21.15.43:0/3677077046 shutdown_connections 2024-03-26T08:21:04.554 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.603+0000 7fba46a00700 1 --2- 172.21.15.43:0/3677077046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba40106170 0x7fba40106590 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:04.554 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.603+0000 7fba46a00700 1 -- 172.21.15.43:0/3677077046 >> 172.21.15.43:0/3677077046 conn(0x7fba40101e50 msgr2=0x7fba40104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:04.554 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.603+0000 7fba46a00700 1 -- 172.21.15.43:0/3677077046 shutdown_connections 2024-03-26T08:21:04.554 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.603+0000 7fba46a00700 1 -- 172.21.15.43:0/3677077046 wait complete. 2024-03-26T08:21:04.554 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.604+0000 7fba46a00700 1 Processor -- start 2024-03-26T08:21:04.554 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.605+0000 7fba46a00700 1 -- start start 2024-03-26T08:21:04.554 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.605+0000 7fba46a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba40106170 0x7fba4019d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:04.554 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.605+0000 7fba46a00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba40107b00 con 0x7fba40106170 2024-03-26T08:21:04.554 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.605+0000 7fba3fe00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba40106170 0x7fba4019d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:04.555 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.605+0000 7fba3fe00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba40106170 0x7fba4019d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33618/0 (socket says 172.21.15.43:33618) 2024-03-26T08:21:04.555 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.605+0000 7fba3fe00700 1 -- 172.21.15.43:0/3794888093 learned_addr learned my addr 172.21.15.43:0/3794888093 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:04.555 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.606+0000 7fba3fe00700 1 -- 172.21.15.43:0/3794888093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba4019df00 con 0x7fba40106170 2024-03-26T08:21:04.555 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.606+0000 7fba3fe00700 1 --2- 172.21.15.43:0/3794888093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba40106170 0x7fba4019d9c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fba30011a50 tx=0x7fba30004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:04.555 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.606+0000 7fba3d600700 1 -- 172.21.15.43:0/3794888093 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba300039f0 con 0x7fba40106170 2024-03-26T08:21:04.555 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.606+0000 7fba3d600700 1 -- 172.21.15.43:0/3794888093 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fba30003b90 con 0x7fba40106170 2024-03-26T08:21:04.555 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.606+0000 7fba3d600700 1 -- 172.21.15.43:0/3794888093 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba3001a600 con 0x7fba40106170 2024-03-26T08:21:04.555 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.606+0000 7fba46a00700 1 -- 172.21.15.43:0/3794888093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba4019e190 con 0x7fba40106170 2024-03-26T08:21:04.555 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.606+0000 7fba46a00700 1 -- 172.21.15.43:0/3794888093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba401a0e80 con 0x7fba40106170 2024-03-26T08:21:04.555 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.608+0000 7fba3d600700 1 -- 172.21.15.43:0/3794888093 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fba30024a40 con 0x7fba40106170 2024-03-26T08:21:04.555 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.608+0000 7fba46a00700 1 -- 172.21.15.43:0/3794888093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba400637c0 con 0x7fba40106170 2024-03-26T08:21:04.555 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.608+0000 7fba3d600700 1 --2- 172.21.15.43:0/3794888093 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7fba24038f20 0x7fba2403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:04.555 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.608+0000 7fba3d600700 1 -- 172.21.15.43:0/3794888093 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fba3004d3c0 con 0x7fba40106170 2024-03-26T08:21:04.555 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.608+0000 7fba37400700 1 --2- 172.21.15.43:0/3794888093 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7fba24038f20 0x7fba2403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:04.556 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.609+0000 7fba37400700 1 --2- 172.21.15.43:0/3794888093 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7fba24038f20 0x7fba2403b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fba28006fd0 tx=0x7fba28006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:04.556 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.614+0000 7fba3d600700 1 -- 172.21.15.43:0/3794888093 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fba3004cd10 con 0x7fba40106170 2024-03-26T08:21:04.556 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.857+0000 7fba46a00700 1 -- 172.21.15.43:0/3794888093 --> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fba400008d0 con 0x7fba24038f20 2024-03-26T08:21:04.556 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.869+0000 7fba3d600700 1 -- 172.21.15.43:0/3794888093 <== mgr.14120 v2:172.21.15.43:6800/2446912488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fba400008d0 con 0x7fba24038f20 2024-03-26T08:21:04.556 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.872+0000 7fba46a00700 1 -- 172.21.15.43:0/3794888093 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7fba24038f20 msgr2=0x7fba2403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:04.556 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.872+0000 7fba46a00700 1 --2- 172.21.15.43:0/3794888093 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7fba24038f20 0x7fba2403b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fba28006fd0 tx=0x7fba28006e40 comp rx=0 tx=0).stop 2024-03-26T08:21:04.556 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.872+0000 7fba46a00700 1 -- 172.21.15.43:0/3794888093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba40106170 msgr2=0x7fba4019d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:04.556 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.872+0000 7fba46a00700 1 --2- 172.21.15.43:0/3794888093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba40106170 0x7fba4019d9c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fba30011a50 tx=0x7fba30004290 comp rx=0 tx=0).stop 2024-03-26T08:21:04.556 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.872+0000 7fba46a00700 1 -- 172.21.15.43:0/3794888093 shutdown_connections 2024-03-26T08:21:04.556 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.872+0000 7fba46a00700 1 --2- 172.21.15.43:0/3794888093 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7fba24038f20 0x7fba2403b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:04.556 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.872+0000 7fba46a00700 1 --2- 172.21.15.43:0/3794888093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba40106170 0x7fba4019d9c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:04.556 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.872+0000 7fba46a00700 1 -- 172.21.15.43:0/3794888093 >> 172.21.15.43:0/3794888093 conn(0x7fba40101e50 msgr2=0x7fba40194630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:04.556 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.873+0000 7fba46a00700 1 -- 172.21.15.43:0/3794888093 shutdown_connections 2024-03-26T08:21:04.556 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:03.873+0000 7fba46a00700 1 -- 172.21.15.43:0/3794888093 wait complete. 2024-03-26T08:21:06.108 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-26T08:21:06.108 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.157+0000 7f18ae000700 1 Processor -- start 2024-03-26T08:21:06.108 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.157+0000 7f18ae000700 1 -- start start 2024-03-26T08:21:06.108 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.158+0000 7f18ae000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18a81078d0 0x7f18a8107cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:06.109 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.158+0000 7f18ae000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a81082c0 con 0x7f18a81078d0 2024-03-26T08:21:06.109 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.158+0000 7f18a7400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18a81078d0 0x7f18a8107cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:06.109 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.158+0000 7f18a7400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18a81078d0 0x7f18a8107cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42652/0 (socket says 172.21.15.43:42652) 2024-03-26T08:21:06.109 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.158+0000 7f18a7400700 1 -- 172.21.15.43:0/1901222677 learned_addr learned my addr 172.21.15.43:0/1901222677 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:06.109 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.159+0000 7f18a7400700 1 -- 172.21.15.43:0/1901222677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a8108a80 con 0x7f18a81078d0 2024-03-26T08:21:06.109 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.159+0000 7f18a7400700 1 --2- 172.21.15.43:0/1901222677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18a81078d0 0x7f18a8107cf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f18980098a0 tx=0x7f1898009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5b6ec70621b08a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:06.109 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.159+0000 7f18a6000700 1 -- 172.21.15.43:0/1901222677 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1898004160 con 0x7f18a81078d0 2024-03-26T08:21:06.109 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.159+0000 7f18a6000700 1 -- 172.21.15.43:0/1901222677 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f1898004300 con 0x7f18a81078d0 2024-03-26T08:21:06.109 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.159+0000 7f18a6000700 1 -- 172.21.15.43:0/1901222677 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1898011a80 con 0x7f18a81078d0 2024-03-26T08:21:06.109 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.160+0000 7f18ae000700 1 -- 172.21.15.43:0/1901222677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18a81078d0 msgr2=0x7f18a8107cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:06.109 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.160+0000 7f18ae000700 1 --2- 172.21.15.43:0/1901222677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18a81078d0 0x7f18a8107cf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f18980098a0 tx=0x7f1898009bb0 comp rx=0 tx=0).stop 2024-03-26T08:21:06.110 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.160+0000 7f18ae000700 1 -- 172.21.15.43:0/1901222677 shutdown_connections 2024-03-26T08:21:06.110 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.160+0000 7f18ae000700 1 --2- 172.21.15.43:0/1901222677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18a81078d0 0x7f18a8107cf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:06.110 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.160+0000 7f18ae000700 1 -- 172.21.15.43:0/1901222677 >> 172.21.15.43:0/1901222677 conn(0x7f18a8102ef0 msgr2=0x7f18a8105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:06.110 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.161+0000 7f18ae000700 1 -- 172.21.15.43:0/1901222677 shutdown_connections 2024-03-26T08:21:06.110 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.161+0000 7f18ae000700 1 -- 172.21.15.43:0/1901222677 wait complete. 2024-03-26T08:21:06.110 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.162+0000 7f18ae000700 1 Processor -- start 2024-03-26T08:21:06.110 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.162+0000 7f18ae000700 1 -- start start 2024-03-26T08:21:06.110 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.162+0000 7f18ae000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18a81078d0 0x7f18a819da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:06.110 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.162+0000 7f18ae000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a810be30 con 0x7f18a81078d0 2024-03-26T08:21:06.110 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.162+0000 7f18a7400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18a81078d0 0x7f18a819da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:06.110 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.163+0000 7f18a7400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18a81078d0 0x7f18a819da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42660/0 (socket says 172.21.15.43:42660) 2024-03-26T08:21:06.111 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.163+0000 7f18a7400700 1 -- 172.21.15.43:0/1821990851 learned_addr learned my addr 172.21.15.43:0/1821990851 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:06.111 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.163+0000 7f18a7400700 1 -- 172.21.15.43:0/1821990851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a819df60 con 0x7f18a81078d0 2024-03-26T08:21:06.111 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.163+0000 7f18a7400700 1 --2- 172.21.15.43:0/1821990851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18a81078d0 0x7f18a819da20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1898000c00 tx=0x7f1898004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:06.111 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.164+0000 7f189fe00700 1 -- 172.21.15.43:0/1821990851 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18980039c0 con 0x7f18a81078d0 2024-03-26T08:21:06.111 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.164+0000 7f189fe00700 1 -- 172.21.15.43:0/1821990851 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f1898003b60 con 0x7f18a81078d0 2024-03-26T08:21:06.111 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.164+0000 7f18ae000700 1 -- 172.21.15.43:0/1821990851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a819e1f0 con 0x7f18a81078d0 2024-03-26T08:21:06.111 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.164+0000 7f189fe00700 1 -- 172.21.15.43:0/1821990851 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f189801a6f0 con 0x7f18a81078d0 2024-03-26T08:21:06.111 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.164+0000 7f18ae000700 1 -- 172.21.15.43:0/1821990851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a81a0ee0 con 0x7f18a81078d0 2024-03-26T08:21:06.111 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.165+0000 7f18ae000700 1 -- 172.21.15.43:0/1821990851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a80637c0 con 0x7f18a81078d0 2024-03-26T08:21:06.111 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.165+0000 7f189fe00700 1 -- 172.21.15.43:0/1821990851 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f1898024440 con 0x7f18a81078d0 2024-03-26T08:21:06.111 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.166+0000 7f189fe00700 1 --2- 172.21.15.43:0/1821990851 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f188c038f70 0x7f188c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:06.111 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.166+0000 7f189fe00700 1 -- 172.21.15.43:0/1821990851 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f189804d160 con 0x7f18a81078d0 2024-03-26T08:21:06.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.166+0000 7f18a6a00700 1 --2- 172.21.15.43:0/1821990851 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f188c038f70 0x7f188c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:06.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.167+0000 7f18a6a00700 1 --2- 172.21.15.43:0/1821990851 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f188c038f70 0x7f188c03b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1894006fd0 tx=0x7f1894006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:06.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.172+0000 7f189fe00700 1 -- 172.21.15.43:0/1821990851 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f18980247e0 con 0x7f18a81078d0 2024-03-26T08:21:06.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.402+0000 7f18ae000700 1 -- 172.21.15.43:0/1821990851 --> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f18a8104110 con 0x7f188c038f70 2024-03-26T08:21:06.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.403+0000 7f189fe00700 1 -- 172.21.15.43:0/1821990851 <== mgr.14120 v2:172.21.15.43:6800/2446912488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f18a8104110 con 0x7f188c038f70 2024-03-26T08:21:06.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.406+0000 7f18ae000700 1 -- 172.21.15.43:0/1821990851 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f188c038f70 msgr2=0x7f188c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:06.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.406+0000 7f18ae000700 1 --2- 172.21.15.43:0/1821990851 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f188c038f70 0x7f188c03b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1894006fd0 tx=0x7f1894006e40 comp rx=0 tx=0).stop 2024-03-26T08:21:06.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.406+0000 7f18ae000700 1 -- 172.21.15.43:0/1821990851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18a81078d0 msgr2=0x7f18a819da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:06.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.406+0000 7f18ae000700 1 --2- 172.21.15.43:0/1821990851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18a81078d0 0x7f18a819da20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1898000c00 tx=0x7f1898004290 comp rx=0 tx=0).stop 2024-03-26T08:21:06.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.406+0000 7f18ae000700 1 -- 172.21.15.43:0/1821990851 shutdown_connections 2024-03-26T08:21:06.112 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.406+0000 7f18ae000700 1 --2- 172.21.15.43:0/1821990851 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f188c038f70 0x7f188c03b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:06.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.406+0000 7f18ae000700 1 --2- 172.21.15.43:0/1821990851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18a81078d0 0x7f18a819da20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:06.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.406+0000 7f18ae000700 1 -- 172.21.15.43:0/1821990851 >> 172.21.15.43:0/1821990851 conn(0x7f18a8102ef0 msgr2=0x7f18a8103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:06.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.406+0000 7f18ae000700 1 -- 172.21.15.43:0/1821990851 shutdown_connections 2024-03-26T08:21:06.113 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:05.406+0000 7f18ae000700 1 -- 172.21.15.43:0/1821990851 wait complete. 2024-03-26T08:21:06.113 INFO:teuthology.orchestra.run.smithi043.stdout:Generating ssh key... 2024-03-26T08:21:06.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:06 smithi043 ceph-mon[31929]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:21:07.347 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:07 smithi043 ceph-mon[31929]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:21:07.348 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:07 smithi043 ceph-mon[31929]: Generating ssh key... 2024-03-26T08:21:07.905 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:07 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: Generating public/private rsa key pair. 2024-03-26T08:21:07.906 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:07 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: Your identification has been saved in /tmp/tmp3x16acrz/key. 2024-03-26T08:21:07.906 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:07 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: Your public key has been saved in /tmp/tmp3x16acrz/key.pub. 2024-03-26T08:21:07.906 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:07 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: The key fingerprint is: 2024-03-26T08:21:07.906 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:07 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: SHA256:IiwE1gPvlBECgvdBFcWHgauGDQhI4Vpw6AZcmeRI25c ceph-88759420-eb49-11ee-95d0-87774f69a715 2024-03-26T08:21:07.906 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:07 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: The key's randomart image is: 2024-03-26T08:21:07.906 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:07 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: +---[RSA 3072]----+ 2024-03-26T08:21:07.906 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:07 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: |@OB*=.o=oo | 2024-03-26T08:21:07.906 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:07 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: |@=O=+ o o . | 2024-03-26T08:21:07.906 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:07 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: |+*oB.E . . | 2024-03-26T08:21:07.906 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:07 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: |o=+.o . | 2024-03-26T08:21:07.907 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:07 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: |o .=o.. S | 2024-03-26T08:21:07.907 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:07 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: | ..+. . | 2024-03-26T08:21:07.907 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:07 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: | . | 2024-03-26T08:21:07.907 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:07 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: | | 2024-03-26T08:21:07.907 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:07 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: | | 2024-03-26T08:21:07.907 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:07 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: +----[SHA256]-----+ 2024-03-26T08:21:08.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.714+0000 7f5435a00700 1 Processor -- start 2024-03-26T08:21:08.400 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.715+0000 7f5435a00700 1 -- start start 2024-03-26T08:21:08.400 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.715+0000 7f5435a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5430109ae0 0x7f5430109f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:08.400 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.715+0000 7f5435a00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543010a4d0 con 0x7f5430109ae0 2024-03-26T08:21:08.400 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.716+0000 7f542ea00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5430109ae0 0x7f5430109f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:08.400 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.716+0000 7f542ea00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5430109ae0 0x7f5430109f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42666/0 (socket says 172.21.15.43:42666) 2024-03-26T08:21:08.400 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.716+0000 7f542ea00700 1 -- 172.21.15.43:0/435569767 learned_addr learned my addr 172.21.15.43:0/435569767 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:08.400 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.716+0000 7f542ea00700 1 -- 172.21.15.43:0/435569767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543010ac40 con 0x7f5430109ae0 2024-03-26T08:21:08.400 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.717+0000 7f542ea00700 1 --2- 172.21.15.43:0/435569767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5430109ae0 0x7f5430109f00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f54200098a0 tx=0x7f5420009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8532935162376b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:08.401 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.717+0000 7f542e000700 1 -- 172.21.15.43:0/435569767 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5420004160 con 0x7f5430109ae0 2024-03-26T08:21:08.401 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.717+0000 7f542e000700 1 -- 172.21.15.43:0/435569767 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f5420004300 con 0x7f5430109ae0 2024-03-26T08:21:08.401 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.717+0000 7f542e000700 1 -- 172.21.15.43:0/435569767 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5420011a80 con 0x7f5430109ae0 2024-03-26T08:21:08.401 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.717+0000 7f5435a00700 1 -- 172.21.15.43:0/435569767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5430109ae0 msgr2=0x7f5430109f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:08.401 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.717+0000 7f5435a00700 1 --2- 172.21.15.43:0/435569767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5430109ae0 0x7f5430109f00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f54200098a0 tx=0x7f5420009bb0 comp rx=0 tx=0).stop 2024-03-26T08:21:08.401 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.718+0000 7f5435a00700 1 -- 172.21.15.43:0/435569767 shutdown_connections 2024-03-26T08:21:08.401 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.718+0000 7f5435a00700 1 --2- 172.21.15.43:0/435569767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5430109ae0 0x7f5430109f00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:08.401 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.718+0000 7f5435a00700 1 -- 172.21.15.43:0/435569767 >> 172.21.15.43:0/435569767 conn(0x7f543007ced0 msgr2=0x7f54301074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:08.401 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.718+0000 7f5435a00700 1 -- 172.21.15.43:0/435569767 shutdown_connections 2024-03-26T08:21:08.402 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.718+0000 7f5435a00700 1 -- 172.21.15.43:0/435569767 wait complete. 2024-03-26T08:21:08.402 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.719+0000 7f5435a00700 1 Processor -- start 2024-03-26T08:21:08.402 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.719+0000 7f5435a00700 1 -- start start 2024-03-26T08:21:08.402 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.719+0000 7f5435a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5430109ae0 0x7f5430080b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:08.402 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.719+0000 7f5435a00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543010dff0 con 0x7f5430109ae0 2024-03-26T08:21:08.402 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.719+0000 7f542ea00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5430109ae0 0x7f5430080b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:08.402 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.720+0000 7f542ea00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5430109ae0 0x7f5430080b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42674/0 (socket says 172.21.15.43:42674) 2024-03-26T08:21:08.402 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.720+0000 7f542ea00700 1 -- 172.21.15.43:0/13123074 learned_addr learned my addr 172.21.15.43:0/13123074 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:08.402 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.720+0000 7f542ea00700 1 -- 172.21.15.43:0/13123074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54300810d0 con 0x7f5430109ae0 2024-03-26T08:21:08.402 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.720+0000 7f542ea00700 1 --2- 172.21.15.43:0/13123074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5430109ae0 0x7f5430080b90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5420000c00 tx=0x7f5420004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:08.402 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.721+0000 7f5427e00700 1 -- 172.21.15.43:0/13123074 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54200039c0 con 0x7f5430109ae0 2024-03-26T08:21:08.402 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.721+0000 7f5435a00700 1 -- 172.21.15.43:0/13123074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5430081360 con 0x7f5430109ae0 2024-03-26T08:21:08.402 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.721+0000 7f5427e00700 1 -- 172.21.15.43:0/13123074 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f5420003b60 con 0x7f5430109ae0 2024-03-26T08:21:08.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.721+0000 7f5427e00700 1 -- 172.21.15.43:0/13123074 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f542001a6f0 con 0x7f5430109ae0 2024-03-26T08:21:08.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.721+0000 7f5435a00700 1 -- 172.21.15.43:0/13123074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f543007d920 con 0x7f5430109ae0 2024-03-26T08:21:08.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.722+0000 7f5427e00700 1 -- 172.21.15.43:0/13123074 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f5420024a40 con 0x7f5430109ae0 2024-03-26T08:21:08.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.722+0000 7f5435a00700 1 -- 172.21.15.43:0/13123074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5430081530 con 0x7f5430109ae0 2024-03-26T08:21:08.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.722+0000 7f5427e00700 1 --2- 172.21.15.43:0/13123074 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f5414038f70 0x7f541403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:08.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.722+0000 7f5427e00700 1 -- 172.21.15.43:0/13123074 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f542004d250 con 0x7f5430109ae0 2024-03-26T08:21:08.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.723+0000 7f5426000700 1 --2- 172.21.15.43:0/13123074 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f5414038f70 0x7f541403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:08.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.724+0000 7f5426000700 1 --2- 172.21.15.43:0/13123074 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f5414038f70 0x7f541403b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f541800ad30 tx=0x7f54180093f0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:08.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.728+0000 7f5427e00700 1 -- 172.21.15.43:0/13123074 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5430081530 con 0x7f5430109ae0 2024-03-26T08:21:08.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:06.956+0000 7f5435a00700 1 -- 172.21.15.43:0/13123074 --> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f543007e070 con 0x7f5414038f70 2024-03-26T08:21:08.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:07.641+0000 7f5427e00700 1 -- 172.21.15.43:0/13123074 <== mgr.14120 v2:172.21.15.43:6800/2446912488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f543007e070 con 0x7f5414038f70 2024-03-26T08:21:08.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:07.645+0000 7f5435a00700 1 -- 172.21.15.43:0/13123074 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f5414038f70 msgr2=0x7f541403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:08.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:07.645+0000 7f5435a00700 1 --2- 172.21.15.43:0/13123074 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f5414038f70 0x7f541403b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f541800ad30 tx=0x7f54180093f0 comp rx=0 tx=0).stop 2024-03-26T08:21:08.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:07.645+0000 7f5435a00700 1 -- 172.21.15.43:0/13123074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5430109ae0 msgr2=0x7f5430080b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:08.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:07.645+0000 7f5435a00700 1 --2- 172.21.15.43:0/13123074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5430109ae0 0x7f5430080b90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5420000c00 tx=0x7f5420004290 comp rx=0 tx=0).stop 2024-03-26T08:21:08.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:07.645+0000 7f5435a00700 1 -- 172.21.15.43:0/13123074 shutdown_connections 2024-03-26T08:21:08.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:07.645+0000 7f5435a00700 1 --2- 172.21.15.43:0/13123074 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f5414038f70 0x7f541403b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:08.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:07.645+0000 7f5435a00700 1 --2- 172.21.15.43:0/13123074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5430109ae0 0x7f5430080b90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:08.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:07.645+0000 7f5435a00700 1 -- 172.21.15.43:0/13123074 >> 172.21.15.43:0/13123074 conn(0x7f543007ced0 msgr2=0x7f5430105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:08.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:07.645+0000 7f5435a00700 1 -- 172.21.15.43:0/13123074 shutdown_connections 2024-03-26T08:21:08.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:07.645+0000 7f5435a00700 1 -- 172.21.15.43:0/13123074 wait complete. 2024-03-26T08:21:08.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:08 smithi043 ceph-mon[31929]: from='mgr.14120 172.21.15.43:0/1789936270' entity='mgr.x' 2024-03-26T08:21:08.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:08 smithi043 ceph-mon[31929]: from='mgr.14120 172.21.15.43:0/1789936270' entity='mgr.x' 2024-03-26T08:21:09.862 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3Xreypq2Hj5PzvATciwoEDKxj5ZfDIlDL/e5h04FXc3kNaKw4+wJHCD0Jl22T+ur8QwcCykDbFgloFFNCeET/soJxduH1ogE/6ezbhAQ+2Po599PepXKd7IgpgMWchsBL/+/7b/9e5lUuxvWWETZn2FoHNe80NeiyjH5w+dwiomQcoxIuNWzbNIO9NqpRMJ1oPmFi2JGQiu5lSzfGLRksw/qzTWmfigY1y4VH+7XjWfJARPDrDs7QsjboHFogqExj7spYbSPFy7WksXkKenupJP8QLStROuKsxmFCXMGoIB+/EwBHTs21UuMwdm1uZ148jOaJWVWcmKkLTMrB7gw6TvnZPRLgWbF3cutf9GvueXD2IsCqZHzKXPTr11y4UJhgTJxhYsPzillXwawHSefUlxHHOvI1W9k4u8k3oNj2JX//XMl4aalrCyHwfXI6U+r2PJAFTNPKy73v3D2WOejUETMAvxpaWM8dkeMol/l/QCSKJRKDX+HBZYVts0C817E= ceph-88759420-eb49-11ee-95d0-87774f69a715 2024-03-26T08:21:09.863 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.048+0000 7fbcebe00700 1 Processor -- start 2024-03-26T08:21:09.863 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.048+0000 7fbcebe00700 1 -- start start 2024-03-26T08:21:09.863 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.048+0000 7fbcebe00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec109910 0x7fbcec109d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:09.863 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.048+0000 7fbcebe00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec10a300 con 0x7fbcec109910 2024-03-26T08:21:09.863 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.049+0000 7fbceaa00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec109910 0x7fbcec109d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:09.864 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.049+0000 7fbceaa00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec109910 0x7fbcec109d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42676/0 (socket says 172.21.15.43:42676) 2024-03-26T08:21:09.864 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.049+0000 7fbceaa00700 1 -- 172.21.15.43:0/2783146720 learned_addr learned my addr 172.21.15.43:0/2783146720 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:09.865 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.049+0000 7fbceaa00700 1 -- 172.21.15.43:0/2783146720 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec10aa70 con 0x7fbcec109910 2024-03-26T08:21:09.865 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.050+0000 7fbceaa00700 1 --2- 172.21.15.43:0/2783146720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec109910 0x7fbcec109d30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc0098a0 tx=0x7fbcdc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2ae0585536bce server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:09.865 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.050+0000 7fbce9600700 1 -- 172.21.15.43:0/2783146720 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcdc004160 con 0x7fbcec109910 2024-03-26T08:21:09.865 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.050+0000 7fbce9600700 1 -- 172.21.15.43:0/2783146720 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fbcdc004300 con 0x7fbcec109910 2024-03-26T08:21:09.865 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.050+0000 7fbce9600700 1 -- 172.21.15.43:0/2783146720 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcdc011a80 con 0x7fbcec109910 2024-03-26T08:21:09.865 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.051+0000 7fbcebe00700 1 -- 172.21.15.43:0/2783146720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec109910 msgr2=0x7fbcec109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:09.865 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.051+0000 7fbcebe00700 1 --2- 172.21.15.43:0/2783146720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec109910 0x7fbcec109d30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc0098a0 tx=0x7fbcdc009bb0 comp rx=0 tx=0).stop 2024-03-26T08:21:09.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.051+0000 7fbcebe00700 1 -- 172.21.15.43:0/2783146720 shutdown_connections 2024-03-26T08:21:09.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.051+0000 7fbcebe00700 1 --2- 172.21.15.43:0/2783146720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec109910 0x7fbcec109d30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:09.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.051+0000 7fbcebe00700 1 -- 172.21.15.43:0/2783146720 >> 172.21.15.43:0/2783146720 conn(0x7fbcec07cd80 msgr2=0x7fbcec1074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:09.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.051+0000 7fbcebe00700 1 -- 172.21.15.43:0/2783146720 shutdown_connections 2024-03-26T08:21:09.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.051+0000 7fbcebe00700 1 -- 172.21.15.43:0/2783146720 wait complete. 2024-03-26T08:21:09.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.052+0000 7fbcebe00700 1 Processor -- start 2024-03-26T08:21:09.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.052+0000 7fbcebe00700 1 -- start start 2024-03-26T08:21:09.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.053+0000 7fbcebe00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec109910 0x7fbcec19d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:09.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.053+0000 7fbcebe00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec10de20 con 0x7fbcec109910 2024-03-26T08:21:09.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.053+0000 7fbceaa00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec109910 0x7fbcec19d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:09.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.053+0000 7fbceaa00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec109910 0x7fbcec19d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42682/0 (socket says 172.21.15.43:42682) 2024-03-26T08:21:09.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.053+0000 7fbceaa00700 1 -- 172.21.15.43:0/1701693256 learned_addr learned my addr 172.21.15.43:0/1701693256 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:09.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.053+0000 7fbceaa00700 1 -- 172.21.15.43:0/1701693256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec19dce0 con 0x7fbcec109910 2024-03-26T08:21:09.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.054+0000 7fbceaa00700 1 --2- 172.21.15.43:0/1701693256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec109910 0x7fbcec19d7a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc000c00 tx=0x7fbcdc011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:09.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.054+0000 7fbce3400700 1 -- 172.21.15.43:0/1701693256 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcdc0039f0 con 0x7fbcec109910 2024-03-26T08:21:09.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.054+0000 7fbce3400700 1 -- 172.21.15.43:0/1701693256 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fbcdc003b90 con 0x7fbcec109910 2024-03-26T08:21:09.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.054+0000 7fbce3400700 1 -- 172.21.15.43:0/1701693256 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcdc01a740 con 0x7fbcec109910 2024-03-26T08:21:09.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.054+0000 7fbcebe00700 1 -- 172.21.15.43:0/1701693256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcec19df70 con 0x7fbcec109910 2024-03-26T08:21:09.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.054+0000 7fbcebe00700 1 -- 172.21.15.43:0/1701693256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcec1a0c60 con 0x7fbcec109910 2024-03-26T08:21:09.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.055+0000 7fbcebe00700 1 -- 172.21.15.43:0/1701693256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcec0637c0 con 0x7fbcec109910 2024-03-26T08:21:09.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.055+0000 7fbce3400700 1 -- 172.21.15.43:0/1701693256 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fbcdc024af0 con 0x7fbcec109910 2024-03-26T08:21:09.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.056+0000 7fbce3400700 1 --2- 172.21.15.43:0/1701693256 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7fbcd0038f70 0x7fbcd003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:09.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.056+0000 7fbce3400700 1 -- 172.21.15.43:0/1701693256 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbcdc04d210 con 0x7fbcec109910 2024-03-26T08:21:09.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.056+0000 7fbcea000700 1 --2- 172.21.15.43:0/1701693256 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7fbcd0038f70 0x7fbcd003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:09.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.057+0000 7fbcea000700 1 --2- 172.21.15.43:0/1701693256 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7fbcd0038f70 0x7fbcd003b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8006fd0 tx=0x7fbcd8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:09.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.061+0000 7fbce3400700 1 -- 172.21.15.43:0/1701693256 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbcdc00e260 con 0x7fbcec109910 2024-03-26T08:21:09.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.288+0000 7fbcebe00700 1 -- 172.21.15.43:0/1701693256 --> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fbcec1072d0 con 0x7fbcd0038f70 2024-03-26T08:21:09.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.289+0000 7fbce3400700 1 -- 172.21.15.43:0/1701693256 <== mgr.14120 v2:172.21.15.43:6800/2446912488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fbcec1072d0 con 0x7fbcd0038f70 2024-03-26T08:21:09.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.292+0000 7fbcebe00700 1 -- 172.21.15.43:0/1701693256 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7fbcd0038f70 msgr2=0x7fbcd003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:09.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.292+0000 7fbcebe00700 1 --2- 172.21.15.43:0/1701693256 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7fbcd0038f70 0x7fbcd003b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8006fd0 tx=0x7fbcd8006e40 comp rx=0 tx=0).stop 2024-03-26T08:21:09.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.292+0000 7fbcebe00700 1 -- 172.21.15.43:0/1701693256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec109910 msgr2=0x7fbcec19d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:09.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.292+0000 7fbcebe00700 1 --2- 172.21.15.43:0/1701693256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec109910 0x7fbcec19d7a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc000c00 tx=0x7fbcdc011b00 comp rx=0 tx=0).stop 2024-03-26T08:21:09.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.292+0000 7fbcebe00700 1 -- 172.21.15.43:0/1701693256 shutdown_connections 2024-03-26T08:21:09.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.292+0000 7fbcebe00700 1 --2- 172.21.15.43:0/1701693256 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7fbcd0038f70 0x7fbcd003b430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:09.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.293+0000 7fbcebe00700 1 --2- 172.21.15.43:0/1701693256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcec109910 0x7fbcec19d7a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:09.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.293+0000 7fbcebe00700 1 -- 172.21.15.43:0/1701693256 >> 172.21.15.43:0/1701693256 conn(0x7fbcec07cd80 msgr2=0x7fbcec106bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:09.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.293+0000 7fbcebe00700 1 -- 172.21.15.43:0/1701693256 shutdown_connections 2024-03-26T08:21:09.873 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:09.293+0000 7fbcebe00700 1 -- 172.21.15.43:0/1701693256 wait complete. 2024-03-26T08:21:09.873 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-26T08:21:09.873 INFO:teuthology.orchestra.run.smithi043.stdout:Adding key to root@localhost authorized_keys... 2024-03-26T08:21:09.873 INFO:teuthology.orchestra.run.smithi043.stdout:Adding host smithi043... 2024-03-26T08:21:09.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:09 smithi043 ceph-mon[31929]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:21:11.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:11 smithi043 ceph-mon[31929]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi043", "addr": "172.21.15.43", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:21:12.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:12 smithi043 ceph-mon[31929]: Deploying cephadm binary to smithi043 2024-03-26T08:21:18.029 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Added host 'smithi043' with addr '172.21.15.43' 2024-03-26T08:21:18.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.529+0000 7f872e800700 1 Processor -- start 2024-03-26T08:21:18.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.530+0000 7f872e800700 1 -- start start 2024-03-26T08:21:18.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.530+0000 7f872e800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8728109910 0x7f8728109d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:18.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.530+0000 7f872e800700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872810a300 con 0x7f8728109910 2024-03-26T08:21:18.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.531+0000 7f872d400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8728109910 0x7f8728109d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:18.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.531+0000 7f872d400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8728109910 0x7f8728109d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42684/0 (socket says 172.21.15.43:42684) 2024-03-26T08:21:18.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.531+0000 7f872d400700 1 -- 172.21.15.43:0/3788272759 learned_addr learned my addr 172.21.15.43:0/3788272759 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:18.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.532+0000 7f872d400700 1 -- 172.21.15.43:0/3788272759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872810aa70 con 0x7f8728109910 2024-03-26T08:21:18.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.532+0000 7f872d400700 1 --2- 172.21.15.43:0/3788272759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8728109910 0x7f8728109d30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f871c0098a0 tx=0x7f871c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6270ec0e3d711344 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:18.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.533+0000 7f8727e00700 1 -- 172.21.15.43:0/3788272759 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f871c004160 con 0x7f8728109910 2024-03-26T08:21:18.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.533+0000 7f8727e00700 1 -- 172.21.15.43:0/3788272759 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f871c004300 con 0x7f8728109910 2024-03-26T08:21:18.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.533+0000 7f8727e00700 1 -- 172.21.15.43:0/3788272759 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f871c011a80 con 0x7f8728109910 2024-03-26T08:21:18.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.533+0000 7f872e800700 1 -- 172.21.15.43:0/3788272759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8728109910 msgr2=0x7f8728109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:18.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.533+0000 7f872e800700 1 --2- 172.21.15.43:0/3788272759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8728109910 0x7f8728109d30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f871c0098a0 tx=0x7f871c009bb0 comp rx=0 tx=0).stop 2024-03-26T08:21:18.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.533+0000 7f872e800700 1 -- 172.21.15.43:0/3788272759 shutdown_connections 2024-03-26T08:21:18.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.533+0000 7f872e800700 1 --2- 172.21.15.43:0/3788272759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8728109910 0x7f8728109d30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:18.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.533+0000 7f872e800700 1 -- 172.21.15.43:0/3788272759 >> 172.21.15.43:0/3788272759 conn(0x7f872807cd80 msgr2=0x7f87281074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:18.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.534+0000 7f872e800700 1 -- 172.21.15.43:0/3788272759 shutdown_connections 2024-03-26T08:21:18.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.534+0000 7f872e800700 1 -- 172.21.15.43:0/3788272759 wait complete. 2024-03-26T08:21:18.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.535+0000 7f872e800700 1 Processor -- start 2024-03-26T08:21:18.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.535+0000 7f872e800700 1 -- start start 2024-03-26T08:21:18.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.535+0000 7f872e800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8728109910 0x7f872819d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:18.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.535+0000 7f872e800700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872810de20 con 0x7f8728109910 2024-03-26T08:21:18.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.535+0000 7f872d400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8728109910 0x7f872819d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:18.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.535+0000 7f872d400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8728109910 0x7f872819d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42686/0 (socket says 172.21.15.43:42686) 2024-03-26T08:21:18.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.535+0000 7f872d400700 1 -- 172.21.15.43:0/244960059 learned_addr learned my addr 172.21.15.43:0/244960059 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:18.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.536+0000 7f872d400700 1 -- 172.21.15.43:0/244960059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872819dd00 con 0x7f8728109910 2024-03-26T08:21:18.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.536+0000 7f872d400700 1 --2- 172.21.15.43:0/244960059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8728109910 0x7f872819d7c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f871c000c00 tx=0x7f871c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:18.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.536+0000 7f8726000700 1 -- 172.21.15.43:0/244960059 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f871c0039f0 con 0x7f8728109910 2024-03-26T08:21:18.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.536+0000 7f8726000700 1 -- 172.21.15.43:0/244960059 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f871c003b90 con 0x7f8728109910 2024-03-26T08:21:18.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.537+0000 7f8726000700 1 -- 172.21.15.43:0/244960059 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f871c01a740 con 0x7f8728109910 2024-03-26T08:21:18.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.537+0000 7f872e800700 1 -- 172.21.15.43:0/244960059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f872819df90 con 0x7f8728109910 2024-03-26T08:21:18.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.537+0000 7f872e800700 1 -- 172.21.15.43:0/244960059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87281a0c80 con 0x7f8728109910 2024-03-26T08:21:18.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.538+0000 7f8726000700 1 -- 172.21.15.43:0/244960059 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f871c024af0 con 0x7f8728109910 2024-03-26T08:21:18.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.538+0000 7f872e800700 1 -- 172.21.15.43:0/244960059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87280637c0 con 0x7f8728109910 2024-03-26T08:21:18.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.538+0000 7f8726000700 1 --2- 172.21.15.43:0/244960059 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f8710038f70 0x7f871003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:18.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.538+0000 7f8726000700 1 -- 172.21.15.43:0/244960059 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f871c04d170 con 0x7f8728109910 2024-03-26T08:21:18.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.539+0000 7f872ca00700 1 --2- 172.21.15.43:0/244960059 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f8710038f70 0x7f871003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:18.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.539+0000 7f872ca00700 1 --2- 172.21.15.43:0/244960059 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f8710038f70 0x7f871003b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8714006fd0 tx=0x7f8714006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:18.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.544+0000 7f8726000700 1 -- 172.21.15.43:0/244960059 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f871c00e260 con 0x7f8728109910 2024-03-26T08:21:18.034 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:10.777+0000 7f872e800700 1 -- 172.21.15.43:0/244960059 --> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi043", "addr": "172.21.15.43", "target": ["mon-mgr", ""]}) v1 -- 0x7f87281072f0 con 0x7f8710038f70 2024-03-26T08:21:18.034 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:17.416+0000 7f8726000700 1 -- 172.21.15.43:0/244960059 <== mgr.14120 v2:172.21.15.43:6800/2446912488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f87281072f0 con 0x7f8710038f70 2024-03-26T08:21:18.034 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:17.420+0000 7f872e800700 1 -- 172.21.15.43:0/244960059 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f8710038f70 msgr2=0x7f871003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:18.034 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:17.420+0000 7f872e800700 1 --2- 172.21.15.43:0/244960059 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f8710038f70 0x7f871003b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8714006fd0 tx=0x7f8714006e40 comp rx=0 tx=0).stop 2024-03-26T08:21:18.034 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:17.420+0000 7f872e800700 1 -- 172.21.15.43:0/244960059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8728109910 msgr2=0x7f872819d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:18.034 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:17.420+0000 7f872e800700 1 --2- 172.21.15.43:0/244960059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8728109910 0x7f872819d7c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f871c000c00 tx=0x7f871c011b00 comp rx=0 tx=0).stop 2024-03-26T08:21:18.034 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:17.420+0000 7f872e800700 1 -- 172.21.15.43:0/244960059 shutdown_connections 2024-03-26T08:21:18.034 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:17.420+0000 7f872e800700 1 --2- 172.21.15.43:0/244960059 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f8710038f70 0x7f871003b430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:18.034 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:17.420+0000 7f872e800700 1 --2- 172.21.15.43:0/244960059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8728109910 0x7f872819d7c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:18.034 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:17.420+0000 7f872e800700 1 -- 172.21.15.43:0/244960059 >> 172.21.15.43:0/244960059 conn(0x7f872807cd80 msgr2=0x7f8728106be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:18.034 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:17.420+0000 7f872e800700 1 -- 172.21.15.43:0/244960059 shutdown_connections 2024-03-26T08:21:18.034 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:17.421+0000 7f872e800700 1 -- 172.21.15.43:0/244960059 wait complete. 2024-03-26T08:21:18.035 INFO:teuthology.orchestra.run.smithi043.stdout:Deploying unmanaged mon service... 2024-03-26T08:21:18.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:18 smithi043 ceph-mon[31929]: from='mgr.14120 172.21.15.43:0/1789936270' entity='mgr.x' 2024-03-26T08:21:18.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:18 smithi043 ceph-mon[31929]: Added host smithi043 2024-03-26T08:21:18.532 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:18 smithi043 ceph-mon[31929]: from='mgr.14120 172.21.15.43:0/1789936270' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:21:19.565 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-26T08:21:19.567 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.708+0000 7f1c52c00700 1 Processor -- start 2024-03-26T08:21:19.567 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.708+0000 7f1c52c00700 1 -- start start 2024-03-26T08:21:19.567 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.708+0000 7f1c52c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c4c109ac0 0x7f1c4c109ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:19.567 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.708+0000 7f1c52c00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c10a4b0 con 0x7f1c4c109ac0 2024-03-26T08:21:19.567 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.709+0000 7f1c4be00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c4c109ac0 0x7f1c4c109ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:19.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.709+0000 7f1c4be00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c4c109ac0 0x7f1c4c109ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48324/0 (socket says 172.21.15.43:48324) 2024-03-26T08:21:19.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.709+0000 7f1c4be00700 1 -- 172.21.15.43:0/2986390527 learned_addr learned my addr 172.21.15.43:0/2986390527 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:19.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.709+0000 7f1c4be00700 1 -- 172.21.15.43:0/2986390527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4c10ac70 con 0x7f1c4c109ac0 2024-03-26T08:21:19.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.710+0000 7f1c4be00700 1 --2- 172.21.15.43:0/2986390527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c4c109ac0 0x7f1c4c109ee0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c0098a0 tx=0x7f1c3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c8e0b9d71036028 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:19.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.710+0000 7f1c4aa00700 1 -- 172.21.15.43:0/2986390527 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c3c004160 con 0x7f1c4c109ac0 2024-03-26T08:21:19.568 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.710+0000 7f1c4aa00700 1 -- 172.21.15.43:0/2986390527 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f1c3c004300 con 0x7f1c4c109ac0 2024-03-26T08:21:19.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.710+0000 7f1c4aa00700 1 -- 172.21.15.43:0/2986390527 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c3c011a80 con 0x7f1c4c109ac0 2024-03-26T08:21:19.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.711+0000 7f1c52c00700 1 -- 172.21.15.43:0/2986390527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c4c109ac0 msgr2=0x7f1c4c109ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:19.570 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.711+0000 7f1c52c00700 1 --2- 172.21.15.43:0/2986390527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c4c109ac0 0x7f1c4c109ee0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c0098a0 tx=0x7f1c3c009bb0 comp rx=0 tx=0).stop 2024-03-26T08:21:19.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.711+0000 7f1c52c00700 1 -- 172.21.15.43:0/2986390527 shutdown_connections 2024-03-26T08:21:19.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.711+0000 7f1c52c00700 1 --2- 172.21.15.43:0/2986390527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c4c109ac0 0x7f1c4c109ee0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:19.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.711+0000 7f1c52c00700 1 -- 172.21.15.43:0/2986390527 >> 172.21.15.43:0/2986390527 conn(0x7f1c4c07cee0 msgr2=0x7f1c4c1074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:19.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.711+0000 7f1c52c00700 1 -- 172.21.15.43:0/2986390527 shutdown_connections 2024-03-26T08:21:19.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.711+0000 7f1c52c00700 1 -- 172.21.15.43:0/2986390527 wait complete. 2024-03-26T08:21:19.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.712+0000 7f1c52c00700 1 Processor -- start 2024-03-26T08:21:19.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.712+0000 7f1c52c00700 1 -- start start 2024-03-26T08:21:19.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.712+0000 7f1c52c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c4c109ac0 0x7f1c4c19da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:19.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.712+0000 7f1c52c00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c10e020 con 0x7f1c4c109ac0 2024-03-26T08:21:19.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.713+0000 7f1c4be00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c4c109ac0 0x7f1c4c19da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:19.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.713+0000 7f1c4be00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c4c109ac0 0x7f1c4c19da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48332/0 (socket says 172.21.15.43:48332) 2024-03-26T08:21:19.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.713+0000 7f1c4be00700 1 -- 172.21.15.43:0/1665492897 learned_addr learned my addr 172.21.15.43:0/1665492897 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:19.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.713+0000 7f1c4be00700 1 -- 172.21.15.43:0/1665492897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4c19dfb0 con 0x7f1c4c109ac0 2024-03-26T08:21:19.571 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.713+0000 7f1c4be00700 1 --2- 172.21.15.43:0/1665492897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c4c109ac0 0x7f1c4c19da70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c006af0 tx=0x7f1c3c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:19.572 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.714+0000 7f1c48c00700 1 -- 172.21.15.43:0/1665492897 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c3c003930 con 0x7f1c4c109ac0 2024-03-26T08:21:19.572 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.714+0000 7f1c48c00700 1 -- 172.21.15.43:0/1665492897 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f1c3c003ad0 con 0x7f1c4c109ac0 2024-03-26T08:21:19.572 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.714+0000 7f1c48c00700 1 -- 172.21.15.43:0/1665492897 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c3c0195e0 con 0x7f1c4c109ac0 2024-03-26T08:21:19.572 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.714+0000 7f1c52c00700 1 -- 172.21.15.43:0/1665492897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c4c19e240 con 0x7f1c4c109ac0 2024-03-26T08:21:19.572 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.714+0000 7f1c52c00700 1 -- 172.21.15.43:0/1665492897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4c1a0f30 con 0x7f1c4c109ac0 2024-03-26T08:21:19.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.715+0000 7f1c48c00700 1 -- 172.21.15.43:0/1665492897 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f1c3c019890 con 0x7f1c4c109ac0 2024-03-26T08:21:19.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.715+0000 7f1c52c00700 1 -- 172.21.15.43:0/1665492897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c4c0637c0 con 0x7f1c4c109ac0 2024-03-26T08:21:19.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.716+0000 7f1c48c00700 1 --2- 172.21.15.43:0/1665492897 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f1c30038f20 0x7f1c3003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:19.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.716+0000 7f1c48c00700 1 -- 172.21.15.43:0/1665492897 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1c3c04d2e0 con 0x7f1c4c109ac0 2024-03-26T08:21:19.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.716+0000 7f1c4b400700 1 --2- 172.21.15.43:0/1665492897 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f1c30038f20 0x7f1c3003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:19.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.717+0000 7f1c4b400700 1 --2- 172.21.15.43:0/1665492897 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f1c30038f20 0x7f1c3003b3e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1c38006fd0 tx=0x7f1c38006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:19.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.722+0000 7f1c48c00700 1 -- 172.21.15.43:0/1665492897 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1c3c01a430 con 0x7f1c4c109ac0 2024-03-26T08:21:19.574 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.963+0000 7f1c52c00700 1 -- 172.21.15.43:0/1665492897 --> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f1c4c106360 con 0x7f1c30038f20 2024-03-26T08:21:19.575 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.976+0000 7f1c48c00700 1 -- 172.21.15.43:0/1665492897 <== mgr.14120 v2:172.21.15.43:6800/2446912488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f1c4c106360 con 0x7f1c30038f20 2024-03-26T08:21:19.575 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.979+0000 7f1c52c00700 1 -- 172.21.15.43:0/1665492897 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f1c30038f20 msgr2=0x7f1c3003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:19.575 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.979+0000 7f1c52c00700 1 --2- 172.21.15.43:0/1665492897 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f1c30038f20 0x7f1c3003b3e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1c38006fd0 tx=0x7f1c38006e40 comp rx=0 tx=0).stop 2024-03-26T08:21:19.575 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.979+0000 7f1c52c00700 1 -- 172.21.15.43:0/1665492897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c4c109ac0 msgr2=0x7f1c4c19da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:19.575 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.979+0000 7f1c52c00700 1 --2- 172.21.15.43:0/1665492897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c4c109ac0 0x7f1c4c19da70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c006af0 tx=0x7f1c3c011b70 comp rx=0 tx=0).stop 2024-03-26T08:21:19.575 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.979+0000 7f1c52c00700 1 -- 172.21.15.43:0/1665492897 shutdown_connections 2024-03-26T08:21:19.575 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.979+0000 7f1c52c00700 1 --2- 172.21.15.43:0/1665492897 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f1c30038f20 0x7f1c3003b3e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:19.575 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.979+0000 7f1c52c00700 1 --2- 172.21.15.43:0/1665492897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c4c109ac0 0x7f1c4c19da70 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:19.575 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.979+0000 7f1c52c00700 1 -- 172.21.15.43:0/1665492897 >> 172.21.15.43:0/1665492897 conn(0x7f1c4c07cee0 msgr2=0x7f1c4c105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:19.575 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.979+0000 7f1c52c00700 1 -- 172.21.15.43:0/1665492897 shutdown_connections 2024-03-26T08:21:19.575 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:18.979+0000 7f1c52c00700 1 -- 172.21.15.43:0/1665492897 wait complete. 2024-03-26T08:21:19.575 INFO:teuthology.orchestra.run.smithi043.stdout:Deploying unmanaged mgr service... 2024-03-26T08:21:20.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:19 smithi043 ceph-mon[31929]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:21:20.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:19 smithi043 ceph-mon[31929]: Saving service mon spec with placement count:5 2024-03-26T08:21:20.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:19 smithi043 ceph-mon[31929]: from='mgr.14120 172.21.15.43:0/1789936270' entity='mgr.x' 2024-03-26T08:21:21.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-26T08:21:21.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.429+0000 7f2eae600700 1 Processor -- start 2024-03-26T08:21:21.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.430+0000 7f2eae600700 1 -- start start 2024-03-26T08:21:21.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.430+0000 7f2eae600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea8107b40 0x7f2ea8107f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:21.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.430+0000 7f2eae600700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea8108530 con 0x7f2ea8107b40 2024-03-26T08:21:21.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.431+0000 7f2ead200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea8107b40 0x7f2ea8107f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:21.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.431+0000 7f2ead200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea8107b40 0x7f2ea8107f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48346/0 (socket says 172.21.15.43:48346) 2024-03-26T08:21:21.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.431+0000 7f2ead200700 1 -- 172.21.15.43:0/1409765524 learned_addr learned my addr 172.21.15.43:0/1409765524 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:21.418 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.431+0000 7f2ead200700 1 -- 172.21.15.43:0/1409765524 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea81067a0 con 0x7f2ea8107b40 2024-03-26T08:21:21.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.432+0000 7f2ead200700 1 --2- 172.21.15.43:0/1409765524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea8107b40 0x7f2ea8107f60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c0098a0 tx=0x7f2e9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3abc93377c68ef server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:21.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.432+0000 7f2ea7400700 1 -- 172.21.15.43:0/1409765524 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e9c004160 con 0x7f2ea8107b40 2024-03-26T08:21:21.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.432+0000 7f2ea7400700 1 -- 172.21.15.43:0/1409765524 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f2e9c004300 con 0x7f2ea8107b40 2024-03-26T08:21:21.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.432+0000 7f2ea7400700 1 -- 172.21.15.43:0/1409765524 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e9c011a80 con 0x7f2ea8107b40 2024-03-26T08:21:21.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.433+0000 7f2eae600700 1 -- 172.21.15.43:0/1409765524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea8107b40 msgr2=0x7f2ea8107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:21.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.433+0000 7f2eae600700 1 --2- 172.21.15.43:0/1409765524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea8107b40 0x7f2ea8107f60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c0098a0 tx=0x7f2e9c009bb0 comp rx=0 tx=0).stop 2024-03-26T08:21:21.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.433+0000 7f2eae600700 1 -- 172.21.15.43:0/1409765524 shutdown_connections 2024-03-26T08:21:21.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.433+0000 7f2eae600700 1 --2- 172.21.15.43:0/1409765524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea8107b40 0x7f2ea8107f60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:21.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.433+0000 7f2eae600700 1 -- 172.21.15.43:0/1409765524 >> 172.21.15.43:0/1409765524 conn(0x7f2ea81012b0 msgr2=0x7f2ea81036f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:21.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.433+0000 7f2eae600700 1 -- 172.21.15.43:0/1409765524 shutdown_connections 2024-03-26T08:21:21.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.433+0000 7f2eae600700 1 -- 172.21.15.43:0/1409765524 wait complete. 2024-03-26T08:21:21.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.434+0000 7f2eae600700 1 Processor -- start 2024-03-26T08:21:21.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.434+0000 7f2eae600700 1 -- start start 2024-03-26T08:21:21.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.435+0000 7f2eae600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea8107b40 0x7f2ea819b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:21.419 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.435+0000 7f2eae600700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea810ab60 con 0x7f2ea8107b40 2024-03-26T08:21:21.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.435+0000 7f2ead200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea8107b40 0x7f2ea819b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:21.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.435+0000 7f2ead200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea8107b40 0x7f2ea819b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48362/0 (socket says 172.21.15.43:48362) 2024-03-26T08:21:21.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.435+0000 7f2ead200700 1 -- 172.21.15.43:0/3109208247 learned_addr learned my addr 172.21.15.43:0/3109208247 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:21.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.435+0000 7f2ead200700 1 -- 172.21.15.43:0/3109208247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea819bba0 con 0x7f2ea8107b40 2024-03-26T08:21:21.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.436+0000 7f2ead200700 1 --2- 172.21.15.43:0/3109208247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea8107b40 0x7f2ea819b660 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c011a50 tx=0x7f2e9c004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:21.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.436+0000 7f2ea5600700 1 -- 172.21.15.43:0/3109208247 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e9c0039f0 con 0x7f2ea8107b40 2024-03-26T08:21:21.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.436+0000 7f2ea5600700 1 -- 172.21.15.43:0/3109208247 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f2e9c003b90 con 0x7f2ea8107b40 2024-03-26T08:21:21.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.436+0000 7f2ea5600700 1 -- 172.21.15.43:0/3109208247 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e9c01a600 con 0x7f2ea8107b40 2024-03-26T08:21:21.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.436+0000 7f2eae600700 1 -- 172.21.15.43:0/3109208247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea819be30 con 0x7f2ea8107b40 2024-03-26T08:21:21.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.436+0000 7f2eae600700 1 -- 172.21.15.43:0/3109208247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea819eb20 con 0x7f2ea8107b40 2024-03-26T08:21:21.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.438+0000 7f2eae600700 1 -- 172.21.15.43:0/3109208247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ea80637c0 con 0x7f2ea8107b40 2024-03-26T08:21:21.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.438+0000 7f2ea5600700 1 -- 172.21.15.43:0/3109208247 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f2e9c024a40 con 0x7f2ea8107b40 2024-03-26T08:21:21.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.439+0000 7f2ea5600700 1 --2- 172.21.15.43:0/3109208247 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f2e90038f70 0x7f2e9003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:21.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.439+0000 7f2ea5600700 1 -- 172.21.15.43:0/3109208247 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2e9c04d350 con 0x7f2ea8107b40 2024-03-26T08:21:21.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.439+0000 7f2ea7e00700 1 --2- 172.21.15.43:0/3109208247 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f2e90038f70 0x7f2e9003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:21.420 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.440+0000 7f2ea7e00700 1 --2- 172.21.15.43:0/3109208247 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f2e90038f70 0x7f2e9003b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2e94006fd0 tx=0x7f2e94006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:21.421 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.444+0000 7f2ea5600700 1 -- 172.21.15.43:0/3109208247 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2e9c04cce0 con 0x7f2ea8107b40 2024-03-26T08:21:21.421 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.690+0000 7f2eae600700 1 -- 172.21.15.43:0/3109208247 --> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f2ea80008d0 con 0x7f2e90038f70 2024-03-26T08:21:21.421 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.700+0000 7f2ea5600700 1 -- 172.21.15.43:0/3109208247 <== mgr.14120 v2:172.21.15.43:6800/2446912488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f2ea80008d0 con 0x7f2e90038f70 2024-03-26T08:21:21.421 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.703+0000 7f2eae600700 1 -- 172.21.15.43:0/3109208247 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f2e90038f70 msgr2=0x7f2e9003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:21.421 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.703+0000 7f2eae600700 1 --2- 172.21.15.43:0/3109208247 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f2e90038f70 0x7f2e9003b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2e94006fd0 tx=0x7f2e94006e40 comp rx=0 tx=0).stop 2024-03-26T08:21:21.421 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.703+0000 7f2eae600700 1 -- 172.21.15.43:0/3109208247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea8107b40 msgr2=0x7f2ea819b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:21.421 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.703+0000 7f2eae600700 1 --2- 172.21.15.43:0/3109208247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea8107b40 0x7f2ea819b660 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c011a50 tx=0x7f2e9c004290 comp rx=0 tx=0).stop 2024-03-26T08:21:21.421 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.704+0000 7f2eae600700 1 -- 172.21.15.43:0/3109208247 shutdown_connections 2024-03-26T08:21:21.421 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.704+0000 7f2eae600700 1 --2- 172.21.15.43:0/3109208247 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f2e90038f70 0x7f2e9003b430 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:21.421 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.704+0000 7f2eae600700 1 --2- 172.21.15.43:0/3109208247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ea8107b40 0x7f2ea819b660 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:21.421 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.704+0000 7f2eae600700 1 -- 172.21.15.43:0/3109208247 >> 172.21.15.43:0/3109208247 conn(0x7f2ea81012b0 msgr2=0x7f2ea81021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:21.421 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.704+0000 7f2eae600700 1 -- 172.21.15.43:0/3109208247 shutdown_connections 2024-03-26T08:21:21.421 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:20.704+0000 7f2eae600700 1 -- 172.21.15.43:0/3109208247 wait complete. 2024-03-26T08:21:22.041 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:21 smithi043 ceph-mon[31929]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:21:22.041 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:21 smithi043 ceph-mon[31929]: Saving service mgr spec with placement count:2 2024-03-26T08:21:22.041 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:21 smithi043 ceph-mon[31929]: from='mgr.14120 172.21.15.43:0/1789936270' entity='mgr.x' 2024-03-26T08:21:22.041 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:21 smithi043 ceph-mon[31929]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:21:23.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:22 smithi043 ceph-mon[31929]: from='mgr.14120 172.21.15.43:0/1789936270' entity='mgr.x' 2024-03-26T08:21:23.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:22 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2992159738' entity='client.admin' 2024-03-26T08:21:23.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:22 smithi043 ceph-mon[31929]: from='mgr.14120 172.21.15.43:0/1789936270' entity='mgr.x' 2024-03-26T08:21:23.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.475+0000 7f6121600700 1 Processor -- start 2024-03-26T08:21:23.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.476+0000 7f6121600700 1 -- start start 2024-03-26T08:21:23.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.476+0000 7f6121600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f611c108300 0x7f611c10a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:23.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.476+0000 7f6121600700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611c07aec0 con 0x7f611c108300 2024-03-26T08:21:23.478 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.476+0000 7f611aa00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f611c108300 0x7f611c10a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:23.478 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.477+0000 7f611aa00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f611c108300 0x7f611c10a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48374/0 (socket says 172.21.15.43:48374) 2024-03-26T08:21:23.478 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.477+0000 7f611aa00700 1 -- 172.21.15.43:0/1583567576 learned_addr learned my addr 172.21.15.43:0/1583567576 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:23.478 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.477+0000 7f611aa00700 1 -- 172.21.15.43:0/1583567576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f611c10ac30 con 0x7f611c108300 2024-03-26T08:21:23.478 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.477+0000 7f611aa00700 1 --2- 172.21.15.43:0/1583567576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f611c108300 0x7f611c10a6f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f610c0098a0 tx=0x7f610c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d234b01c395ce5d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:23.478 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.478+0000 7f6119600700 1 -- 172.21.15.43:0/1583567576 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f610c004160 con 0x7f611c108300 2024-03-26T08:21:23.478 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.478+0000 7f6119600700 1 -- 172.21.15.43:0/1583567576 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f610c004300 con 0x7f611c108300 2024-03-26T08:21:23.478 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.478+0000 7f6119600700 1 -- 172.21.15.43:0/1583567576 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f610c011a50 con 0x7f611c108300 2024-03-26T08:21:23.478 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.479+0000 7f6121600700 1 -- 172.21.15.43:0/1583567576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f611c108300 msgr2=0x7f611c10a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:23.478 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.479+0000 7f6121600700 1 --2- 172.21.15.43:0/1583567576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f611c108300 0x7f611c10a6f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f610c0098a0 tx=0x7f610c009bb0 comp rx=0 tx=0).stop 2024-03-26T08:21:23.478 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.479+0000 7f6121600700 1 -- 172.21.15.43:0/1583567576 shutdown_connections 2024-03-26T08:21:23.478 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.479+0000 7f6121600700 1 --2- 172.21.15.43:0/1583567576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f611c108300 0x7f611c10a6f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:23.478 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.479+0000 7f6121600700 1 -- 172.21.15.43:0/1583567576 >> 172.21.15.43:0/1583567576 conn(0x7f611c101e10 msgr2=0x7f611c104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:23.479 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.479+0000 7f6121600700 1 -- 172.21.15.43:0/1583567576 shutdown_connections 2024-03-26T08:21:23.479 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.479+0000 7f6121600700 1 -- 172.21.15.43:0/1583567576 wait complete. 2024-03-26T08:21:23.479 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.480+0000 7f6121600700 1 Processor -- start 2024-03-26T08:21:23.479 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.480+0000 7f6121600700 1 -- start start 2024-03-26T08:21:23.479 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.481+0000 7f6121600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f611c108300 0x7f611c19d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:23.479 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.481+0000 7f6121600700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611c10bc40 con 0x7f611c108300 2024-03-26T08:21:23.479 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.481+0000 7f611aa00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f611c108300 0x7f611c19d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:23.479 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.481+0000 7f611aa00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f611c108300 0x7f611c19d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48384/0 (socket says 172.21.15.43:48384) 2024-03-26T08:21:23.479 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.481+0000 7f611aa00700 1 -- 172.21.15.43:0/2992159738 learned_addr learned my addr 172.21.15.43:0/2992159738 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:23.479 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.481+0000 7f611aa00700 1 -- 172.21.15.43:0/2992159738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f611c19de50 con 0x7f611c108300 2024-03-26T08:21:23.479 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.482+0000 7f611aa00700 1 --2- 172.21.15.43:0/2992159738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f611c108300 0x7f611c19d910 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f610c006b20 tx=0x7f610c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:23.479 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.482+0000 7f6113400700 1 -- 172.21.15.43:0/2992159738 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f610c003b70 con 0x7f611c108300 2024-03-26T08:21:23.479 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.482+0000 7f6113400700 1 -- 172.21.15.43:0/2992159738 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f610c01a8e0 con 0x7f611c108300 2024-03-26T08:21:23.479 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.482+0000 7f6121600700 1 -- 172.21.15.43:0/2992159738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f611c19e0e0 con 0x7f611c108300 2024-03-26T08:21:23.479 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.482+0000 7f6113400700 1 -- 172.21.15.43:0/2992159738 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f610c0249b0 con 0x7f611c108300 2024-03-26T08:21:23.480 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.482+0000 7f6121600700 1 -- 172.21.15.43:0/2992159738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f611c1a0dd0 con 0x7f611c108300 2024-03-26T08:21:23.480 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.483+0000 7f6113400700 1 -- 172.21.15.43:0/2992159738 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f610c022020 con 0x7f611c108300 2024-03-26T08:21:23.480 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.483+0000 7f6121600700 1 -- 172.21.15.43:0/2992159738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f611c0637c0 con 0x7f611c108300 2024-03-26T08:21:23.480 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.484+0000 7f6113400700 1 --2- 172.21.15.43:0/2992159738 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f6100038f20 0x7f610003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:23.480 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.484+0000 7f6113400700 1 -- 172.21.15.43:0/2992159738 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f610c04d6b0 con 0x7f611c108300 2024-03-26T08:21:23.480 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.484+0000 7f611a000700 1 --2- 172.21.15.43:0/2992159738 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f6100038f20 0x7f610003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:23.480 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.485+0000 7f611a000700 1 --2- 172.21.15.43:0/2992159738 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f6100038f20 0x7f610003b3e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6108006fd0 tx=0x7f6108006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:23.480 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.489+0000 7f6113400700 1 -- 172.21.15.43:0/2992159738 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f610c0172b0 con 0x7f611c108300 2024-03-26T08:21:23.480 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.721+0000 7f6121600700 1 -- 172.21.15.43:0/2992159738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f611c1a13a0 con 0x7f611c108300 2024-03-26T08:21:23.480 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.725+0000 7f6113400700 1 -- 172.21.15.43:0/2992159738 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f610c04d000 con 0x7f611c108300 2024-03-26T08:21:23.480 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.728+0000 7f6121600700 1 -- 172.21.15.43:0/2992159738 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f6100038f20 msgr2=0x7f610003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:23.480 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.728+0000 7f6121600700 1 --2- 172.21.15.43:0/2992159738 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f6100038f20 0x7f610003b3e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6108006fd0 tx=0x7f6108006e40 comp rx=0 tx=0).stop 2024-03-26T08:21:23.480 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.728+0000 7f6121600700 1 -- 172.21.15.43:0/2992159738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f611c108300 msgr2=0x7f611c19d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:23.480 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.728+0000 7f6121600700 1 --2- 172.21.15.43:0/2992159738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f611c108300 0x7f611c19d910 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f610c006b20 tx=0x7f610c011ad0 comp rx=0 tx=0).stop 2024-03-26T08:21:23.480 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.728+0000 7f6121600700 1 -- 172.21.15.43:0/2992159738 shutdown_connections 2024-03-26T08:21:23.481 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.728+0000 7f6121600700 1 --2- 172.21.15.43:0/2992159738 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f6100038f20 0x7f610003b3e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:23.481 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.728+0000 7f6121600700 1 --2- 172.21.15.43:0/2992159738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f611c108300 0x7f611c19d910 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:23.481 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.728+0000 7f6121600700 1 -- 172.21.15.43:0/2992159738 >> 172.21.15.43:0/2992159738 conn(0x7f611c101e10 msgr2=0x7f611c1027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:23.481 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.728+0000 7f6121600700 1 -- 172.21.15.43:0/2992159738 shutdown_connections 2024-03-26T08:21:23.481 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:22.728+0000 7f6121600700 1 -- 172.21.15.43:0/2992159738 wait complete. 2024-03-26T08:21:24.092 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:23 smithi043 ceph-mon[31929]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:21:25.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.097+0000 7f9315200700 1 Processor -- start 2024-03-26T08:21:25.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.097+0000 7f9315200700 1 -- start start 2024-03-26T08:21:25.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.097+0000 7f9315200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f931007ce40 0x7f931007b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:25.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.097+0000 7f9315200700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931007b830 con 0x7f931007ce40 2024-03-26T08:21:25.259 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.098+0000 7f930de00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f931007ce40 0x7f931007b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:25.260 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.098+0000 7f930de00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f931007ce40 0x7f931007b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49902/0 (socket says 172.21.15.43:49902) 2024-03-26T08:21:25.260 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.098+0000 7f930de00700 1 -- 172.21.15.43:0/104972961 learned_addr learned my addr 172.21.15.43:0/104972961 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:25.260 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.098+0000 7f930de00700 1 -- 172.21.15.43:0/104972961 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931007b9b0 con 0x7f931007ce40 2024-03-26T08:21:25.260 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.099+0000 7f930de00700 1 --2- 172.21.15.43:0/104972961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f931007ce40 0x7f931007b2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9300009e80 tx=0x7f930000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9513e89847c9aa0b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:25.260 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.099+0000 7f930d400700 1 -- 172.21.15.43:0/104972961 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9300017070 con 0x7f931007ce40 2024-03-26T08:21:25.260 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.099+0000 7f930d400700 1 -- 172.21.15.43:0/104972961 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f930000b8c0 con 0x7f931007ce40 2024-03-26T08:21:25.260 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.099+0000 7f930d400700 1 -- 172.21.15.43:0/104972961 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9300004490 con 0x7f931007ce40 2024-03-26T08:21:25.260 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.100+0000 7f9315200700 1 -- 172.21.15.43:0/104972961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f931007ce40 msgr2=0x7f931007b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:25.260 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.100+0000 7f9315200700 1 --2- 172.21.15.43:0/104972961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f931007ce40 0x7f931007b2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9300009e80 tx=0x7f930000b330 comp rx=0 tx=0).stop 2024-03-26T08:21:25.260 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.100+0000 7f9315200700 1 -- 172.21.15.43:0/104972961 shutdown_connections 2024-03-26T08:21:25.260 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.100+0000 7f9315200700 1 --2- 172.21.15.43:0/104972961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f931007ce40 0x7f931007b2f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:25.262 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.100+0000 7f9315200700 1 -- 172.21.15.43:0/104972961 >> 172.21.15.43:0/104972961 conn(0x7f9310102f00 msgr2=0x7f9310105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:25.262 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.100+0000 7f9315200700 1 -- 172.21.15.43:0/104972961 shutdown_connections 2024-03-26T08:21:25.262 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.100+0000 7f9315200700 1 -- 172.21.15.43:0/104972961 wait complete. 2024-03-26T08:21:25.262 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.101+0000 7f9315200700 1 Processor -- start 2024-03-26T08:21:25.262 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.101+0000 7f9315200700 1 -- start start 2024-03-26T08:21:25.262 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.102+0000 7f9315200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f931007ce40 0x7f931019d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:25.262 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.102+0000 7f9315200700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931010de20 con 0x7f931007ce40 2024-03-26T08:21:25.262 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.102+0000 7f930de00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f931007ce40 0x7f931019d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:25.262 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.102+0000 7f930de00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f931007ce40 0x7f931019d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49916/0 (socket says 172.21.15.43:49916) 2024-03-26T08:21:25.263 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.102+0000 7f930de00700 1 -- 172.21.15.43:0/1088690488 learned_addr learned my addr 172.21.15.43:0/1088690488 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:25.263 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.103+0000 7f930de00700 1 -- 172.21.15.43:0/1088690488 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931019de60 con 0x7f931007ce40 2024-03-26T08:21:25.263 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.103+0000 7f930de00700 1 --2- 172.21.15.43:0/1088690488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f931007ce40 0x7f931019d920 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9300000f80 tx=0x7f9300004300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:25.263 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.103+0000 7f9307400700 1 -- 172.21.15.43:0/1088690488 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9300020020 con 0x7f931007ce40 2024-03-26T08:21:25.265 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.103+0000 7f9307400700 1 -- 172.21.15.43:0/1088690488 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9300012430 con 0x7f931007ce40 2024-03-26T08:21:25.265 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.103+0000 7f9307400700 1 -- 172.21.15.43:0/1088690488 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9300017040 con 0x7f931007ce40 2024-03-26T08:21:25.265 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.103+0000 7f9315200700 1 -- 172.21.15.43:0/1088690488 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931019e0f0 con 0x7f931007ce40 2024-03-26T08:21:25.265 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.104+0000 7f9315200700 1 -- 172.21.15.43:0/1088690488 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93101a0de0 con 0x7f931007ce40 2024-03-26T08:21:25.265 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.105+0000 7f9307400700 1 -- 172.21.15.43:0/1088690488 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f9300023020 con 0x7f931007ce40 2024-03-26T08:21:25.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.105+0000 7f9307400700 1 --2- 172.21.15.43:0/1088690488 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f92f4038f70 0x7f92f403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:25.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.105+0000 7f9307400700 1 -- 172.21.15.43:0/1088690488 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f930000f050 con 0x7f931007ce40 2024-03-26T08:21:25.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.106+0000 7f9305400700 1 --2- 172.21.15.43:0/1088690488 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f92f4038f70 0x7f92f403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:25.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.105+0000 7f9315200700 1 -- 172.21.15.43:0/1088690488 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93100637c0 con 0x7f931007ce40 2024-03-26T08:21:25.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.107+0000 7f9305400700 1 --2- 172.21.15.43:0/1088690488 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f92f4038f70 0x7f92f403b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f92f8006fd0 tx=0x7f92f8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:25.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.113+0000 7f9307400700 1 -- 172.21.15.43:0/1088690488 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f930002a020 con 0x7f931007ce40 2024-03-26T08:21:25.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.342+0000 7f9315200700 1 -- 172.21.15.43:0/1088690488 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f93101a0fb0 con 0x7f931007ce40 2024-03-26T08:21:25.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.346+0000 7f9307400700 1 -- 172.21.15.43:0/1088690488 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f93101a0fb0 con 0x7f931007ce40 2024-03-26T08:21:25.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.349+0000 7f9315200700 1 -- 172.21.15.43:0/1088690488 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f92f4038f70 msgr2=0x7f92f403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:25.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.349+0000 7f9315200700 1 --2- 172.21.15.43:0/1088690488 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f92f4038f70 0x7f92f403b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f92f8006fd0 tx=0x7f92f8006e40 comp rx=0 tx=0).stop 2024-03-26T08:21:25.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.349+0000 7f9315200700 1 -- 172.21.15.43:0/1088690488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f931007ce40 msgr2=0x7f931019d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:25.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.349+0000 7f9315200700 1 --2- 172.21.15.43:0/1088690488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f931007ce40 0x7f931019d920 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9300000f80 tx=0x7f9300004300 comp rx=0 tx=0).stop 2024-03-26T08:21:25.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.349+0000 7f9315200700 1 -- 172.21.15.43:0/1088690488 shutdown_connections 2024-03-26T08:21:25.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.349+0000 7f9315200700 1 --2- 172.21.15.43:0/1088690488 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f92f4038f70 0x7f92f403b430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:25.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.349+0000 7f9315200700 1 --2- 172.21.15.43:0/1088690488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f931007ce40 0x7f931019d920 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:25.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.349+0000 7f9315200700 1 -- 172.21.15.43:0/1088690488 >> 172.21.15.43:0/1088690488 conn(0x7f9310102f00 msgr2=0x7f9310103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:25.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.350+0000 7f9315200700 1 -- 172.21.15.43:0/1088690488 shutdown_connections 2024-03-26T08:21:25.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:24.350+0000 7f9315200700 1 -- 172.21.15.43:0/1088690488 wait complete. 2024-03-26T08:21:25.267 INFO:teuthology.orchestra.run.smithi043.stdout:Enabling the dashboard module... 2024-03-26T08:21:25.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:25 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1088690488' entity='client.admin' 2024-03-26T08:21:25.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:25 smithi043 ceph-mon[31929]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:21:27.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:26 smithi043 ceph-mon[31929]: from='mgr.14120 172.21.15.43:0/1789936270' entity='mgr.x' 2024-03-26T08:21:27.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:26 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3844345088' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-26T08:21:27.401 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:27 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: ignoring --setuser ceph since I am not root 2024-03-26T08:21:27.401 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:27 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: ignoring --setgroup ceph since I am not root 2024-03-26T08:21:27.782 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.221+0000 7fb719800700 1 Processor -- start 2024-03-26T08:21:27.782 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.221+0000 7fb719800700 1 -- start start 2024-03-26T08:21:27.782 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.222+0000 7fb719800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7141078c0 0x7fb714107ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:27.782 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.222+0000 7fb719800700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7141082b0 con 0x7fb7141078c0 2024-03-26T08:21:27.782 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.222+0000 7fb712a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7141078c0 0x7fb714107ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:27.783 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.222+0000 7fb712a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7141078c0 0x7fb714107ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49932/0 (socket says 172.21.15.43:49932) 2024-03-26T08:21:27.783 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.222+0000 7fb712a00700 1 -- 172.21.15.43:0/1535956093 learned_addr learned my addr 172.21.15.43:0/1535956093 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:27.783 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.223+0000 7fb712a00700 1 -- 172.21.15.43:0/1535956093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb714108a70 con 0x7fb7141078c0 2024-03-26T08:21:27.783 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.223+0000 7fb712a00700 1 --2- 172.21.15.43:0/1535956093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7141078c0 0x7fb714107ce0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb7040098a0 tx=0x7fb704009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4883766cbb9b11d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:27.783 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.223+0000 7fb711600700 1 -- 172.21.15.43:0/1535956093 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb704004160 con 0x7fb7141078c0 2024-03-26T08:21:27.783 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.223+0000 7fb711600700 1 -- 172.21.15.43:0/1535956093 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb704004300 con 0x7fb7141078c0 2024-03-26T08:21:27.784 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.223+0000 7fb711600700 1 -- 172.21.15.43:0/1535956093 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb704011a50 con 0x7fb7141078c0 2024-03-26T08:21:27.784 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.224+0000 7fb719800700 1 -- 172.21.15.43:0/1535956093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7141078c0 msgr2=0x7fb714107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:27.784 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.224+0000 7fb719800700 1 --2- 172.21.15.43:0/1535956093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7141078c0 0x7fb714107ce0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb7040098a0 tx=0x7fb704009bb0 comp rx=0 tx=0).stop 2024-03-26T08:21:27.784 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.224+0000 7fb719800700 1 -- 172.21.15.43:0/1535956093 shutdown_connections 2024-03-26T08:21:27.784 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.224+0000 7fb719800700 1 --2- 172.21.15.43:0/1535956093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7141078c0 0x7fb714107ce0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:27.784 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.224+0000 7fb719800700 1 -- 172.21.15.43:0/1535956093 >> 172.21.15.43:0/1535956093 conn(0x7fb714102f00 msgr2=0x7fb714105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:27.784 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.225+0000 7fb719800700 1 -- 172.21.15.43:0/1535956093 shutdown_connections 2024-03-26T08:21:27.785 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.225+0000 7fb719800700 1 -- 172.21.15.43:0/1535956093 wait complete. 2024-03-26T08:21:27.785 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.226+0000 7fb719800700 1 Processor -- start 2024-03-26T08:21:27.785 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.226+0000 7fb719800700 1 -- start start 2024-03-26T08:21:27.785 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.226+0000 7fb719800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7141078c0 0x7fb71419b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:27.785 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.226+0000 7fb719800700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71410be20 con 0x7fb7141078c0 2024-03-26T08:21:27.785 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.226+0000 7fb712a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7141078c0 0x7fb71419b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:27.786 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.226+0000 7fb712a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7141078c0 0x7fb71419b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49936/0 (socket says 172.21.15.43:49936) 2024-03-26T08:21:27.786 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.226+0000 7fb712a00700 1 -- 172.21.15.43:0/3844345088 learned_addr learned my addr 172.21.15.43:0/3844345088 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:27.786 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.227+0000 7fb712a00700 1 -- 172.21.15.43:0/3844345088 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb71419bc70 con 0x7fb7141078c0 2024-03-26T08:21:27.786 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.227+0000 7fb712a00700 1 --2- 172.21.15.43:0/3844345088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7141078c0 0x7fb71419b730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb704006b20 tx=0x7fb704011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:27.786 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.227+0000 7fb70b400700 1 -- 172.21.15.43:0/3844345088 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7040039e0 con 0x7fb7141078c0 2024-03-26T08:21:27.786 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.227+0000 7fb70b400700 1 -- 172.21.15.43:0/3844345088 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb70401a8e0 con 0x7fb7141078c0 2024-03-26T08:21:27.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.228+0000 7fb719800700 1 -- 172.21.15.43:0/3844345088 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb71419bf00 con 0x7fb7141078c0 2024-03-26T08:21:27.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.228+0000 7fb70b400700 1 -- 172.21.15.43:0/3844345088 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7040247e0 con 0x7fb7141078c0 2024-03-26T08:21:27.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.228+0000 7fb719800700 1 -- 172.21.15.43:0/3844345088 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb71419ebf0 con 0x7fb7141078c0 2024-03-26T08:21:27.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.229+0000 7fb70b400700 1 -- 172.21.15.43:0/3844345088 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fb704022020 con 0x7fb7141078c0 2024-03-26T08:21:27.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.229+0000 7fb719800700 1 -- 172.21.15.43:0/3844345088 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7140637c0 con 0x7fb7141078c0 2024-03-26T08:21:27.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.230+0000 7fb70b400700 1 --2- 172.21.15.43:0/3844345088 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7fb6f8038f70 0x7fb6f803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:27.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.230+0000 7fb70b400700 1 -- 172.21.15.43:0/3844345088 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb70404d210 con 0x7fb7141078c0 2024-03-26T08:21:27.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.230+0000 7fb712000700 1 --2- 172.21.15.43:0/3844345088 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7fb6f8038f70 0x7fb6f803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:27.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.231+0000 7fb712000700 1 --2- 172.21.15.43:0/3844345088 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7fb6f8038f70 0x7fb6f803b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb700006fd0 tx=0x7fb700006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:27.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.236+0000 7fb70b400700 1 -- 172.21.15.43:0/3844345088 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb70402c3a0 con 0x7fb7141078c0 2024-03-26T08:21:27.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.526+0000 7fb719800700 1 -- 172.21.15.43:0/3844345088 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fb71419f1a0 con 0x7fb7141078c0 2024-03-26T08:21:27.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.928+0000 7fb70b400700 1 -- 172.21.15.43:0/3844345088 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fb704016260 con 0x7fb7141078c0 2024-03-26T08:21:27.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.932+0000 7fb70b400700 1 -- 172.21.15.43:0/3844345088 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7fb70404f0e0 con 0x7fb7141078c0 2024-03-26T08:21:27.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.934+0000 7fb719800700 1 -- 172.21.15.43:0/3844345088 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7fb6f8038f70 msgr2=0x7fb6f803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:27.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.934+0000 7fb719800700 1 --2- 172.21.15.43:0/3844345088 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7fb6f8038f70 0x7fb6f803b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb700006fd0 tx=0x7fb700006e40 comp rx=0 tx=0).stop 2024-03-26T08:21:27.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.934+0000 7fb719800700 1 -- 172.21.15.43:0/3844345088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7141078c0 msgr2=0x7fb71419b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:27.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.934+0000 7fb719800700 1 --2- 172.21.15.43:0/3844345088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7141078c0 0x7fb71419b730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb704006b20 tx=0x7fb704011ad0 comp rx=0 tx=0).stop 2024-03-26T08:21:27.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.934+0000 7fb719800700 1 -- 172.21.15.43:0/3844345088 shutdown_connections 2024-03-26T08:21:27.789 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.934+0000 7fb719800700 1 --2- 172.21.15.43:0/3844345088 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7fb6f8038f70 0x7fb6f803b430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:27.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.934+0000 7fb719800700 1 --2- 172.21.15.43:0/3844345088 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7141078c0 0x7fb71419b730 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:27.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.934+0000 7fb719800700 1 -- 172.21.15.43:0/3844345088 >> 172.21.15.43:0/3844345088 conn(0x7fb714102f00 msgr2=0x7fb714103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:27.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.934+0000 7fb719800700 1 -- 172.21.15.43:0/3844345088 shutdown_connections 2024-03-26T08:21:27.790 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:26.935+0000 7fb719800700 1 -- 172.21.15.43:0/3844345088 wait complete. 2024-03-26T08:21:28.067 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:27 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3844345088' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-26T08:21:28.067 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:27 smithi043 ceph-mon[31929]: mgrmap e9: x(active, since 25s) 2024-03-26T08:21:28.067 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:27 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:27.787+0000 7fbf8235a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T08:21:28.067 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:27 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:27.915+0000 7fbf8235a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T08:21:28.904 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:28 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:28.746+0000 7fbf8235a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T08:21:28.904 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:28 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:28.871+0000 7fbf8235a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T08:21:29.195 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:28 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2590710168' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-26T08:21:29.196 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:29 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:29.071+0000 7fbf8235a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T08:21:29.196 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:29 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:29.200+0000 7fbf8235a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T08:21:29.307 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-03-26T08:21:29.307 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-26T08:21:29.308 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-26T08:21:29.308 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-26T08:21:29.308 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-26T08:21:29.308 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-03-26T08:21:29.309 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.344+0000 7f0a37200700 1 Processor -- start 2024-03-26T08:21:29.309 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.344+0000 7f0a37200700 1 -- start start 2024-03-26T08:21:29.309 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.344+0000 7f0a37200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3807ce40 0x7f0a3807b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:29.309 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.344+0000 7f0a37200700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3807b830 con 0x7f0a3807ce40 2024-03-26T08:21:29.310 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.345+0000 7f0a35e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3807ce40 0x7f0a3807b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:29.310 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.345+0000 7f0a35e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3807ce40 0x7f0a3807b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49978/0 (socket says 172.21.15.43:49978) 2024-03-26T08:21:29.310 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.345+0000 7f0a35e00700 1 -- 172.21.15.43:0/1060109221 learned_addr learned my addr 172.21.15.43:0/1060109221 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:29.310 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.346+0000 7f0a35e00700 1 -- 172.21.15.43:0/1060109221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3807b9b0 con 0x7f0a3807ce40 2024-03-26T08:21:29.310 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.346+0000 7f0a35e00700 1 --2- 172.21.15.43:0/1060109221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3807ce40 0x7f0a3807b2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0a280098a0 tx=0x7f0a28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c721144e7cf766f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:29.314 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.346+0000 7f0a34a00700 1 -- 172.21.15.43:0/1060109221 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a28004160 con 0x7f0a3807ce40 2024-03-26T08:21:29.315 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.346+0000 7f0a34a00700 1 -- 172.21.15.43:0/1060109221 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a28004300 con 0x7f0a3807ce40 2024-03-26T08:21:29.315 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.346+0000 7f0a34a00700 1 -- 172.21.15.43:0/1060109221 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a28011a50 con 0x7f0a3807ce40 2024-03-26T08:21:29.315 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.347+0000 7f0a37200700 1 -- 172.21.15.43:0/1060109221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3807ce40 msgr2=0x7f0a3807b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:29.315 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.347+0000 7f0a37200700 1 --2- 172.21.15.43:0/1060109221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3807ce40 0x7f0a3807b2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0a280098a0 tx=0x7f0a28009bb0 comp rx=0 tx=0).stop 2024-03-26T08:21:29.315 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.347+0000 7f0a37200700 1 -- 172.21.15.43:0/1060109221 shutdown_connections 2024-03-26T08:21:29.315 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.347+0000 7f0a37200700 1 --2- 172.21.15.43:0/1060109221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3807ce40 0x7f0a3807b2f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:29.315 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.347+0000 7f0a37200700 1 -- 172.21.15.43:0/1060109221 >> 172.21.15.43:0/1060109221 conn(0x7f0a38102d00 msgr2=0x7f0a38105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:29.315 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.347+0000 7f0a37200700 1 -- 172.21.15.43:0/1060109221 shutdown_connections 2024-03-26T08:21:29.315 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.347+0000 7f0a37200700 1 -- 172.21.15.43:0/1060109221 wait complete. 2024-03-26T08:21:29.315 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.348+0000 7f0a37200700 1 Processor -- start 2024-03-26T08:21:29.315 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.349+0000 7f0a37200700 1 -- start start 2024-03-26T08:21:29.315 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.349+0000 7f0a37200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3807ce40 0x7f0a3819d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:29.316 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.349+0000 7f0a37200700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3810dc20 con 0x7f0a3807ce40 2024-03-26T08:21:29.316 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.349+0000 7f0a35e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3807ce40 0x7f0a3819d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:29.316 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.349+0000 7f0a35e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3807ce40 0x7f0a3819d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49984/0 (socket says 172.21.15.43:49984) 2024-03-26T08:21:29.316 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.349+0000 7f0a35e00700 1 -- 172.21.15.43:0/2590710168 learned_addr learned my addr 172.21.15.43:0/2590710168 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:29.316 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.350+0000 7f0a35e00700 1 -- 172.21.15.43:0/2590710168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3819dc60 con 0x7f0a3807ce40 2024-03-26T08:21:29.316 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.350+0000 7f0a35e00700 1 --2- 172.21.15.43:0/2590710168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3807ce40 0x7f0a3819d720 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0a28006b20 tx=0x7f0a28011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:29.316 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.350+0000 7f0a2ea00700 1 -- 172.21.15.43:0/2590710168 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a2801a430 con 0x7f0a3807ce40 2024-03-26T08:21:29.316 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.350+0000 7f0a2ea00700 1 -- 172.21.15.43:0/2590710168 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a2801aa80 con 0x7f0a3807ce40 2024-03-26T08:21:29.316 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.350+0000 7f0a2ea00700 1 -- 172.21.15.43:0/2590710168 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a28024a20 con 0x7f0a3807ce40 2024-03-26T08:21:29.316 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.350+0000 7f0a37200700 1 -- 172.21.15.43:0/2590710168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3819def0 con 0x7f0a3807ce40 2024-03-26T08:21:29.316 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.351+0000 7f0a37200700 1 -- 172.21.15.43:0/2590710168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a381a0be0 con 0x7f0a3807ce40 2024-03-26T08:21:29.316 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.351+0000 7f0a37200700 1 -- 172.21.15.43:0/2590710168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a380637c0 con 0x7f0a3807ce40 2024-03-26T08:21:29.316 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.352+0000 7f0a2ea00700 1 -- 172.21.15.43:0/2590710168 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f0a28022020 con 0x7f0a3807ce40 2024-03-26T08:21:29.316 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.352+0000 7f0a2ea00700 1 --2- 172.21.15.43:0/2590710168 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f0a1c038fd0 0x7f0a1c03b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:29.316 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.352+0000 7f0a2ea00700 1 -- 172.21.15.43:0/2590710168 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0a2804d2f0 con 0x7f0a3807ce40 2024-03-26T08:21:29.317 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.352+0000 7f0a35400700 1 -- 172.21.15.43:0/2590710168 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f0a1c038fd0 msgr2=0x7f0a1c03b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/2446912488 2024-03-26T08:21:29.317 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.353+0000 7f0a35400700 1 --2- 172.21.15.43:0/2590710168 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f0a1c038fd0 0x7f0a1c03b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T08:21:29.317 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.358+0000 7f0a2ea00700 1 -- 172.21.15.43:0/2590710168 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0a28016260 con 0x7f0a3807ce40 2024-03-26T08:21:29.319 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.554+0000 7f0a35400700 1 -- 172.21.15.43:0/2590710168 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f0a1c038fd0 msgr2=0x7f0a1c03b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/2446912488 2024-03-26T08:21:29.319 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.554+0000 7f0a35400700 1 --2- 172.21.15.43:0/2590710168 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f0a1c038fd0 0x7f0a1c03b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T08:21:29.319 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.640+0000 7f0a37200700 1 -- 172.21.15.43:0/2590710168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f0a381a0e50 con 0x7f0a3807ce40 2024-03-26T08:21:29.319 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.641+0000 7f0a2ea00700 1 -- 172.21.15.43:0/2590710168 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f0a2800be50 con 0x7f0a3807ce40 2024-03-26T08:21:29.319 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.644+0000 7f0a37200700 1 -- 172.21.15.43:0/2590710168 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f0a1c038fd0 msgr2=0x7f0a1c03b490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T08:21:29.319 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.644+0000 7f0a37200700 1 --2- 172.21.15.43:0/2590710168 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f0a1c038fd0 0x7f0a1c03b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:29.319 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.644+0000 7f0a37200700 1 -- 172.21.15.43:0/2590710168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3807ce40 msgr2=0x7f0a3819d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:29.319 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.644+0000 7f0a37200700 1 --2- 172.21.15.43:0/2590710168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3807ce40 0x7f0a3819d720 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0a28006b20 tx=0x7f0a28011ad0 comp rx=0 tx=0).stop 2024-03-26T08:21:29.319 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.644+0000 7f0a37200700 1 -- 172.21.15.43:0/2590710168 shutdown_connections 2024-03-26T08:21:29.319 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.644+0000 7f0a37200700 1 --2- 172.21.15.43:0/2590710168 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f0a1c038fd0 0x7f0a1c03b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:29.319 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.644+0000 7f0a37200700 1 --2- 172.21.15.43:0/2590710168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a3807ce40 0x7f0a3819d720 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:29.319 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.644+0000 7f0a37200700 1 -- 172.21.15.43:0/2590710168 >> 172.21.15.43:0/2590710168 conn(0x7f0a38102d00 msgr2=0x7f0a381037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:29.319 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.644+0000 7f0a37200700 1 -- 172.21.15.43:0/2590710168 shutdown_connections 2024-03-26T08:21:29.319 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:28.644+0000 7f0a37200700 1 -- 172.21.15.43:0/2590710168 wait complete. 2024-03-26T08:21:29.320 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for the mgr to restart... 2024-03-26T08:21:29.320 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr epoch 9... 2024-03-26T08:21:29.474 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:29 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:29.317+0000 7fbf8235a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T08:21:29.789 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:29 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:29.670+0000 7fbf8235a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T08:21:30.061 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:29 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:29.818+0000 7fbf8235a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T08:21:30.062 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:29 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:29.937+0000 7fbf8235a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T08:21:30.062 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:30 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:30.066+0000 7fbf8235a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T08:21:30.404 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:30 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:30.291+0000 7fbf8235a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T08:21:31.654 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:31 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:31.605+0000 7fbf8235a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T08:21:32.154 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:31 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:31.783+0000 7fbf8235a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T08:21:32.595 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:32 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:32.212+0000 7fbf8235a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T08:21:32.595 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:32 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:32.335+0000 7fbf8235a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T08:21:32.596 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:32 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:32.599+0000 7fbf8235a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T08:21:33.154 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:32 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:32.853+0000 7fbf8235a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T08:21:33.526 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:33 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:33.244+0000 7fbf8235a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T08:21:33.822 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:33 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:33.574+0000 7fbf8235a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T08:21:34.154 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:33 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:33.898+0000 7fbf8235a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T08:21:35.154 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:35 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:34.998+0000 7fbf8235a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T08:21:36.140 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:35 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:35.718+0000 7fbf8235a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T08:21:36.140 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:35 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:35.852+0000 7fbf8235a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T08:21:36.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:36 smithi043 ceph-mon[31929]: Active manager daemon x restarted 2024-03-26T08:21:36.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:36 smithi043 ceph-mon[31929]: Activating manager daemon x 2024-03-26T08:21:36.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:36 smithi043 ceph-mon[31929]: osdmap e3: 0 total, 0 up, 0 in 2024-03-26T08:21:36.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:36 smithi043 ceph-mon[31929]: mgrmap e10: x(active, starting, since 0.00952617s) 2024-03-26T08:21:36.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:36 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:21:36.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:36 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T08:21:36.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:36 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T08:21:36.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:36 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T08:21:36.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:36 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T08:21:36.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:36 smithi043 ceph-mon[31929]: Manager daemon x is now available 2024-03-26T08:21:36.405 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:21:36 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:21:36.144+0000 7fbf8235a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T08:21:37.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:37 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:21:37.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:37 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:21:37.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:37 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-26T08:21:37.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:37 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-26T08:21:37.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:37 smithi043 ceph-mon[31929]: [26/Mar/2024:08:21:36] ENGINE Bus STARTING 2024-03-26T08:21:37.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:37 smithi043 ceph-mon[31929]: [26/Mar/2024:08:21:37] ENGINE Serving on http://172.21.15.43:8765 2024-03-26T08:21:37.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:37 smithi043 ceph-mon[31929]: mgrmap e11: x(active, since 1.01028s) 2024-03-26T08:21:37.838 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-03-26T08:21:37.838 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-26T08:21:37.838 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-26T08:21:37.839 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-03-26T08:21:37.839 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.950+0000 7f2280600700 1 Processor -- start 2024-03-26T08:21:37.839 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.950+0000 7f2280600700 1 -- start start 2024-03-26T08:21:37.839 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.950+0000 7f2280600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22781060f0 0x7f2278106510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:37.839 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.950+0000 7f2280600700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227807aec0 con 0x7f22781060f0 2024-03-26T08:21:37.840 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.951+0000 7f227dc00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22781060f0 0x7f2278106510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:37.840 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.951+0000 7f227dc00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22781060f0 0x7f2278106510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49990/0 (socket says 172.21.15.43:49990) 2024-03-26T08:21:37.840 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.951+0000 7f227dc00700 1 -- 172.21.15.43:0/1241610149 learned_addr learned my addr 172.21.15.43:0/1241610149 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:37.840 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.952+0000 7f227dc00700 1 -- 172.21.15.43:0/1241610149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2278106a50 con 0x7f22781060f0 2024-03-26T08:21:37.841 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.952+0000 7f227dc00700 1 --2- 172.21.15.43:0/1241610149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22781060f0 0x7f2278106510 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f226c0098a0 tx=0x7f226c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa88e6fa79d40f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:37.841 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.952+0000 7f2277e00700 1 -- 172.21.15.43:0/1241610149 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f226c004160 con 0x7f22781060f0 2024-03-26T08:21:37.841 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.952+0000 7f2277e00700 1 -- 172.21.15.43:0/1241610149 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f226c004300 con 0x7f22781060f0 2024-03-26T08:21:37.841 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.952+0000 7f2277e00700 1 -- 172.21.15.43:0/1241610149 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f226c011a50 con 0x7f22781060f0 2024-03-26T08:21:37.841 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.953+0000 7f2280600700 1 -- 172.21.15.43:0/1241610149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22781060f0 msgr2=0x7f2278106510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:37.841 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.953+0000 7f2280600700 1 --2- 172.21.15.43:0/1241610149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22781060f0 0x7f2278106510 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f226c0098a0 tx=0x7f226c009bb0 comp rx=0 tx=0).stop 2024-03-26T08:21:37.841 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.953+0000 7f2280600700 1 -- 172.21.15.43:0/1241610149 shutdown_connections 2024-03-26T08:21:37.843 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.953+0000 7f2280600700 1 --2- 172.21.15.43:0/1241610149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22781060f0 0x7f2278106510 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:37.843 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.953+0000 7f2280600700 1 -- 172.21.15.43:0/1241610149 >> 172.21.15.43:0/1241610149 conn(0x7f2278101e10 msgr2=0x7f2278104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:37.843 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.953+0000 7f2280600700 1 -- 172.21.15.43:0/1241610149 shutdown_connections 2024-03-26T08:21:37.843 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.953+0000 7f2280600700 1 -- 172.21.15.43:0/1241610149 wait complete. 2024-03-26T08:21:37.843 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.955+0000 7f2280600700 1 Processor -- start 2024-03-26T08:21:37.843 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.955+0000 7f2280600700 1 -- start start 2024-03-26T08:21:37.843 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.955+0000 7f2280600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22781060f0 0x7f227819d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:37.843 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.955+0000 7f2280600700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2278107ac0 con 0x7f22781060f0 2024-03-26T08:21:37.843 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.955+0000 7f227dc00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22781060f0 0x7f227819d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:37.843 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.955+0000 7f227dc00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22781060f0 0x7f227819d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50004/0 (socket says 172.21.15.43:50004) 2024-03-26T08:21:37.843 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.955+0000 7f227dc00700 1 -- 172.21.15.43:0/2929403211 learned_addr learned my addr 172.21.15.43:0/2929403211 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:37.845 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.956+0000 7f227dc00700 1 -- 172.21.15.43:0/2929403211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227819de10 con 0x7f22781060f0 2024-03-26T08:21:37.845 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.956+0000 7f227dc00700 1 --2- 172.21.15.43:0/2929403211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22781060f0 0x7f227819d8d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f226c011a20 tx=0x7f226c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:37.845 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.956+0000 7f2276000700 1 -- 172.21.15.43:0/2929403211 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f226c003b70 con 0x7f22781060f0 2024-03-26T08:21:37.845 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.956+0000 7f2276000700 1 -- 172.21.15.43:0/2929403211 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f226c01a8e0 con 0x7f22781060f0 2024-03-26T08:21:37.845 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.956+0000 7f2276000700 1 -- 172.21.15.43:0/2929403211 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f226c024990 con 0x7f22781060f0 2024-03-26T08:21:37.846 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.956+0000 7f2280600700 1 -- 172.21.15.43:0/2929403211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f227819e0a0 con 0x7f22781060f0 2024-03-26T08:21:37.846 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.957+0000 7f2280600700 1 -- 172.21.15.43:0/2929403211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22781a0d90 con 0x7f22781060f0 2024-03-26T08:21:37.846 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.958+0000 7f2276000700 1 -- 172.21.15.43:0/2929403211 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f226c022020 con 0x7f22781060f0 2024-03-26T08:21:37.846 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.958+0000 7f2276000700 1 --2- 172.21.15.43:0/2929403211 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f2260038f80 0x7f226003b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:37.846 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.958+0000 7f2276000700 1 -- 172.21.15.43:0/2929403211 --> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f226003bb50 con 0x7f2260038f80 2024-03-26T08:21:37.846 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.958+0000 7f2276000700 1 -- 172.21.15.43:0/2929403211 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f226c04d3a0 con 0x7f22781060f0 2024-03-26T08:21:37.846 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.958+0000 7f227d200700 1 -- 172.21.15.43:0/2929403211 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f2260038f80 msgr2=0x7f226003b440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/2446912488 2024-03-26T08:21:37.846 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:29.959+0000 7f227d200700 1 --2- 172.21.15.43:0/2929403211 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f2260038f80 0x7f226003b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T08:21:37.846 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:30.160+0000 7f227d200700 1 -- 172.21.15.43:0/2929403211 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f2260038f80 msgr2=0x7f226003b440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/2446912488 2024-03-26T08:21:37.846 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:30.160+0000 7f227d200700 1 --2- 172.21.15.43:0/2929403211 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f2260038f80 0x7f226003b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T08:21:37.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:30.561+0000 7f227d200700 1 -- 172.21.15.43:0/2929403211 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f2260038f80 msgr2=0x7f226003b440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/2446912488 2024-03-26T08:21:37.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:30.561+0000 7f227d200700 1 --2- 172.21.15.43:0/2929403211 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f2260038f80 0x7f226003b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-26T08:21:37.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:31.362+0000 7f227d200700 1 -- 172.21.15.43:0/2929403211 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f2260038f80 msgr2=0x7f226003b440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/2446912488 2024-03-26T08:21:37.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:31.362+0000 7f227d200700 1 --2- 172.21.15.43:0/2929403211 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f2260038f80 0x7f226003b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-26T08:21:37.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:32.964+0000 7f227d200700 1 -- 172.21.15.43:0/2929403211 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f2260038f80 msgr2=0x7f226003b440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/2446912488 2024-03-26T08:21:37.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:32.964+0000 7f227d200700 1 --2- 172.21.15.43:0/2929403211 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f2260038f80 0x7f226003b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-26T08:21:37.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:36.154+0000 7f2276000700 1 -- 172.21.15.43:0/2929403211 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mgrmap(e 10) v1 ==== 45886+0+0 (secure 0 0 0) 0x7f226c01a430 con 0x7f22781060f0 2024-03-26T08:21:37.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:36.154+0000 7f2276000700 1 -- 172.21.15.43:0/2929403211 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f2260038f80 msgr2=0x7f226003b440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T08:21:37.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:36.154+0000 7f2276000700 1 --2- 172.21.15.43:0/2929403211 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f2260038f80 0x7f226003b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:37.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:37.156+0000 7f2276000700 1 -- 172.21.15.43:0/2929403211 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 11) v1 ==== 46013+0+0 (secure 0 0 0) 0x7f226c04f2e0 con 0x7f22781060f0 2024-03-26T08:21:37.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:37.156+0000 7f2276000700 1 --2- 172.21.15.43:0/2929403211 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f226003ca30 0x7f226003ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:37.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:37.156+0000 7f2276000700 1 -- 172.21.15.43:0/2929403211 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f226003bb50 con 0x7f226003ca30 2024-03-26T08:21:37.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:37.157+0000 7f227d200700 1 --2- 172.21.15.43:0/2929403211 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f226003ca30 0x7f226003ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:37.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:37.159+0000 7f227d200700 1 --2- 172.21.15.43:0/2929403211 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f226003ca30 0x7f226003ee20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2264003a10 tx=0x7f2264009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:37.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:37.159+0000 7f2276000700 1 -- 172.21.15.43:0/2929403211 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f226003bb50 con 0x7f226003ca30 2024-03-26T08:21:37.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:37.166+0000 7f2280600700 1 -- 172.21.15.43:0/2929403211 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f22780637c0 con 0x7f226003ca30 2024-03-26T08:21:37.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:37.167+0000 7f2276000700 1 -- 172.21.15.43:0/2929403211 <== mgr.14152 v2:172.21.15.43:6800/3390816575 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f22780637c0 con 0x7f226003ca30 2024-03-26T08:21:37.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:37.167+0000 7f2280600700 1 -- 172.21.15.43:0/2929403211 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f226003ca30 msgr2=0x7f226003ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:37.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:37.167+0000 7f2280600700 1 --2- 172.21.15.43:0/2929403211 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f226003ca30 0x7f226003ee20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2264003a10 tx=0x7f2264009270 comp rx=0 tx=0).stop 2024-03-26T08:21:37.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:37.167+0000 7f2280600700 1 -- 172.21.15.43:0/2929403211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22781060f0 msgr2=0x7f227819d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:37.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:37.167+0000 7f2280600700 1 --2- 172.21.15.43:0/2929403211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22781060f0 0x7f227819d8d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f226c011a20 tx=0x7f226c011b00 comp rx=0 tx=0).stop 2024-03-26T08:21:37.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:37.167+0000 7f2280600700 1 -- 172.21.15.43:0/2929403211 shutdown_connections 2024-03-26T08:21:37.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:37.167+0000 7f2280600700 1 --2- 172.21.15.43:0/2929403211 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f226003ca30 0x7f226003ee20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:37.851 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:37.167+0000 7f2280600700 1 --2- 172.21.15.43:0/2929403211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22781060f0 0x7f227819d8d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:37.851 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:37.167+0000 7f2280600700 1 --2- 172.21.15.43:0/2929403211 >> [v2:172.21.15.43:6800/2446912488,v1:172.21.15.43:6801/2446912488] conn(0x7f2260038f80 0x7f226003b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:37.851 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:37.167+0000 7f2280600700 1 -- 172.21.15.43:0/2929403211 >> 172.21.15.43:0/2929403211 conn(0x7f2278101e10 msgr2=0x7f2278194510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:37.851 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:37.167+0000 7f2280600700 1 -- 172.21.15.43:0/2929403211 shutdown_connections 2024-03-26T08:21:37.851 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:37.167+0000 7f2280600700 1 -- 172.21.15.43:0/2929403211 wait complete. 2024-03-26T08:21:37.852 INFO:teuthology.orchestra.run.smithi043.stdout:mgr epoch 9 is available 2024-03-26T08:21:37.852 INFO:teuthology.orchestra.run.smithi043.stdout:Generating a dashboard self-signed certificate... 2024-03-26T08:21:38.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:38 smithi043 ceph-mon[31929]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-26T08:21:38.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:38 smithi043 ceph-mon[31929]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-26T08:21:38.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:38 smithi043 ceph-mon[31929]: [26/Mar/2024:08:21:37] ENGINE Serving on https://172.21.15.43:7150 2024-03-26T08:21:38.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:38 smithi043 ceph-mon[31929]: [26/Mar/2024:08:21:37] ENGINE Bus STARTED 2024-03-26T08:21:39.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:39 smithi043 ceph-mon[31929]: mgrmap e12: x(active, since 2s) 2024-03-26T08:21:40.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:40 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:21:40.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:40 smithi043 ceph-mon[31929]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:21:40.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:40 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:21:40.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:40 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:21:40.508 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-26T08:21:40.508 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.215+0000 7f4bcfc00700 1 Processor -- start 2024-03-26T08:21:40.508 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.216+0000 7f4bcfc00700 1 -- start start 2024-03-26T08:21:40.508 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.216+0000 7f4bcfc00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bc8107d50 0x7f4bc8108170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:40.508 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.216+0000 7f4bcfc00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc8108740 con 0x7f4bc8107d50 2024-03-26T08:21:40.508 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.217+0000 7f4bcd200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bc8107d50 0x7f4bc8108170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:40.509 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.217+0000 7f4bcd200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bc8107d50 0x7f4bc8108170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38378/0 (socket says 172.21.15.43:38378) 2024-03-26T08:21:40.509 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.217+0000 7f4bcd200700 1 -- 172.21.15.43:0/3606200922 learned_addr learned my addr 172.21.15.43:0/3606200922 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:40.509 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.217+0000 7f4bcd200700 1 -- 172.21.15.43:0/3606200922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc81088c0 con 0x7f4bc8107d50 2024-03-26T08:21:40.509 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.218+0000 7f4bcd200700 1 --2- 172.21.15.43:0/3606200922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bc8107d50 0x7f4bc8108170 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc0098a0 tx=0x7f4bbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb27757feabe9e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:40.509 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.218+0000 7f4bc7400700 1 -- 172.21.15.43:0/3606200922 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bbc004160 con 0x7f4bc8107d50 2024-03-26T08:21:40.509 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.218+0000 7f4bc7400700 1 -- 172.21.15.43:0/3606200922 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4bbc004300 con 0x7f4bc8107d50 2024-03-26T08:21:40.509 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.218+0000 7f4bc7400700 1 -- 172.21.15.43:0/3606200922 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bbc011a50 con 0x7f4bc8107d50 2024-03-26T08:21:40.509 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.219+0000 7f4bcfc00700 1 -- 172.21.15.43:0/3606200922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bc8107d50 msgr2=0x7f4bc8108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:40.509 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.219+0000 7f4bcfc00700 1 --2- 172.21.15.43:0/3606200922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bc8107d50 0x7f4bc8108170 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc0098a0 tx=0x7f4bbc009bb0 comp rx=0 tx=0).stop 2024-03-26T08:21:40.509 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.219+0000 7f4bcfc00700 1 -- 172.21.15.43:0/3606200922 shutdown_connections 2024-03-26T08:21:40.509 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.219+0000 7f4bcfc00700 1 --2- 172.21.15.43:0/3606200922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bc8107d50 0x7f4bc8108170 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:40.509 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.219+0000 7f4bcfc00700 1 -- 172.21.15.43:0/3606200922 >> 172.21.15.43:0/3606200922 conn(0x7f4bc8101560 msgr2=0x7f4bc8103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:40.509 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.219+0000 7f4bcfc00700 1 -- 172.21.15.43:0/3606200922 shutdown_connections 2024-03-26T08:21:40.509 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.219+0000 7f4bcfc00700 1 -- 172.21.15.43:0/3606200922 wait complete. 2024-03-26T08:21:40.510 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.220+0000 7f4bcfc00700 1 Processor -- start 2024-03-26T08:21:40.510 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.220+0000 7f4bcfc00700 1 -- start start 2024-03-26T08:21:40.510 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.221+0000 7f4bcfc00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bc8107d50 0x7f4bc819b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:40.510 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.221+0000 7f4bcfc00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc810acd0 con 0x7f4bc8107d50 2024-03-26T08:21:40.510 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.221+0000 7f4bcd200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bc8107d50 0x7f4bc819b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:40.510 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.221+0000 7f4bcd200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bc8107d50 0x7f4bc819b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38382/0 (socket says 172.21.15.43:38382) 2024-03-26T08:21:40.510 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.221+0000 7f4bcd200700 1 -- 172.21.15.43:0/550136658 learned_addr learned my addr 172.21.15.43:0/550136658 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:40.510 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.221+0000 7f4bcd200700 1 -- 172.21.15.43:0/550136658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc819bcf0 con 0x7f4bc8107d50 2024-03-26T08:21:40.510 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.222+0000 7f4bcd200700 1 --2- 172.21.15.43:0/550136658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bc8107d50 0x7f4bc819b7b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc006b20 tx=0x7f4bbc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:40.510 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.222+0000 7f4bc5600700 1 -- 172.21.15.43:0/550136658 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bbc0039e0 con 0x7f4bc8107d50 2024-03-26T08:21:40.510 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.222+0000 7f4bc5600700 1 -- 172.21.15.43:0/550136658 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4bbc01a8e0 con 0x7f4bc8107d50 2024-03-26T08:21:40.510 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.222+0000 7f4bc5600700 1 -- 172.21.15.43:0/550136658 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bbc0247e0 con 0x7f4bc8107d50 2024-03-26T08:21:40.510 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.222+0000 7f4bcfc00700 1 -- 172.21.15.43:0/550136658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc819bf80 con 0x7f4bc8107d50 2024-03-26T08:21:40.511 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.222+0000 7f4bcfc00700 1 -- 172.21.15.43:0/550136658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc819ec70 con 0x7f4bc8107d50 2024-03-26T08:21:40.511 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.223+0000 7f4bc5600700 1 -- 172.21.15.43:0/550136658 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 12) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f4bbc022020 con 0x7f4bc8107d50 2024-03-26T08:21:40.511 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.224+0000 7f4bcfc00700 1 -- 172.21.15.43:0/550136658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc80637c0 con 0x7f4bc8107d50 2024-03-26T08:21:40.511 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.224+0000 7f4bc5600700 1 --2- 172.21.15.43:0/550136658 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f4bb0039000 0x7f4bb003b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:40.511 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.224+0000 7f4bc5600700 1 -- 172.21.15.43:0/550136658 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4bbc04d580 con 0x7f4bc8107d50 2024-03-26T08:21:40.511 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.224+0000 7f4bc7e00700 1 --2- 172.21.15.43:0/550136658 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f4bb0039000 0x7f4bb003b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:40.511 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.225+0000 7f4bc7e00700 1 --2- 172.21.15.43:0/550136658 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f4bb0039000 0x7f4bb003b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8006fd0 tx=0x7f4bb8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:40.511 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.230+0000 7f4bc5600700 1 -- 172.21.15.43:0/550136658 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4bbc02d910 con 0x7f4bc8107d50 2024-03-26T08:21:40.511 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.477+0000 7f4bcfc00700 1 -- 172.21.15.43:0/550136658 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f4bc802dd30 con 0x7f4bb0039000 2024-03-26T08:21:40.511 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.710+0000 7f4bc5600700 1 -- 172.21.15.43:0/550136658 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f4bc802dd30 con 0x7f4bb0039000 2024-03-26T08:21:40.511 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.714+0000 7f4bcfc00700 1 -- 172.21.15.43:0/550136658 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f4bb0039000 msgr2=0x7f4bb003b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:40.511 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.714+0000 7f4bcfc00700 1 --2- 172.21.15.43:0/550136658 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f4bb0039000 0x7f4bb003b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8006fd0 tx=0x7f4bb8006e40 comp rx=0 tx=0).stop 2024-03-26T08:21:40.511 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.714+0000 7f4bcfc00700 1 -- 172.21.15.43:0/550136658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bc8107d50 msgr2=0x7f4bc819b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:40.511 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.714+0000 7f4bcfc00700 1 --2- 172.21.15.43:0/550136658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bc8107d50 0x7f4bc819b7b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc006b20 tx=0x7f4bbc011ad0 comp rx=0 tx=0).stop 2024-03-26T08:21:40.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.714+0000 7f4bcfc00700 1 -- 172.21.15.43:0/550136658 shutdown_connections 2024-03-26T08:21:40.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.714+0000 7f4bcfc00700 1 --2- 172.21.15.43:0/550136658 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f4bb0039000 0x7f4bb003b4c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:40.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.714+0000 7f4bcfc00700 1 --2- 172.21.15.43:0/550136658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bc8107d50 0x7f4bc819b7b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:40.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.714+0000 7f4bcfc00700 1 -- 172.21.15.43:0/550136658 >> 172.21.15.43:0/550136658 conn(0x7f4bc8101560 msgr2=0x7f4bc81023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:40.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.714+0000 7f4bcfc00700 1 -- 172.21.15.43:0/550136658 shutdown_connections 2024-03-26T08:21:40.512 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:39.714+0000 7f4bcfc00700 1 -- 172.21.15.43:0/550136658 wait complete. 2024-03-26T08:21:40.512 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial admin user... 2024-03-26T08:21:42.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:42 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:21:42.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:42 smithi043 ceph-mon[31929]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:21:42.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:42 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:21:42.463 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$PZ5TaZ7kcDR2Uesi5SIziOAidKUwe8ETXNZgKI2cGs1b.s4K80Ewu", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711441301, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-26T08:21:42.463 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.168+0000 7f85d0800700 1 Processor -- start 2024-03-26T08:21:42.463 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.168+0000 7f85d0800700 1 -- start start 2024-03-26T08:21:42.463 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.168+0000 7f85d0800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85c807ce40 0x7f85c807b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:42.463 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.168+0000 7f85d0800700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c807b830 con 0x7f85c807ce40 2024-03-26T08:21:42.463 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.169+0000 7f85cde00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85c807ce40 0x7f85c807b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:42.463 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.169+0000 7f85cde00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85c807ce40 0x7f85c807b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38398/0 (socket says 172.21.15.43:38398) 2024-03-26T08:21:42.464 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.169+0000 7f85cde00700 1 -- 172.21.15.43:0/3260081020 learned_addr learned my addr 172.21.15.43:0/3260081020 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:42.464 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.169+0000 7f85cde00700 1 -- 172.21.15.43:0/3260081020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c807b9b0 con 0x7f85c807ce40 2024-03-26T08:21:42.464 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.170+0000 7f85cde00700 1 --2- 172.21.15.43:0/3260081020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85c807ce40 0x7f85c807b2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0098a0 tx=0x7f85bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a081a5bac0d380 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:42.464 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.170+0000 7f85cca00700 1 -- 172.21.15.43:0/3260081020 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f85bc004160 con 0x7f85c807ce40 2024-03-26T08:21:42.464 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.170+0000 7f85cca00700 1 -- 172.21.15.43:0/3260081020 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f85bc004300 con 0x7f85c807ce40 2024-03-26T08:21:42.464 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.170+0000 7f85cca00700 1 -- 172.21.15.43:0/3260081020 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f85bc011a50 con 0x7f85c807ce40 2024-03-26T08:21:42.464 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.171+0000 7f85d0800700 1 -- 172.21.15.43:0/3260081020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85c807ce40 msgr2=0x7f85c807b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:42.464 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.171+0000 7f85d0800700 1 --2- 172.21.15.43:0/3260081020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85c807ce40 0x7f85c807b2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0098a0 tx=0x7f85bc009bb0 comp rx=0 tx=0).stop 2024-03-26T08:21:42.464 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.171+0000 7f85d0800700 1 -- 172.21.15.43:0/3260081020 shutdown_connections 2024-03-26T08:21:42.464 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.171+0000 7f85d0800700 1 --2- 172.21.15.43:0/3260081020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85c807ce40 0x7f85c807b2f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:42.464 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.171+0000 7f85d0800700 1 -- 172.21.15.43:0/3260081020 >> 172.21.15.43:0/3260081020 conn(0x7f85c8102f70 msgr2=0x7f85c8105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:42.464 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.171+0000 7f85d0800700 1 -- 172.21.15.43:0/3260081020 shutdown_connections 2024-03-26T08:21:42.464 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.171+0000 7f85d0800700 1 -- 172.21.15.43:0/3260081020 wait complete. 2024-03-26T08:21:42.464 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.172+0000 7f85d0800700 1 Processor -- start 2024-03-26T08:21:42.464 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.172+0000 7f85d0800700 1 -- start start 2024-03-26T08:21:42.465 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.172+0000 7f85d0800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85c807ce40 0x7f85c819d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:42.465 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.173+0000 7f85d0800700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c810de90 con 0x7f85c807ce40 2024-03-26T08:21:42.465 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.173+0000 7f85cde00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85c807ce40 0x7f85c819d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:42.465 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.173+0000 7f85cde00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85c807ce40 0x7f85c819d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38410/0 (socket says 172.21.15.43:38410) 2024-03-26T08:21:42.465 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.173+0000 7f85cde00700 1 -- 172.21.15.43:0/402339703 learned_addr learned my addr 172.21.15.43:0/402339703 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:42.465 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.173+0000 7f85cde00700 1 -- 172.21.15.43:0/402339703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c819de50 con 0x7f85c807ce40 2024-03-26T08:21:42.465 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.174+0000 7f85cde00700 1 --2- 172.21.15.43:0/402339703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85c807ce40 0x7f85c819d910 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f85bc006b20 tx=0x7f85bc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:42.465 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.174+0000 7f85c2a00700 1 -- 172.21.15.43:0/402339703 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f85bc01a430 con 0x7f85c807ce40 2024-03-26T08:21:42.465 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.174+0000 7f85c2a00700 1 -- 172.21.15.43:0/402339703 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f85bc01aa80 con 0x7f85c807ce40 2024-03-26T08:21:42.465 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.174+0000 7f85c2a00700 1 -- 172.21.15.43:0/402339703 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f85bc024a20 con 0x7f85c807ce40 2024-03-26T08:21:42.465 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.175+0000 7f85d0800700 1 -- 172.21.15.43:0/402339703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c819e0e0 con 0x7f85c807ce40 2024-03-26T08:21:42.465 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.175+0000 7f85d0800700 1 -- 172.21.15.43:0/402339703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c81a0dd0 con 0x7f85c807ce40 2024-03-26T08:21:42.465 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.176+0000 7f85c2a00700 1 -- 172.21.15.43:0/402339703 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 12) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f85bc022020 con 0x7f85c807ce40 2024-03-26T08:21:42.465 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.176+0000 7f85d0800700 1 -- 172.21.15.43:0/402339703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c80637c0 con 0x7f85c807ce40 2024-03-26T08:21:42.465 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.177+0000 7f85c2a00700 1 --2- 172.21.15.43:0/402339703 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f85b0038fd0 0x7f85b003b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:42.466 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.177+0000 7f85c2a00700 1 -- 172.21.15.43:0/402339703 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f85bc04d350 con 0x7f85c807ce40 2024-03-26T08:21:42.466 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.177+0000 7f85cd400700 1 --2- 172.21.15.43:0/402339703 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f85b0038fd0 0x7f85b003b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:42.466 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.178+0000 7f85cd400700 1 --2- 172.21.15.43:0/402339703 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f85b0038fd0 0x7f85b003b490 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f85b8006fd0 tx=0x7f85b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:42.466 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.183+0000 7f85c2a00700 1 -- 172.21.15.43:0/402339703 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f85bc0094f0 con 0x7f85c807ce40 2024-03-26T08:21:42.466 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.451+0000 7f85d0800700 1 -- 172.21.15.43:0/402339703 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f85c81a1060 con 0x7f85b0038fd0 2024-03-26T08:21:42.466 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.717+0000 7f85c2a00700 1 -- 172.21.15.43:0/402339703 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f85c81a1060 con 0x7f85b0038fd0 2024-03-26T08:21:42.466 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.720+0000 7f85d0800700 1 -- 172.21.15.43:0/402339703 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f85b0038fd0 msgr2=0x7f85b003b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:42.466 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.720+0000 7f85d0800700 1 --2- 172.21.15.43:0/402339703 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f85b0038fd0 0x7f85b003b490 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f85b8006fd0 tx=0x7f85b8006e40 comp rx=0 tx=0).stop 2024-03-26T08:21:42.466 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.720+0000 7f85d0800700 1 -- 172.21.15.43:0/402339703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85c807ce40 msgr2=0x7f85c819d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:42.466 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.720+0000 7f85d0800700 1 --2- 172.21.15.43:0/402339703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85c807ce40 0x7f85c819d910 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f85bc006b20 tx=0x7f85bc011ad0 comp rx=0 tx=0).stop 2024-03-26T08:21:42.466 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.721+0000 7f85d0800700 1 -- 172.21.15.43:0/402339703 shutdown_connections 2024-03-26T08:21:42.466 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.721+0000 7f85d0800700 1 --2- 172.21.15.43:0/402339703 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f85b0038fd0 0x7f85b003b490 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:42.466 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.721+0000 7f85d0800700 1 --2- 172.21.15.43:0/402339703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85c807ce40 0x7f85c819d910 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:42.467 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.721+0000 7f85d0800700 1 -- 172.21.15.43:0/402339703 >> 172.21.15.43:0/402339703 conn(0x7f85c8102f70 msgr2=0x7f85c8103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:42.467 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.721+0000 7f85d0800700 1 -- 172.21.15.43:0/402339703 shutdown_connections 2024-03-26T08:21:42.467 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:41.721+0000 7f85d0800700 1 -- 172.21.15.43:0/402339703 wait complete. 2024-03-26T08:21:42.467 INFO:teuthology.orchestra.run.smithi043.stdout:Fetching dashboard port number... 2024-03-26T08:21:43.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:43 smithi043 ceph-mon[31929]: mgrmap e13: x(active, since 6s) 2024-03-26T08:21:43.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:43 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1749453719' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-26T08:21:44.553 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 8443 2024-03-26T08:21:44.553 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.048+0000 7fca4a600700 1 Processor -- start 2024-03-26T08:21:44.553 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.048+0000 7fca4a600700 1 -- start start 2024-03-26T08:21:44.554 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.048+0000 7fca4a600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca44107890 0x7fca44107cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:44.554 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.048+0000 7fca4a600700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca44108280 con 0x7fca44107890 2024-03-26T08:21:44.554 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.049+0000 7fca43400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca44107890 0x7fca44107cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:44.554 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.049+0000 7fca43400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca44107890 0x7fca44107cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38412/0 (socket says 172.21.15.43:38412) 2024-03-26T08:21:44.554 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.049+0000 7fca43400700 1 -- 172.21.15.43:0/4143439494 learned_addr learned my addr 172.21.15.43:0/4143439494 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:44.554 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.049+0000 7fca43400700 1 -- 172.21.15.43:0/4143439494 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca44108a40 con 0x7fca44107890 2024-03-26T08:21:44.554 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.050+0000 7fca43400700 1 --2- 172.21.15.43:0/4143439494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca44107890 0x7fca44107cb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fca340098a0 tx=0x7fca34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72cfdbd28d94005c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:44.554 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.050+0000 7fca42000700 1 -- 172.21.15.43:0/4143439494 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca34004160 con 0x7fca44107890 2024-03-26T08:21:44.555 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.050+0000 7fca42000700 1 -- 172.21.15.43:0/4143439494 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fca34004300 con 0x7fca44107890 2024-03-26T08:21:44.555 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.050+0000 7fca42000700 1 -- 172.21.15.43:0/4143439494 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca34011a50 con 0x7fca44107890 2024-03-26T08:21:44.557 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.051+0000 7fca4a600700 1 -- 172.21.15.43:0/4143439494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca44107890 msgr2=0x7fca44107cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:44.557 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.051+0000 7fca4a600700 1 --2- 172.21.15.43:0/4143439494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca44107890 0x7fca44107cb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fca340098a0 tx=0x7fca34009bb0 comp rx=0 tx=0).stop 2024-03-26T08:21:44.557 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.051+0000 7fca4a600700 1 -- 172.21.15.43:0/4143439494 shutdown_connections 2024-03-26T08:21:44.557 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.051+0000 7fca4a600700 1 --2- 172.21.15.43:0/4143439494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca44107890 0x7fca44107cb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:44.557 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.051+0000 7fca4a600700 1 -- 172.21.15.43:0/4143439494 >> 172.21.15.43:0/4143439494 conn(0x7fca44102e50 msgr2=0x7fca44105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:44.557 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.051+0000 7fca4a600700 1 -- 172.21.15.43:0/4143439494 shutdown_connections 2024-03-26T08:21:44.557 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.051+0000 7fca4a600700 1 -- 172.21.15.43:0/4143439494 wait complete. 2024-03-26T08:21:44.557 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.052+0000 7fca4a600700 1 Processor -- start 2024-03-26T08:21:44.557 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.052+0000 7fca4a600700 1 -- start start 2024-03-26T08:21:44.557 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.053+0000 7fca4a600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca44107890 0x7fca44199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:44.557 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.053+0000 7fca4a600700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4407b0f0 con 0x7fca44107890 2024-03-26T08:21:44.557 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.053+0000 7fca43400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca44107890 0x7fca44199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:44.557 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.053+0000 7fca43400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca44107890 0x7fca44199590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38414/0 (socket says 172.21.15.43:38414) 2024-03-26T08:21:44.558 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.053+0000 7fca43400700 1 -- 172.21.15.43:0/1749453719 learned_addr learned my addr 172.21.15.43:0/1749453719 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:44.558 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.054+0000 7fca43400700 1 -- 172.21.15.43:0/1749453719 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca44199ad0 con 0x7fca44107890 2024-03-26T08:21:44.558 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.054+0000 7fca43400700 1 --2- 172.21.15.43:0/1749453719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca44107890 0x7fca44199590 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fca34011a20 tx=0x7fca34011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:44.558 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.054+0000 7fca3be00700 1 -- 172.21.15.43:0/1749453719 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca34003af0 con 0x7fca44107890 2024-03-26T08:21:44.558 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.054+0000 7fca3be00700 1 -- 172.21.15.43:0/1749453719 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fca3401a8e0 con 0x7fca44107890 2024-03-26T08:21:44.560 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.054+0000 7fca3be00700 1 -- 172.21.15.43:0/1749453719 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca34024860 con 0x7fca44107890 2024-03-26T08:21:44.560 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.054+0000 7fca4a600700 1 -- 172.21.15.43:0/1749453719 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca44199d60 con 0x7fca44107890 2024-03-26T08:21:44.560 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.055+0000 7fca4a600700 1 -- 172.21.15.43:0/1749453719 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4419ca50 con 0x7fca44107890 2024-03-26T08:21:44.560 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.056+0000 7fca3be00700 1 -- 172.21.15.43:0/1749453719 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fca34022020 con 0x7fca44107890 2024-03-26T08:21:44.560 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.056+0000 7fca4a600700 1 -- 172.21.15.43:0/1749453719 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca440637c0 con 0x7fca44107890 2024-03-26T08:21:44.561 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.056+0000 7fca3be00700 1 --2- 172.21.15.43:0/1749453719 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fca2803d470 0x7fca2803f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:44.561 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.057+0000 7fca3be00700 1 -- 172.21.15.43:0/1749453719 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fca3404d2d0 con 0x7fca44107890 2024-03-26T08:21:44.561 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.057+0000 7fca42a00700 1 --2- 172.21.15.43:0/1749453719 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fca2803d470 0x7fca2803f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:44.561 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.058+0000 7fca42a00700 1 --2- 172.21.15.43:0/1749453719 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fca2803d470 0x7fca2803f930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fca30006fd0 tx=0x7fca30006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:44.561 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.062+0000 7fca3be00700 1 -- 172.21.15.43:0/1749453719 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fca340094f0 con 0x7fca44107890 2024-03-26T08:21:44.561 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.292+0000 7fca4a600700 1 -- 172.21.15.43:0/1749453719 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fca4402dc60 con 0x7fca44107890 2024-03-26T08:21:44.561 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.293+0000 7fca3be00700 1 -- 172.21.15.43:0/1749453719 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fca3402ad60 con 0x7fca44107890 2024-03-26T08:21:44.561 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.296+0000 7fca4a600700 1 -- 172.21.15.43:0/1749453719 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fca2803d470 msgr2=0x7fca2803f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:44.561 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.296+0000 7fca4a600700 1 --2- 172.21.15.43:0/1749453719 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fca2803d470 0x7fca2803f930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fca30006fd0 tx=0x7fca30006e40 comp rx=0 tx=0).stop 2024-03-26T08:21:44.561 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.296+0000 7fca4a600700 1 -- 172.21.15.43:0/1749453719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca44107890 msgr2=0x7fca44199590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:44.561 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.296+0000 7fca4a600700 1 --2- 172.21.15.43:0/1749453719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca44107890 0x7fca44199590 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fca34011a20 tx=0x7fca34011b00 comp rx=0 tx=0).stop 2024-03-26T08:21:44.561 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.296+0000 7fca4a600700 1 -- 172.21.15.43:0/1749453719 shutdown_connections 2024-03-26T08:21:44.561 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.297+0000 7fca4a600700 1 --2- 172.21.15.43:0/1749453719 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fca2803d470 0x7fca2803f930 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:44.561 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.297+0000 7fca4a600700 1 --2- 172.21.15.43:0/1749453719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca44107890 0x7fca44199590 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:44.562 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.297+0000 7fca4a600700 1 -- 172.21.15.43:0/1749453719 >> 172.21.15.43:0/1749453719 conn(0x7fca44102e50 msgr2=0x7fca441051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:44.562 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.297+0000 7fca4a600700 1 -- 172.21.15.43:0/1749453719 shutdown_connections 2024-03-26T08:21:44.562 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:43.297+0000 7fca4a600700 1 -- 172.21.15.43:0/1749453719 wait complete. 2024-03-26T08:21:44.598 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-26T08:21:44.599 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-03-26T08:21:44.625 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-26T08:21:44.625 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-03-26T08:21:44.626 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-03-26T08:21:44.626 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-26T08:21:44.633 INFO:teuthology.orchestra.run.smithi043.stdout:Ceph Dashboard is now available at: 2024-03-26T08:21:44.633 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:21:44.633 INFO:teuthology.orchestra.run.smithi043.stdout: URL: https://smithi043.front.sepia.ceph.com:8443/ 2024-03-26T08:21:44.633 INFO:teuthology.orchestra.run.smithi043.stdout: User: admin 2024-03-26T08:21:44.634 INFO:teuthology.orchestra.run.smithi043.stdout: Password: 3cuu2sfp48 2024-03-26T08:21:44.634 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:21:44.635 INFO:teuthology.orchestra.run.smithi043.stdout:Saving cluster configuration to /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/config directory 2024-03-26T08:21:44.637 INFO:teuthology.orchestra.run.smithi043.stdout:Enabling autotune for osd_memory_target 2024-03-26T08:21:46.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.726+0000 7fa643400700 1 Processor -- start 2024-03-26T08:21:46.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.726+0000 7fa643400700 1 -- start start 2024-03-26T08:21:46.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.726+0000 7fa643400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa63c10e0c0 0x7fa63c10e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:46.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.727+0000 7fa643400700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c10eab0 con 0x7fa63c10e0c0 2024-03-26T08:21:46.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.727+0000 7fa640a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa63c10e0c0 0x7fa63c10e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:46.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.727+0000 7fa640a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa63c10e0c0 0x7fa63c10e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51716/0 (socket says 172.21.15.43:51716) 2024-03-26T08:21:46.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.727+0000 7fa640a00700 1 -- 172.21.15.43:0/3068209054 learned_addr learned my addr 172.21.15.43:0/3068209054 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:46.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.728+0000 7fa640a00700 1 -- 172.21.15.43:0/3068209054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63c10efc0 con 0x7fa63c10e0c0 2024-03-26T08:21:46.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.728+0000 7fa640a00700 1 --2- 172.21.15.43:0/3068209054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa63c10e0c0 0x7fa63c10e4e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa62c0098a0 tx=0x7fa62c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546b197cb501b253 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:46.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.728+0000 7fa63be00700 1 -- 172.21.15.43:0/3068209054 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa62c004160 con 0x7fa63c10e0c0 2024-03-26T08:21:46.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.728+0000 7fa63be00700 1 -- 172.21.15.43:0/3068209054 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa62c004300 con 0x7fa63c10e0c0 2024-03-26T08:21:46.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.728+0000 7fa63be00700 1 -- 172.21.15.43:0/3068209054 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa62c011a50 con 0x7fa63c10e0c0 2024-03-26T08:21:46.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.729+0000 7fa643400700 1 -- 172.21.15.43:0/3068209054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa63c10e0c0 msgr2=0x7fa63c10e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:46.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.729+0000 7fa643400700 1 --2- 172.21.15.43:0/3068209054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa63c10e0c0 0x7fa63c10e4e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa62c0098a0 tx=0x7fa62c009bb0 comp rx=0 tx=0).stop 2024-03-26T08:21:46.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.729+0000 7fa643400700 1 -- 172.21.15.43:0/3068209054 shutdown_connections 2024-03-26T08:21:46.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.729+0000 7fa643400700 1 --2- 172.21.15.43:0/3068209054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa63c10e0c0 0x7fa63c10e4e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:46.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.729+0000 7fa643400700 1 -- 172.21.15.43:0/3068209054 >> 172.21.15.43:0/3068209054 conn(0x7fa63c07cee0 msgr2=0x7fa63c07f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:46.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.729+0000 7fa643400700 1 -- 172.21.15.43:0/3068209054 shutdown_connections 2024-03-26T08:21:46.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.729+0000 7fa643400700 1 -- 172.21.15.43:0/3068209054 wait complete. 2024-03-26T08:21:46.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.731+0000 7fa643400700 1 Processor -- start 2024-03-26T08:21:46.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.731+0000 7fa643400700 1 -- start start 2024-03-26T08:21:46.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.731+0000 7fa643400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa63c10e0c0 0x7fa63c1a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:46.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.731+0000 7fa643400700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c112280 con 0x7fa63c10e0c0 2024-03-26T08:21:46.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.732+0000 7fa640a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa63c10e0c0 0x7fa63c1a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:46.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.732+0000 7fa640a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa63c10e0c0 0x7fa63c1a1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51732/0 (socket says 172.21.15.43:51732) 2024-03-26T08:21:46.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.732+0000 7fa640a00700 1 -- 172.21.15.43:0/1020716748 learned_addr learned my addr 172.21.15.43:0/1020716748 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:46.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.732+0000 7fa640a00700 1 -- 172.21.15.43:0/1020716748 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63c1a22a0 con 0x7fa63c10e0c0 2024-03-26T08:21:46.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.732+0000 7fa640a00700 1 --2- 172.21.15.43:0/1020716748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa63c10e0c0 0x7fa63c1a1d60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa62c011a00 tx=0x7fa62c011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:46.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.732+0000 7fa63a000700 1 -- 172.21.15.43:0/1020716748 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa62c003af0 con 0x7fa63c10e0c0 2024-03-26T08:21:46.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.733+0000 7fa63a000700 1 -- 172.21.15.43:0/1020716748 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa62c01a8e0 con 0x7fa63c10e0c0 2024-03-26T08:21:46.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.733+0000 7fa643400700 1 -- 172.21.15.43:0/1020716748 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa63c1a2530 con 0x7fa63c10e0c0 2024-03-26T08:21:46.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.733+0000 7fa63a000700 1 -- 172.21.15.43:0/1020716748 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa62c024840 con 0x7fa63c10e0c0 2024-03-26T08:21:46.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.733+0000 7fa643400700 1 -- 172.21.15.43:0/1020716748 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63c1a5220 con 0x7fa63c10e0c0 2024-03-26T08:21:46.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.734+0000 7fa63a000700 1 -- 172.21.15.43:0/1020716748 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fa62c022020 con 0x7fa63c10e0c0 2024-03-26T08:21:46.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.734+0000 7fa643400700 1 -- 172.21.15.43:0/1020716748 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa63c112400 con 0x7fa63c10e0c0 2024-03-26T08:21:46.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.734+0000 7fa63a000700 1 --2- 172.21.15.43:0/1020716748 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa620039010 0x7fa62003b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:46.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.734+0000 7fa63a000700 1 -- 172.21.15.43:0/1020716748 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa62c04d380 con 0x7fa63c10e0c0 2024-03-26T08:21:46.733 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.735+0000 7fa633e00700 1 --2- 172.21.15.43:0/1020716748 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa620039010 0x7fa62003b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:46.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.735+0000 7fa633e00700 1 --2- 172.21.15.43:0/1020716748 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa620039010 0x7fa62003b4d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa624006fd0 tx=0x7fa624006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:46.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.741+0000 7fa63a000700 1 -- 172.21.15.43:0/1020716748 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa62c00b3e0 con 0x7fa63c10e0c0 2024-03-26T08:21:46.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.968+0000 7fa643400700 1 -- 172.21.15.43:0/1020716748 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fa63c0637c0 con 0x7fa63c10e0c0 2024-03-26T08:21:46.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.969+0000 7fa63a000700 1 -- 172.21.15.43:0/1020716748 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fa62c016260 con 0x7fa63c10e0c0 2024-03-26T08:21:46.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.972+0000 7fa643400700 1 -- 172.21.15.43:0/1020716748 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa620039010 msgr2=0x7fa62003b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:46.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.972+0000 7fa643400700 1 --2- 172.21.15.43:0/1020716748 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa620039010 0x7fa62003b4d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa624006fd0 tx=0x7fa624006e40 comp rx=0 tx=0).stop 2024-03-26T08:21:46.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.972+0000 7fa643400700 1 -- 172.21.15.43:0/1020716748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa63c10e0c0 msgr2=0x7fa63c1a1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:46.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.972+0000 7fa643400700 1 --2- 172.21.15.43:0/1020716748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa63c10e0c0 0x7fa63c1a1d60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa62c011a00 tx=0x7fa62c011ae0 comp rx=0 tx=0).stop 2024-03-26T08:21:46.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.972+0000 7fa643400700 1 -- 172.21.15.43:0/1020716748 shutdown_connections 2024-03-26T08:21:46.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.972+0000 7fa643400700 1 --2- 172.21.15.43:0/1020716748 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa620039010 0x7fa62003b4d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:46.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.972+0000 7fa643400700 1 --2- 172.21.15.43:0/1020716748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa63c10e0c0 0x7fa63c1a1d60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:46.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.972+0000 7fa643400700 1 -- 172.21.15.43:0/1020716748 >> 172.21.15.43:0/1020716748 conn(0x7fa63c07cee0 msgr2=0x7fa63c07d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:46.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.972+0000 7fa643400700 1 -- 172.21.15.43:0/1020716748 shutdown_connections 2024-03-26T08:21:46.734 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:45.972+0000 7fa643400700 1 -- 172.21.15.43:0/1020716748 wait complete. 2024-03-26T08:21:48.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.346+0000 7f2d88000700 1 Processor -- start 2024-03-26T08:21:48.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.347+0000 7f2d88000700 1 -- start start 2024-03-26T08:21:48.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.347+0000 7f2d88000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d8010e0c0 0x7f2d8010e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:48.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.347+0000 7f2d88000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8010eab0 con 0x7f2d8010e0c0 2024-03-26T08:21:48.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.348+0000 7f2d85600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d8010e0c0 0x7f2d8010e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:48.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.348+0000 7f2d85600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d8010e0c0 0x7f2d8010e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51738/0 (socket says 172.21.15.43:51738) 2024-03-26T08:21:48.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.348+0000 7f2d85600700 1 -- 172.21.15.43:0/3952197969 learned_addr learned my addr 172.21.15.43:0/3952197969 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:48.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.348+0000 7f2d85600700 1 -- 172.21.15.43:0/3952197969 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8010efc0 con 0x7f2d8010e0c0 2024-03-26T08:21:48.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.348+0000 7f2d85600700 1 --2- 172.21.15.43:0/3952197969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d8010e0c0 0x7f2d8010e4e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2d740098a0 tx=0x7f2d74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398698ac810925ab server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:48.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.349+0000 7f2d7fe00700 1 -- 172.21.15.43:0/3952197969 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d74004160 con 0x7f2d8010e0c0 2024-03-26T08:21:48.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.349+0000 7f2d7fe00700 1 -- 172.21.15.43:0/3952197969 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2d74004300 con 0x7f2d8010e0c0 2024-03-26T08:21:48.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.349+0000 7f2d7fe00700 1 -- 172.21.15.43:0/3952197969 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d74011a50 con 0x7f2d8010e0c0 2024-03-26T08:21:48.515 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.349+0000 7f2d88000700 1 -- 172.21.15.43:0/3952197969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d8010e0c0 msgr2=0x7f2d8010e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:48.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.349+0000 7f2d88000700 1 --2- 172.21.15.43:0/3952197969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d8010e0c0 0x7f2d8010e4e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2d740098a0 tx=0x7f2d74009bb0 comp rx=0 tx=0).stop 2024-03-26T08:21:48.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.350+0000 7f2d88000700 1 -- 172.21.15.43:0/3952197969 shutdown_connections 2024-03-26T08:21:48.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.350+0000 7f2d88000700 1 --2- 172.21.15.43:0/3952197969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d8010e0c0 0x7f2d8010e4e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:48.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.350+0000 7f2d88000700 1 -- 172.21.15.43:0/3952197969 >> 172.21.15.43:0/3952197969 conn(0x7f2d8007cee0 msgr2=0x7f2d8007f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:48.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.350+0000 7f2d88000700 1 -- 172.21.15.43:0/3952197969 shutdown_connections 2024-03-26T08:21:48.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.350+0000 7f2d88000700 1 -- 172.21.15.43:0/3952197969 wait complete. 2024-03-26T08:21:48.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.351+0000 7f2d88000700 1 Processor -- start 2024-03-26T08:21:48.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.351+0000 7f2d88000700 1 -- start start 2024-03-26T08:21:48.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.351+0000 7f2d88000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d8010e0c0 0x7f2d801a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:48.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.351+0000 7f2d88000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d80112280 con 0x7f2d8010e0c0 2024-03-26T08:21:48.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.352+0000 7f2d85600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d8010e0c0 0x7f2d801a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:48.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.352+0000 7f2d85600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d8010e0c0 0x7f2d801a1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51750/0 (socket says 172.21.15.43:51750) 2024-03-26T08:21:48.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.352+0000 7f2d85600700 1 -- 172.21.15.43:0/3173948577 learned_addr learned my addr 172.21.15.43:0/3173948577 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:48.519 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.352+0000 7f2d85600700 1 -- 172.21.15.43:0/3173948577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d801a22a0 con 0x7f2d8010e0c0 2024-03-26T08:21:48.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.352+0000 7f2d85600700 1 --2- 172.21.15.43:0/3173948577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d8010e0c0 0x7f2d801a1d60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2d74011a00 tx=0x7f2d74011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:48.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.353+0000 7f2d7e000700 1 -- 172.21.15.43:0/3173948577 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d74003af0 con 0x7f2d8010e0c0 2024-03-26T08:21:48.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.353+0000 7f2d7e000700 1 -- 172.21.15.43:0/3173948577 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2d7401a8e0 con 0x7f2d8010e0c0 2024-03-26T08:21:48.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.353+0000 7f2d7e000700 1 -- 172.21.15.43:0/3173948577 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d74024840 con 0x7f2d8010e0c0 2024-03-26T08:21:48.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.353+0000 7f2d88000700 1 -- 172.21.15.43:0/3173948577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d801a2530 con 0x7f2d8010e0c0 2024-03-26T08:21:48.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.353+0000 7f2d88000700 1 -- 172.21.15.43:0/3173948577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d801a5220 con 0x7f2d8010e0c0 2024-03-26T08:21:48.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.354+0000 7f2d7e000700 1 -- 172.21.15.43:0/3173948577 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f2d74022020 con 0x7f2d8010e0c0 2024-03-26T08:21:48.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.354+0000 7f2d88000700 1 -- 172.21.15.43:0/3173948577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d80112400 con 0x7f2d8010e0c0 2024-03-26T08:21:48.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.355+0000 7f2d7e000700 1 --2- 172.21.15.43:0/3173948577 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2d68039010 0x7f2d6803b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:48.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.355+0000 7f2d7e000700 1 -- 172.21.15.43:0/3173948577 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2d7404d4c0 con 0x7f2d8010e0c0 2024-03-26T08:21:48.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.355+0000 7f2d84c00700 1 --2- 172.21.15.43:0/3173948577 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2d68039010 0x7f2d6803b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:48.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.356+0000 7f2d84c00700 1 --2- 172.21.15.43:0/3173948577 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2d68039010 0x7f2d6803b4d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c006fd0 tx=0x7f2d6c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:48.520 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.361+0000 7f2d7e000700 1 -- 172.21.15.43:0/3173948577 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2d7402c350 con 0x7f2d8010e0c0 2024-03-26T08:21:48.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.714+0000 7f2d88000700 1 -- 172.21.15.43:0/3173948577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f2d801a53f0 con 0x7f2d8010e0c0 2024-03-26T08:21:48.521 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.720+0000 7f2d7e000700 1 -- 172.21.15.43:0/3173948577 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (secure 0 0 0) 0x7f2d7402dc40 con 0x7f2d8010e0c0 2024-03-26T08:21:48.522 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.723+0000 7f2d88000700 1 -- 172.21.15.43:0/3173948577 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2d68039010 msgr2=0x7f2d6803b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:48.522 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.723+0000 7f2d88000700 1 --2- 172.21.15.43:0/3173948577 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2d68039010 0x7f2d6803b4d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c006fd0 tx=0x7f2d6c006e40 comp rx=0 tx=0).stop 2024-03-26T08:21:48.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.723+0000 7f2d88000700 1 -- 172.21.15.43:0/3173948577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d8010e0c0 msgr2=0x7f2d801a1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:48.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.723+0000 7f2d88000700 1 --2- 172.21.15.43:0/3173948577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d8010e0c0 0x7f2d801a1d60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2d74011a00 tx=0x7f2d74011ae0 comp rx=0 tx=0).stop 2024-03-26T08:21:48.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.723+0000 7f2d88000700 1 -- 172.21.15.43:0/3173948577 shutdown_connections 2024-03-26T08:21:48.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.723+0000 7f2d88000700 1 --2- 172.21.15.43:0/3173948577 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2d68039010 0x7f2d6803b4d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:48.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.723+0000 7f2d88000700 1 --2- 172.21.15.43:0/3173948577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d8010e0c0 0x7f2d801a1d60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:48.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.723+0000 7f2d88000700 1 -- 172.21.15.43:0/3173948577 >> 172.21.15.43:0/3173948577 conn(0x7f2d8007cee0 msgr2=0x7f2d8007d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:48.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.723+0000 7f2d88000700 1 -- 172.21.15.43:0/3173948577 shutdown_connections 2024-03-26T08:21:48.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-26T08:21:47.724+0000 7f2d88000700 1 -- 172.21.15.43:0/3173948577 wait complete. 2024-03-26T08:21:48.523 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-26T08:21:48.523 INFO:teuthology.orchestra.run.smithi043.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-26T08:21:48.523 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:21:48.523 INFO:teuthology.orchestra.run.smithi043.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-26T08:21:48.523 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:21:48.523 INFO:teuthology.orchestra.run.smithi043.stdout:Or, if you are only running a single cluster on this host: 2024-03-26T08:21:48.523 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:21:48.523 INFO:teuthology.orchestra.run.smithi043.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-26T08:21:48.524 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:21:48.524 INFO:teuthology.orchestra.run.smithi043.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-26T08:21:48.524 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:21:48.524 INFO:teuthology.orchestra.run.smithi043.stdout: ceph telemetry on 2024-03-26T08:21:48.524 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:21:48.524 INFO:teuthology.orchestra.run.smithi043.stdout:For more information see: 2024-03-26T08:21:48.524 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:21:48.524 INFO:teuthology.orchestra.run.smithi043.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-26T08:21:48.524 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:21:48.524 INFO:teuthology.orchestra.run.smithi043.stdout:Bootstrap complete. 2024-03-26T08:21:48.639 INFO:tasks.cephadm:Fetching config... 2024-03-26T08:21:48.640 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-26T08:21:48.640 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-26T08:21:48.690 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-26T08:21:48.691 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-26T08:21:48.691 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-26T08:21:48.736 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-26T08:21:48.736 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-26T08:21:48.736 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-26T08:21:48.840 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:48 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3173948577' entity='client.admin' 2024-03-26T08:21:48.872 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-26T08:21:48.872 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-26T08:21:48.872 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-26T08:21:48.920 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-26T08:21:48.920 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3Xreypq2Hj5PzvATciwoEDKxj5ZfDIlDL/e5h04FXc3kNaKw4+wJHCD0Jl22T+ur8QwcCykDbFgloFFNCeET/soJxduH1ogE/6ezbhAQ+2Po599PepXKd7IgpgMWchsBL/+/7b/9e5lUuxvWWETZn2FoHNe80NeiyjH5w+dwiomQcoxIuNWzbNIO9NqpRMJ1oPmFi2JGQiu5lSzfGLRksw/qzTWmfigY1y4VH+7XjWfJARPDrDs7QsjboHFogqExj7spYbSPFy7WksXkKenupJP8QLStROuKsxmFCXMGoIB+/EwBHTs21UuMwdm1uZ148jOaJWVWcmKkLTMrB7gw6TvnZPRLgWbF3cutf9GvueXD2IsCqZHzKXPTr11y4UJhgTJxhYsPzillXwawHSefUlxHHOvI1W9k4u8k3oNj2JX//XMl4aalrCyHwfXI6U+r2PJAFTNPKy73v3D2WOejUETMAvxpaWM8dkeMol/l/QCSKJRKDX+HBZYVts0C817E= ceph-88759420-eb49-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-26T08:21:49.096 INFO:teuthology.orchestra.run.smithi043.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3Xreypq2Hj5PzvATciwoEDKxj5ZfDIlDL/e5h04FXc3kNaKw4+wJHCD0Jl22T+ur8QwcCykDbFgloFFNCeET/soJxduH1ogE/6ezbhAQ+2Po599PepXKd7IgpgMWchsBL/+/7b/9e5lUuxvWWETZn2FoHNe80NeiyjH5w+dwiomQcoxIuNWzbNIO9NqpRMJ1oPmFi2JGQiu5lSzfGLRksw/qzTWmfigY1y4VH+7XjWfJARPDrDs7QsjboHFogqExj7spYbSPFy7WksXkKenupJP8QLStROuKsxmFCXMGoIB+/EwBHTs21UuMwdm1uZ148jOaJWVWcmKkLTMrB7gw6TvnZPRLgWbF3cutf9GvueXD2IsCqZHzKXPTr11y4UJhgTJxhYsPzillXwawHSefUlxHHOvI1W9k4u8k3oNj2JX//XMl4aalrCyHwfXI6U+r2PJAFTNPKy73v3D2WOejUETMAvxpaWM8dkeMol/l/QCSKJRKDX+HBZYVts0C817E= ceph-88759420-eb49-11ee-95d0-87774f69a715 2024-03-26T08:21:49.175 DEBUG:teuthology.orchestra.run.smithi052:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3Xreypq2Hj5PzvATciwoEDKxj5ZfDIlDL/e5h04FXc3kNaKw4+wJHCD0Jl22T+ur8QwcCykDbFgloFFNCeET/soJxduH1ogE/6ezbhAQ+2Po599PepXKd7IgpgMWchsBL/+/7b/9e5lUuxvWWETZn2FoHNe80NeiyjH5w+dwiomQcoxIuNWzbNIO9NqpRMJ1oPmFi2JGQiu5lSzfGLRksw/qzTWmfigY1y4VH+7XjWfJARPDrDs7QsjboHFogqExj7spYbSPFy7WksXkKenupJP8QLStROuKsxmFCXMGoIB+/EwBHTs21UuMwdm1uZ148jOaJWVWcmKkLTMrB7gw6TvnZPRLgWbF3cutf9GvueXD2IsCqZHzKXPTr11y4UJhgTJxhYsPzillXwawHSefUlxHHOvI1W9k4u8k3oNj2JX//XMl4aalrCyHwfXI6U+r2PJAFTNPKy73v3D2WOejUETMAvxpaWM8dkeMol/l/QCSKJRKDX+HBZYVts0C817E= ceph-88759420-eb49-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-26T08:21:49.321 INFO:teuthology.orchestra.run.smithi052.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3Xreypq2Hj5PzvATciwoEDKxj5ZfDIlDL/e5h04FXc3kNaKw4+wJHCD0Jl22T+ur8QwcCykDbFgloFFNCeET/soJxduH1ogE/6ezbhAQ+2Po599PepXKd7IgpgMWchsBL/+/7b/9e5lUuxvWWETZn2FoHNe80NeiyjH5w+dwiomQcoxIuNWzbNIO9NqpRMJ1oPmFi2JGQiu5lSzfGLRksw/qzTWmfigY1y4VH+7XjWfJARPDrDs7QsjboHFogqExj7spYbSPFy7WksXkKenupJP8QLStROuKsxmFCXMGoIB+/EwBHTs21UuMwdm1uZ148jOaJWVWcmKkLTMrB7gw6TvnZPRLgWbF3cutf9GvueXD2IsCqZHzKXPTr11y4UJhgTJxhYsPzillXwawHSefUlxHHOvI1W9k4u8k3oNj2JX//XMl4aalrCyHwfXI6U+r2PJAFTNPKy73v3D2WOejUETMAvxpaWM8dkeMol/l/QCSKJRKDX+HBZYVts0C817E= ceph-88759420-eb49-11ee-95d0-87774f69a715 2024-03-26T08:21:49.389 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3Xreypq2Hj5PzvATciwoEDKxj5ZfDIlDL/e5h04FXc3kNaKw4+wJHCD0Jl22T+ur8QwcCykDbFgloFFNCeET/soJxduH1ogE/6ezbhAQ+2Po599PepXKd7IgpgMWchsBL/+/7b/9e5lUuxvWWETZn2FoHNe80NeiyjH5w+dwiomQcoxIuNWzbNIO9NqpRMJ1oPmFi2JGQiu5lSzfGLRksw/qzTWmfigY1y4VH+7XjWfJARPDrDs7QsjboHFogqExj7spYbSPFy7WksXkKenupJP8QLStROuKsxmFCXMGoIB+/EwBHTs21UuMwdm1uZ148jOaJWVWcmKkLTMrB7gw6TvnZPRLgWbF3cutf9GvueXD2IsCqZHzKXPTr11y4UJhgTJxhYsPzillXwawHSefUlxHHOvI1W9k4u8k3oNj2JX//XMl4aalrCyHwfXI6U+r2PJAFTNPKy73v3D2WOejUETMAvxpaWM8dkeMol/l/QCSKJRKDX+HBZYVts0C817E= ceph-88759420-eb49-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-26T08:21:49.526 INFO:teuthology.orchestra.run.smithi183.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3Xreypq2Hj5PzvATciwoEDKxj5ZfDIlDL/e5h04FXc3kNaKw4+wJHCD0Jl22T+ur8QwcCykDbFgloFFNCeET/soJxduH1ogE/6ezbhAQ+2Po599PepXKd7IgpgMWchsBL/+/7b/9e5lUuxvWWETZn2FoHNe80NeiyjH5w+dwiomQcoxIuNWzbNIO9NqpRMJ1oPmFi2JGQiu5lSzfGLRksw/qzTWmfigY1y4VH+7XjWfJARPDrDs7QsjboHFogqExj7spYbSPFy7WksXkKenupJP8QLStROuKsxmFCXMGoIB+/EwBHTs21UuMwdm1uZ148jOaJWVWcmKkLTMrB7gw6TvnZPRLgWbF3cutf9GvueXD2IsCqZHzKXPTr11y4UJhgTJxhYsPzillXwawHSefUlxHHOvI1W9k4u8k3oNj2JX//XMl4aalrCyHwfXI6U+r2PJAFTNPKy73v3D2WOejUETMAvxpaWM8dkeMol/l/QCSKJRKDX+HBZYVts0C817E= ceph-88759420-eb49-11ee-95d0-87774f69a715 2024-03-26T08:21:49.578 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-26T08:21:49.994 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:21:52.819 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:52 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:21:52.819 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:52 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:21:52.819 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:52 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi043", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:21:52.819 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:52 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:21:52.819 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:52 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:21:52.819 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:52 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:21:52.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.002+0000 7f64d9a00700 1 -- 172.21.15.43:0/1202019718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d40761c0 msgr2=0x7f64d4074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:52.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.002+0000 7f64d9a00700 1 --2- 172.21.15.43:0/1202019718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d40761c0 0x7f64d4074620 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f64c4009910 tx=0x7f64c4009c20 comp rx=0 tx=0).stop 2024-03-26T08:21:52.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.003+0000 7f64d9a00700 1 -- 172.21.15.43:0/1202019718 shutdown_connections 2024-03-26T08:21:52.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.003+0000 7f64d9a00700 1 --2- 172.21.15.43:0/1202019718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d40761c0 0x7f64d4074620 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:52.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.003+0000 7f64d9a00700 1 -- 172.21.15.43:0/1202019718 >> 172.21.15.43:0/1202019718 conn(0x7f64d40fd160 msgr2=0x7f64d40ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:52.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.003+0000 7f64d9a00700 1 -- 172.21.15.43:0/1202019718 shutdown_connections 2024-03-26T08:21:52.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.003+0000 7f64d9a00700 1 -- 172.21.15.43:0/1202019718 wait complete. 2024-03-26T08:21:52.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.004+0000 7f64d9a00700 1 Processor -- start 2024-03-26T08:21:52.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.004+0000 7f64d9a00700 1 -- start start 2024-03-26T08:21:52.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.004+0000 7f64d9a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d40761c0 0x7f64d41993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:52.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.004+0000 7f64d9a00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d4109d00 con 0x7f64d40761c0 2024-03-26T08:21:52.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.005+0000 7f64d2a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d40761c0 0x7f64d41993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:52.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.005+0000 7f64d2a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d40761c0 0x7f64d41993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51762/0 (socket says 172.21.15.43:51762) 2024-03-26T08:21:52.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.005+0000 7f64d2a00700 1 -- 172.21.15.43:0/80223587 learned_addr learned my addr 172.21.15.43:0/80223587 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:53.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.005+0000 7f64d2a00700 1 -- 172.21.15.43:0/80223587 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d4199920 con 0x7f64d40761c0 2024-03-26T08:21:53.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.006+0000 7f64d2a00700 1 --2- 172.21.15.43:0/80223587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d40761c0 0x7f64d41993e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f64c4005800 tx=0x7f64c4009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:53.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.006+0000 7f64cbe00700 1 -- 172.21.15.43:0/80223587 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64c4027070 con 0x7f64d40761c0 2024-03-26T08:21:53.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.006+0000 7f64cbe00700 1 -- 172.21.15.43:0/80223587 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f64c400be10 con 0x7f64d40761c0 2024-03-26T08:21:53.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.006+0000 7f64cbe00700 1 -- 172.21.15.43:0/80223587 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64c400f460 con 0x7f64d40761c0 2024-03-26T08:21:53.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.006+0000 7f64d9a00700 1 -- 172.21.15.43:0/80223587 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d4199bb0 con 0x7f64d40761c0 2024-03-26T08:21:53.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.006+0000 7f64d9a00700 1 -- 172.21.15.43:0/80223587 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d419c8a0 con 0x7f64d40761c0 2024-03-26T08:21:53.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.007+0000 7f64d9a00700 1 -- 172.21.15.43:0/80223587 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d410a480 con 0x7f64d40761c0 2024-03-26T08:21:53.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.007+0000 7f64cbe00700 1 -- 172.21.15.43:0/80223587 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f64c402a070 con 0x7f64d40761c0 2024-03-26T08:21:53.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.008+0000 7f64cbe00700 1 --2- 172.21.15.43:0/80223587 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f64b8039020 0x7f64b803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:53.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.008+0000 7f64cbe00700 1 -- 172.21.15.43:0/80223587 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f64c401e030 con 0x7f64d40761c0 2024-03-26T08:21:53.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.008+0000 7f64ca000700 1 --2- 172.21.15.43:0/80223587 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f64b8039020 0x7f64b803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:53.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.009+0000 7f64ca000700 1 --2- 172.21.15.43:0/80223587 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f64b8039020 0x7f64b803b4e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f64bc006fd0 tx=0x7f64bc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:53.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.013+0000 7f64cbe00700 1 -- 172.21.15.43:0/80223587 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f64c4013070 con 0x7f64d40761c0 2024-03-26T08:21:53.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.239+0000 7f64d9a00700 1 -- 172.21.15.43:0/80223587 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f64d402db40 con 0x7f64d40761c0 2024-03-26T08:21:53.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.243+0000 7f64cbe00700 1 -- 172.21.15.43:0/80223587 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f64c4031020 con 0x7f64d40761c0 2024-03-26T08:21:53.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.247+0000 7f64d9a00700 1 -- 172.21.15.43:0/80223587 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f64b8039020 msgr2=0x7f64b803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:53.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.247+0000 7f64d9a00700 1 --2- 172.21.15.43:0/80223587 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f64b8039020 0x7f64b803b4e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f64bc006fd0 tx=0x7f64bc006e40 comp rx=0 tx=0).stop 2024-03-26T08:21:53.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.247+0000 7f64d9a00700 1 -- 172.21.15.43:0/80223587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d40761c0 msgr2=0x7f64d41993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:53.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.247+0000 7f64d9a00700 1 --2- 172.21.15.43:0/80223587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d40761c0 0x7f64d41993e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f64c4005800 tx=0x7f64c4009f60 comp rx=0 tx=0).stop 2024-03-26T08:21:53.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.247+0000 7f64d9a00700 1 -- 172.21.15.43:0/80223587 shutdown_connections 2024-03-26T08:21:53.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.247+0000 7f64d9a00700 1 --2- 172.21.15.43:0/80223587 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f64b8039020 0x7f64b803b4e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:53.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.247+0000 7f64d9a00700 1 --2- 172.21.15.43:0/80223587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64d40761c0 0x7f64d41993e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:53.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.247+0000 7f64d9a00700 1 -- 172.21.15.43:0/80223587 >> 172.21.15.43:0/80223587 conn(0x7f64d40fd160 msgr2=0x7f64d40fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:53.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.247+0000 7f64d9a00700 1 -- 172.21.15.43:0/80223587 shutdown_connections 2024-03-26T08:21:53.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:53.247+0000 7f64d9a00700 1 -- 172.21.15.43:0/80223587 wait complete. 2024-03-26T08:21:53.985 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-26T08:21:53.985 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-26T08:21:54.430 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:21:54.579 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:54 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/80223587' entity='client.admin' 2024-03-26T08:21:54.579 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:54 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:21:54.579 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:54 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:21:56.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.375+0000 7fbede800700 1 -- 172.21.15.43:0/3250943913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbed8076b40 msgr2=0x7fbed8076f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:56.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.375+0000 7fbede800700 1 --2- 172.21.15.43:0/3250943913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbed8076b40 0x7fbed8076f60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009910 tx=0x7fbec8009c20 comp rx=0 tx=0).stop 2024-03-26T08:21:56.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.375+0000 7fbede800700 1 -- 172.21.15.43:0/3250943913 shutdown_connections 2024-03-26T08:21:56.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.375+0000 7fbede800700 1 --2- 172.21.15.43:0/3250943913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbed8076b40 0x7fbed8076f60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:56.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.375+0000 7fbede800700 1 -- 172.21.15.43:0/3250943913 >> 172.21.15.43:0/3250943913 conn(0x7fbed80ff3b0 msgr2=0x7fbed81017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:56.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.375+0000 7fbede800700 1 -- 172.21.15.43:0/3250943913 shutdown_connections 2024-03-26T08:21:56.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.375+0000 7fbede800700 1 -- 172.21.15.43:0/3250943913 wait complete. 2024-03-26T08:21:56.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.376+0000 7fbede800700 1 Processor -- start 2024-03-26T08:21:56.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.376+0000 7fbede800700 1 -- start start 2024-03-26T08:21:56.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.377+0000 7fbede800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbed8076b40 0x7fbed819d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:56.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.377+0000 7fbede800700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed810e1d0 con 0x7fbed8076b40 2024-03-26T08:21:56.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.377+0000 7fbed7400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbed8076b40 0x7fbed819d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:56.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.377+0000 7fbed7400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbed8076b40 0x7fbed819d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59228/0 (socket says 172.21.15.43:59228) 2024-03-26T08:21:56.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.377+0000 7fbed7400700 1 -- 172.21.15.43:0/3336182653 learned_addr learned my addr 172.21.15.43:0/3336182653 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:21:56.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.378+0000 7fbed7400700 1 -- 172.21.15.43:0/3336182653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed819dd00 con 0x7fbed8076b40 2024-03-26T08:21:56.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.378+0000 7fbed7400700 1 --2- 172.21.15.43:0/3336182653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbed8076b40 0x7fbed819d7c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbec8005190 tx=0x7fbec8004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:56.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.378+0000 7fbecfe00700 1 -- 172.21.15.43:0/3336182653 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbec8027070 con 0x7fbed8076b40 2024-03-26T08:21:56.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.378+0000 7fbecfe00700 1 -- 172.21.15.43:0/3336182653 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbec8022470 con 0x7fbed8076b40 2024-03-26T08:21:56.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.378+0000 7fbede800700 1 -- 172.21.15.43:0/3336182653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed819df90 con 0x7fbed8076b40 2024-03-26T08:21:56.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.378+0000 7fbecfe00700 1 -- 172.21.15.43:0/3336182653 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbec800f5f0 con 0x7fbed8076b40 2024-03-26T08:21:56.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.379+0000 7fbede800700 1 -- 172.21.15.43:0/3336182653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed81a0c80 con 0x7fbed8076b40 2024-03-26T08:21:56.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.380+0000 7fbecfe00700 1 -- 172.21.15.43:0/3336182653 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fbec802a070 con 0x7fbed8076b40 2024-03-26T08:21:56.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.380+0000 7fbede800700 1 -- 172.21.15.43:0/3336182653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed810e8a0 con 0x7fbed8076b40 2024-03-26T08:21:56.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.380+0000 7fbecfe00700 1 --2- 172.21.15.43:0/3336182653 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fbebc038fc0 0x7fbebc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:21:56.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.380+0000 7fbecfe00700 1 -- 172.21.15.43:0/3336182653 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbec801e030 con 0x7fbed8076b40 2024-03-26T08:21:56.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.381+0000 7fbed6a00700 1 --2- 172.21.15.43:0/3336182653 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fbebc038fc0 0x7fbebc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:21:56.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.381+0000 7fbed6a00700 1 --2- 172.21.15.43:0/3336182653 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fbebc038fc0 0x7fbebc03b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbec4006fd0 tx=0x7fbec4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:21:56.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.386+0000 7fbecfe00700 1 -- 172.21.15.43:0/3336182653 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbec8013070 con 0x7fbed8076b40 2024-03-26T08:21:56.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.627+0000 7fbede800700 1 -- 172.21.15.43:0/3336182653 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fbed802db40 con 0x7fbebc038fc0 2024-03-26T08:21:56.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.632+0000 7fbecfe00700 1 -- 172.21.15.43:0/3336182653 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbed802db40 con 0x7fbebc038fc0 2024-03-26T08:21:56.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.636+0000 7fbede800700 1 -- 172.21.15.43:0/3336182653 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fbebc038fc0 msgr2=0x7fbebc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:56.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.636+0000 7fbede800700 1 --2- 172.21.15.43:0/3336182653 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fbebc038fc0 0x7fbebc03b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbec4006fd0 tx=0x7fbec4006e40 comp rx=0 tx=0).stop 2024-03-26T08:21:56.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.636+0000 7fbede800700 1 -- 172.21.15.43:0/3336182653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbed8076b40 msgr2=0x7fbed819d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:21:56.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.636+0000 7fbede800700 1 --2- 172.21.15.43:0/3336182653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbed8076b40 0x7fbed819d7c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbec8005190 tx=0x7fbec8004f80 comp rx=0 tx=0).stop 2024-03-26T08:21:56.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.636+0000 7fbede800700 1 -- 172.21.15.43:0/3336182653 shutdown_connections 2024-03-26T08:21:56.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.636+0000 7fbede800700 1 --2- 172.21.15.43:0/3336182653 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fbebc038fc0 0x7fbebc03b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:56.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.636+0000 7fbede800700 1 --2- 172.21.15.43:0/3336182653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbed8076b40 0x7fbed819d7c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:21:56.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.636+0000 7fbede800700 1 -- 172.21.15.43:0/3336182653 >> 172.21.15.43:0/3336182653 conn(0x7fbed80ff3b0 msgr2=0x7fbed80ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:21:56.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.636+0000 7fbede800700 1 -- 172.21.15.43:0/3336182653 shutdown_connections 2024-03-26T08:21:56.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:21:56.636+0000 7fbede800700 1 -- 172.21.15.43:0/3336182653 wait complete. 2024-03-26T08:21:57.493 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi052 2024-03-26T08:21:57.493 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-26T08:21:57.493 DEBUG:teuthology.orchestra.run.smithi052:> dd of=/etc/ceph/ceph.conf 2024-03-26T08:21:57.554 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-26T08:21:57.555 DEBUG:teuthology.orchestra.run.smithi052:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T08:21:57.654 INFO:tasks.cephadm:Adding host smithi052 to orchestrator... 2024-03-26T08:21:57.655 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph orch host add smithi052 2024-03-26T08:21:57.729 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:57 smithi043 ceph-mon[31929]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:21:57.730 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:57 smithi043 ceph-mon[31929]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:21:57.730 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:57 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:21:57.730 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:57 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:21:57.730 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:57 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:21:57.730 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:57 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:21:57.730 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:57 smithi043 ceph-mon[31929]: Updating smithi043:/etc/ceph/ceph.conf 2024-03-26T08:21:58.085 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:21:58.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:58 smithi043 ceph-mon[31929]: Updating smithi043:/var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:21:58.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:58 smithi043 ceph-mon[31929]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-03-26T08:21:59.754 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:59 smithi043 ceph-mon[31929]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:21:59.754 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:59 smithi043 ceph-mon[31929]: Updating smithi043:/var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T08:21:59.755 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:59 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:21:59.755 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:59 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:21:59.755 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:21:59 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:00.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.262+0000 7f8fb2600700 1 -- 172.21.15.43:0/478076629 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fac10e0b0 msgr2=0x7f8fac10e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:00.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.262+0000 7f8fb2600700 1 --2- 172.21.15.43:0/478076629 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fac10e0b0 0x7f8fac10e490 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0009910 tx=0x7f8fa0009c20 comp rx=0 tx=0).stop 2024-03-26T08:22:00.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.262+0000 7f8fb2600700 1 -- 172.21.15.43:0/478076629 shutdown_connections 2024-03-26T08:22:00.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.262+0000 7f8fb2600700 1 --2- 172.21.15.43:0/478076629 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fac10e0b0 0x7f8fac10e490 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:00.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.262+0000 7f8fb2600700 1 -- 172.21.15.43:0/478076629 >> 172.21.15.43:0/478076629 conn(0x7f8fac077770 msgr2=0x7f8fac079b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:22:00.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.262+0000 7f8fb2600700 1 -- 172.21.15.43:0/478076629 shutdown_connections 2024-03-26T08:22:00.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.262+0000 7f8fb2600700 1 -- 172.21.15.43:0/478076629 wait complete. 2024-03-26T08:22:00.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.263+0000 7f8fb2600700 1 Processor -- start 2024-03-26T08:22:00.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.264+0000 7f8fb2600700 1 -- start start 2024-03-26T08:22:00.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.264+0000 7f8fb2600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fac10e0b0 0x7f8fac1a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:22:00.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.264+0000 7f8fb2600700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac1124b0 con 0x7f8fac10e0b0 2024-03-26T08:22:00.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.264+0000 7f8fb1200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fac10e0b0 0x7f8fac1a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:22:00.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.264+0000 7f8fb1200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fac10e0b0 0x7f8fac1a3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59252/0 (socket says 172.21.15.43:59252) 2024-03-26T08:22:00.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.264+0000 7f8fb1200700 1 -- 172.21.15.43:0/1671278564 learned_addr learned my addr 172.21.15.43:0/1671278564 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:22:00.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.265+0000 7f8fb1200700 1 -- 172.21.15.43:0/1671278564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fac1a3b70 con 0x7f8fac10e0b0 2024-03-26T08:22:00.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.265+0000 7f8fb1200700 1 --2- 172.21.15.43:0/1671278564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fac10e0b0 0x7f8fac1a3630 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0006010 tx=0x7f8fa0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:22:00.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.265+0000 7f8fa9600700 1 -- 172.21.15.43:0/1671278564 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fa0026070 con 0x7f8fac10e0b0 2024-03-26T08:22:00.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.266+0000 7f8fb2600700 1 -- 172.21.15.43:0/1671278564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fac1a3e60 con 0x7f8fac10e0b0 2024-03-26T08:22:00.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.266+0000 7f8fa9600700 1 -- 172.21.15.43:0/1671278564 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8fa000bba0 con 0x7f8fac10e0b0 2024-03-26T08:22:00.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.266+0000 7f8fa9600700 1 -- 172.21.15.43:0/1671278564 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fa0003900 con 0x7f8fac10e0b0 2024-03-26T08:22:00.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.266+0000 7f8fb2600700 1 -- 172.21.15.43:0/1671278564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fac19d950 con 0x7f8fac10e0b0 2024-03-26T08:22:00.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.267+0000 7f8fb2600700 1 -- 172.21.15.43:0/1671278564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fac1096f0 con 0x7f8fac10e0b0 2024-03-26T08:22:00.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.267+0000 7f8fa9600700 1 -- 172.21.15.43:0/1671278564 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f8fa0029070 con 0x7f8fac10e0b0 2024-03-26T08:22:00.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.268+0000 7f8fa9600700 1 --2- 172.21.15.43:0/1671278564 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8f94038fc0 0x7f8f9403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:22:00.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.268+0000 7f8fa9600700 1 -- 172.21.15.43:0/1671278564 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8fa001e070 con 0x7f8fac10e0b0 2024-03-26T08:22:00.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.268+0000 7f8fabe00700 1 --2- 172.21.15.43:0/1671278564 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8f94038fc0 0x7f8f9403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:22:00.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.269+0000 7f8fabe00700 1 --2- 172.21.15.43:0/1671278564 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8f94038fc0 0x7f8f9403b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8f98006fd0 tx=0x7f8f98006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:22:00.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.274+0000 7f8fa9600700 1 -- 172.21.15.43:0/1671278564 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8fa0013070 con 0x7f8fac10e0b0 2024-03-26T08:22:00.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:00.506+0000 7f8fb2600700 1 -- 172.21.15.43:0/1671278564 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi052", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fac19ec60 con 0x7f8f94038fc0 2024-03-26T08:22:01.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:01 smithi043 ceph-mon[31929]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:01.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:01 smithi043 ceph-mon[31929]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi052", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:22:02.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:02 smithi043 ceph-mon[31929]: Deploying cephadm binary to smithi052 2024-03-26T08:22:03.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:03 smithi043 ceph-mon[31929]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:05.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:05 smithi043 ceph-mon[31929]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:07.397 INFO:teuthology.orchestra.run.smithi043.stdout:Added host 'smithi052' with addr '172.21.15.52' 2024-03-26T08:22:07.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:07.400+0000 7f8fa9600700 1 -- 172.21.15.43:0/1671278564 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f8fac19ec60 con 0x7f8f94038fc0 2024-03-26T08:22:07.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:07.404+0000 7f8fb2600700 1 -- 172.21.15.43:0/1671278564 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8f94038fc0 msgr2=0x7f8f9403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:07.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:07.404+0000 7f8fb2600700 1 --2- 172.21.15.43:0/1671278564 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8f94038fc0 0x7f8f9403b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8f98006fd0 tx=0x7f8f98006e40 comp rx=0 tx=0).stop 2024-03-26T08:22:07.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:07.404+0000 7f8fb2600700 1 -- 172.21.15.43:0/1671278564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fac10e0b0 msgr2=0x7f8fac1a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:07.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:07.404+0000 7f8fb2600700 1 --2- 172.21.15.43:0/1671278564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fac10e0b0 0x7f8fac1a3630 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0006010 tx=0x7f8fa0005780 comp rx=0 tx=0).stop 2024-03-26T08:22:07.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:07.404+0000 7f8fb2600700 1 -- 172.21.15.43:0/1671278564 shutdown_connections 2024-03-26T08:22:07.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:07.404+0000 7f8fb2600700 1 --2- 172.21.15.43:0/1671278564 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8f94038fc0 0x7f8f9403b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:07.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:07.404+0000 7f8fb2600700 1 --2- 172.21.15.43:0/1671278564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fac10e0b0 0x7f8fac1a3630 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:07.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:07.404+0000 7f8fb2600700 1 -- 172.21.15.43:0/1671278564 >> 172.21.15.43:0/1671278564 conn(0x7f8fac077770 msgr2=0x7f8fac077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:22:07.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:07.405+0000 7f8fb2600700 1 -- 172.21.15.43:0/1671278564 shutdown_connections 2024-03-26T08:22:07.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:07.405+0000 7f8fb2600700 1 -- 172.21.15.43:0/1671278564 wait complete. 2024-03-26T08:22:07.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:07 smithi043 ceph-mon[31929]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:07.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:07 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:07.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:07 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:22:08.230 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph orch host ls --format=json 2024-03-26T08:22:08.700 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:22:08.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:08 smithi043 ceph-mon[31929]: Added host smithi052 2024-03-26T08:22:08.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:08 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:10.081 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:09 smithi043 ceph-mon[31929]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:10.081 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:09 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:10.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:10.879+0000 7f4b2f200700 1 -- 172.21.15.43:0/2773265394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b300745c0 msgr2=0x7f4b300749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:10.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:10.879+0000 7f4b2f200700 1 --2- 172.21.15.43:0/2773265394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b300745c0 0x7f4b300749a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009910 tx=0x7f4b20009c20 comp rx=0 tx=0).stop 2024-03-26T08:22:10.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:10.879+0000 7f4b2f200700 1 -- 172.21.15.43:0/2773265394 shutdown_connections 2024-03-26T08:22:10.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:10.879+0000 7f4b2f200700 1 --2- 172.21.15.43:0/2773265394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b300745c0 0x7f4b300749a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:10.879+0000 7f4b2f200700 1 -- 172.21.15.43:0/2773265394 >> 172.21.15.43:0/2773265394 conn(0x7f4b300fd4c0 msgr2=0x7f4b300ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:22:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:10.879+0000 7f4b2f200700 1 -- 172.21.15.43:0/2773265394 shutdown_connections 2024-03-26T08:22:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:10.879+0000 7f4b2f200700 1 -- 172.21.15.43:0/2773265394 wait complete. 2024-03-26T08:22:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:10.880+0000 7f4b2f200700 1 Processor -- start 2024-03-26T08:22:10.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:10.881+0000 7f4b2f200700 1 -- start start 2024-03-26T08:22:10.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:10.881+0000 7f4b2f200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b300745c0 0x7f4b30199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:22:10.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:10.881+0000 7f4b2f200700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3010e110 con 0x7f4b300745c0 2024-03-26T08:22:10.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:10.881+0000 7f4b2de00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b300745c0 0x7f4b30199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:22:10.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:10.881+0000 7f4b2de00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b300745c0 0x7f4b30199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59346/0 (socket says 172.21.15.43:59346) 2024-03-26T08:22:10.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:10.881+0000 7f4b2de00700 1 -- 172.21.15.43:0/4152089175 learned_addr learned my addr 172.21.15.43:0/4152089175 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:22:10.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:10.882+0000 7f4b2de00700 1 -- 172.21.15.43:0/4152089175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b30199bc0 con 0x7f4b300745c0 2024-03-26T08:22:10.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:10.882+0000 7f4b2de00700 1 --2- 172.21.15.43:0/4152089175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b300745c0 0x7f4b30199680 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4b20006010 tx=0x7f4b20005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:22:10.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:10.882+0000 7f4b26a00700 1 -- 172.21.15.43:0/4152089175 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b20026070 con 0x7f4b300745c0 2024-03-26T08:22:10.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:10.882+0000 7f4b26a00700 1 -- 172.21.15.43:0/4152089175 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4b2000bc00 con 0x7f4b300745c0 2024-03-26T08:22:10.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:10.882+0000 7f4b2f200700 1 -- 172.21.15.43:0/4152089175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b30199e50 con 0x7f4b300745c0 2024-03-26T08:22:10.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:10.882+0000 7f4b26a00700 1 -- 172.21.15.43:0/4152089175 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b20003910 con 0x7f4b300745c0 2024-03-26T08:22:10.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:10.883+0000 7f4b2f200700 1 -- 172.21.15.43:0/4152089175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b3019a2b0 con 0x7f4b300745c0 2024-03-26T08:22:10.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:10.884+0000 7f4b26a00700 1 -- 172.21.15.43:0/4152089175 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f4b20029070 con 0x7f4b300745c0 2024-03-26T08:22:10.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:10.884+0000 7f4b2f200700 1 -- 172.21.15.43:0/4152089175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b30075c70 con 0x7f4b300745c0 2024-03-26T08:22:10.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:10.884+0000 7f4b26a00700 1 --2- 172.21.15.43:0/4152089175 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f4b14038fc0 0x7f4b1403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:22:10.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:10.884+0000 7f4b26a00700 1 -- 172.21.15.43:0/4152089175 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4b2001e070 con 0x7f4b300745c0 2024-03-26T08:22:10.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:10.884+0000 7f4b2d400700 1 --2- 172.21.15.43:0/4152089175 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f4b14038fc0 0x7f4b1403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:22:10.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:10.885+0000 7f4b2d400700 1 --2- 172.21.15.43:0/4152089175 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f4b14038fc0 0x7f4b1403b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c006fd0 tx=0x7f4b1c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:22:10.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:10.890+0000 7f4b26a00700 1 -- 172.21.15.43:0/4152089175 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4b20013070 con 0x7f4b300745c0 2024-03-26T08:22:11.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:11.124+0000 7f4b2f200700 1 -- 172.21.15.43:0/4152089175 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4b3019aa50 con 0x7f4b14038fc0 2024-03-26T08:22:11.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:11.126+0000 7f4b26a00700 1 -- 172.21.15.43:0/4152089175 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7f4b3019aa50 con 0x7f4b14038fc0 2024-03-26T08:22:11.121 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:22:11.121 INFO:teuthology.orchestra.run.smithi043.stdout:[{"addr": "172.21.15.43", "hostname": "smithi043", "labels": [], "status": ""}, {"addr": "172.21.15.52", "hostname": "smithi052", "labels": [], "status": ""}] 2024-03-26T08:22:11.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:11.129+0000 7f4b2f200700 1 -- 172.21.15.43:0/4152089175 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f4b14038fc0 msgr2=0x7f4b1403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:11.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:11.129+0000 7f4b2f200700 1 --2- 172.21.15.43:0/4152089175 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f4b14038fc0 0x7f4b1403b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c006fd0 tx=0x7f4b1c006e40 comp rx=0 tx=0).stop 2024-03-26T08:22:11.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:11.129+0000 7f4b2f200700 1 -- 172.21.15.43:0/4152089175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b300745c0 msgr2=0x7f4b30199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:11.129+0000 7f4b2f200700 1 --2- 172.21.15.43:0/4152089175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b300745c0 0x7f4b30199680 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4b20006010 tx=0x7f4b20005780 comp rx=0 tx=0).stop 2024-03-26T08:22:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:11.129+0000 7f4b2f200700 1 -- 172.21.15.43:0/4152089175 shutdown_connections 2024-03-26T08:22:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:11.129+0000 7f4b2f200700 1 --2- 172.21.15.43:0/4152089175 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f4b14038fc0 0x7f4b1403b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:11.129+0000 7f4b2f200700 1 --2- 172.21.15.43:0/4152089175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b300745c0 0x7f4b30199680 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:11.129+0000 7f4b2f200700 1 -- 172.21.15.43:0/4152089175 >> 172.21.15.43:0/4152089175 conn(0x7f4b300fd4c0 msgr2=0x7f4b30107c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:22:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:11.129+0000 7f4b2f200700 1 -- 172.21.15.43:0/4152089175 shutdown_connections 2024-03-26T08:22:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:11.130+0000 7f4b2f200700 1 -- 172.21.15.43:0/4152089175 wait complete. 2024-03-26T08:22:11.854 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:11 smithi043 ceph-mon[31929]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:11.854 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:11 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:11.921 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi183 2024-03-26T08:22:11.921 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-03-26T08:22:11.921 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.conf 2024-03-26T08:22:11.978 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-03-26T08:22:11.979 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T08:22:12.075 INFO:tasks.cephadm:Adding host smithi183 to orchestrator... 2024-03-26T08:22:12.075 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph orch host add smithi183 2024-03-26T08:22:12.529 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:22:12.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:12 smithi043 ceph-mon[31929]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:22:13.818 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:13 smithi043 ceph-mon[31929]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:14.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.599+0000 7f0dbba00700 1 -- 172.21.15.43:0/1515783921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0db4103ff0 msgr2=0x7f0db41043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:14.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.599+0000 7f0dbba00700 1 --2- 172.21.15.43:0/1515783921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0db4103ff0 0x7f0db41043d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0da8009910 tx=0x7f0da8009c20 comp rx=0 tx=0).stop 2024-03-26T08:22:14.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.599+0000 7f0dbba00700 1 -- 172.21.15.43:0/1515783921 shutdown_connections 2024-03-26T08:22:14.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.599+0000 7f0dbba00700 1 --2- 172.21.15.43:0/1515783921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0db4103ff0 0x7f0db41043d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:14.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.599+0000 7f0dbba00700 1 -- 172.21.15.43:0/1515783921 >> 172.21.15.43:0/1515783921 conn(0x7f0db40ff880 msgr2=0x7f0db4101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:22:14.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.600+0000 7f0dbba00700 1 -- 172.21.15.43:0/1515783921 shutdown_connections 2024-03-26T08:22:14.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.600+0000 7f0dbba00700 1 -- 172.21.15.43:0/1515783921 wait complete. 2024-03-26T08:22:14.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.601+0000 7f0dbba00700 1 Processor -- start 2024-03-26T08:22:14.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.601+0000 7f0dbba00700 1 -- start start 2024-03-26T08:22:14.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.602+0000 7f0dbba00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0db4103ff0 0x7f0db419f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:22:14.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.602+0000 7f0dbba00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db410e270 con 0x7f0db4103ff0 2024-03-26T08:22:14.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.602+0000 7f0db9000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0db4103ff0 0x7f0db419f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:22:14.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.602+0000 7f0db9000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0db4103ff0 0x7f0db419f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35298/0 (socket says 172.21.15.43:35298) 2024-03-26T08:22:14.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.602+0000 7f0db9000700 1 -- 172.21.15.43:0/3376249134 learned_addr learned my addr 172.21.15.43:0/3376249134 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:22:14.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.603+0000 7f0db9000700 1 -- 172.21.15.43:0/3376249134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db419f970 con 0x7f0db4103ff0 2024-03-26T08:22:14.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.603+0000 7f0db9000700 1 --2- 172.21.15.43:0/3376249134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0db4103ff0 0x7f0db419f430 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0da8006010 tx=0x7f0da8005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:22:14.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.603+0000 7f0db1600700 1 -- 172.21.15.43:0/3376249134 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0da8026070 con 0x7f0db4103ff0 2024-03-26T08:22:14.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.603+0000 7f0db1600700 1 -- 172.21.15.43:0/3376249134 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0da800bba0 con 0x7f0db4103ff0 2024-03-26T08:22:14.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.603+0000 7f0db1600700 1 -- 172.21.15.43:0/3376249134 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0da80038a0 con 0x7f0db4103ff0 2024-03-26T08:22:14.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.603+0000 7f0dbba00700 1 -- 172.21.15.43:0/3376249134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db419fc60 con 0x7f0db4103ff0 2024-03-26T08:22:14.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.603+0000 7f0dbba00700 1 -- 172.21.15.43:0/3376249134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db41995f0 con 0x7f0db4103ff0 2024-03-26T08:22:14.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.605+0000 7f0dbba00700 1 -- 172.21.15.43:0/3376249134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db41054d0 con 0x7f0db4103ff0 2024-03-26T08:22:14.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.605+0000 7f0db1600700 1 -- 172.21.15.43:0/3376249134 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f0da8029070 con 0x7f0db4103ff0 2024-03-26T08:22:14.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.605+0000 7f0db1600700 1 --2- 172.21.15.43:0/3376249134 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0d9c039020 0x7f0d9c03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:22:14.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.605+0000 7f0db1600700 1 -- 172.21.15.43:0/3376249134 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0da801e070 con 0x7f0db4103ff0 2024-03-26T08:22:14.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.606+0000 7f0db3e00700 1 --2- 172.21.15.43:0/3376249134 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0d9c039020 0x7f0d9c03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:22:14.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.606+0000 7f0db3e00700 1 --2- 172.21.15.43:0/3376249134 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0d9c039020 0x7f0d9c03b4e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0da0006fd0 tx=0x7f0da0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:22:14.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.611+0000 7f0db1600700 1 -- 172.21.15.43:0/3376249134 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0da8052020 con 0x7f0db4103ff0 2024-03-26T08:22:14.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:14.841+0000 7f0dbba00700 1 -- 172.21.15.43:0/3376249134 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}) v1 -- 0x7f0db419ab20 con 0x7f0d9c039020 2024-03-26T08:22:16.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:15 smithi043 ceph-mon[31929]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:16.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:15 smithi043 ceph-mon[31929]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:22:17.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:17 smithi043 ceph-mon[31929]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:17.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:17 smithi043 ceph-mon[31929]: Deploying cephadm binary to smithi183 2024-03-26T08:22:17.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:17 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:17.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:17 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:17.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:17 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:17.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:17 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:17.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:17 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi052", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:22:17.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:17 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:22:17.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:17 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:22:17.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:17 smithi043 ceph-mon[31929]: Updating smithi052:/etc/ceph/ceph.conf 2024-03-26T08:22:18.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:18 smithi043 ceph-mon[31929]: Updating smithi052:/var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:22:19.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:19 smithi043 ceph-mon[31929]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:21.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:21.597+0000 7f0db1600700 1 -- 172.21.15.43:0/3376249134 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f0db419ab20 con 0x7f0d9c039020 2024-03-26T08:22:21.593 INFO:teuthology.orchestra.run.smithi043.stdout:Added host 'smithi183' with addr '172.21.15.183' 2024-03-26T08:22:21.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:21.601+0000 7f0dbba00700 1 -- 172.21.15.43:0/3376249134 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0d9c039020 msgr2=0x7f0d9c03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:21.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:21.601+0000 7f0dbba00700 1 --2- 172.21.15.43:0/3376249134 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0d9c039020 0x7f0d9c03b4e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0da0006fd0 tx=0x7f0da0006e40 comp rx=0 tx=0).stop 2024-03-26T08:22:21.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:21.601+0000 7f0dbba00700 1 -- 172.21.15.43:0/3376249134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0db4103ff0 msgr2=0x7f0db419f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:21.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:21.601+0000 7f0dbba00700 1 --2- 172.21.15.43:0/3376249134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0db4103ff0 0x7f0db419f430 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0da8006010 tx=0x7f0da8005780 comp rx=0 tx=0).stop 2024-03-26T08:22:21.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:21.602+0000 7f0dbba00700 1 -- 172.21.15.43:0/3376249134 shutdown_connections 2024-03-26T08:22:21.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:21.602+0000 7f0dbba00700 1 --2- 172.21.15.43:0/3376249134 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0d9c039020 0x7f0d9c03b4e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:21.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:21.602+0000 7f0dbba00700 1 --2- 172.21.15.43:0/3376249134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0db4103ff0 0x7f0db419f430 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:21.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:21.602+0000 7f0dbba00700 1 -- 172.21.15.43:0/3376249134 >> 172.21.15.43:0/3376249134 conn(0x7f0db40ff880 msgr2=0x7f0db41059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:22:21.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:21.602+0000 7f0dbba00700 1 -- 172.21.15.43:0/3376249134 shutdown_connections 2024-03-26T08:22:21.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:21.602+0000 7f0dbba00700 1 -- 172.21.15.43:0/3376249134 wait complete. 2024-03-26T08:22:21.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:21 smithi043 ceph-mon[31929]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:21.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:21 smithi043 ceph-mon[31929]: Updating smithi052:/etc/ceph/ceph.client.admin.keyring 2024-03-26T08:22:22.342 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph orch host ls --format=json 2024-03-26T08:22:22.787 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:22:22.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:22 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:22.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:22 smithi043 ceph-mon[31929]: Added host smithi183 2024-03-26T08:22:22.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:22 smithi043 ceph-mon[31929]: Updating smithi052:/var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T08:22:23.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:23 smithi043 ceph-mon[31929]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:23.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:23 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:23.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:23 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:23.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:23 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:23.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:23 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:22:24.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:24.838+0000 7f855fe00700 1 -- 172.21.15.43:0/2990918425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8560106120 msgr2=0x7f8560106500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:24.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:24.838+0000 7f855fe00700 1 --2- 172.21.15.43:0/2990918425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8560106120 0x7f8560106500 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f855000aa20 tx=0x7f855000ad30 comp rx=0 tx=0).stop 2024-03-26T08:22:24.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:24.839+0000 7f855fe00700 1 -- 172.21.15.43:0/2990918425 shutdown_connections 2024-03-26T08:22:24.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:24.839+0000 7f855fe00700 1 --2- 172.21.15.43:0/2990918425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8560106120 0x7f8560106500 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:24.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:24.839+0000 7f855fe00700 1 -- 172.21.15.43:0/2990918425 >> 172.21.15.43:0/2990918425 conn(0x7f85600fb6e0 msgr2=0x7f85600fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:22:24.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:24.839+0000 7f855fe00700 1 -- 172.21.15.43:0/2990918425 shutdown_connections 2024-03-26T08:22:24.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:24.839+0000 7f855fe00700 1 -- 172.21.15.43:0/2990918425 wait complete. 2024-03-26T08:22:24.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:24.840+0000 7f855fe00700 1 Processor -- start 2024-03-26T08:22:24.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:24.840+0000 7f855fe00700 1 -- start start 2024-03-26T08:22:24.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:24.840+0000 7f855fe00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8560106120 0x7f8560199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:22:24.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:24.840+0000 7f855fe00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8560109ce0 con 0x7f8560106120 2024-03-26T08:22:24.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:24.841+0000 7f855ea00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8560106120 0x7f8560199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:22:24.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:24.841+0000 7f855ea00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8560106120 0x7f8560199590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42246/0 (socket says 172.21.15.43:42246) 2024-03-26T08:22:24.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:24.841+0000 7f855ea00700 1 -- 172.21.15.43:0/196497262 learned_addr learned my addr 172.21.15.43:0/196497262 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:22:24.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:24.841+0000 7f855ea00700 1 -- 172.21.15.43:0/196497262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8560199ad0 con 0x7f8560106120 2024-03-26T08:22:24.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:24.841+0000 7f855ea00700 1 --2- 172.21.15.43:0/196497262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8560106120 0x7f8560199590 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f855000c6d0 tx=0x7f8550004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:22:24.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:24.842+0000 7f8557400700 1 -- 172.21.15.43:0/196497262 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8550005410 con 0x7f8560106120 2024-03-26T08:22:24.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:24.842+0000 7f8557400700 1 -- 172.21.15.43:0/196497262 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f85500055b0 con 0x7f8560106120 2024-03-26T08:22:24.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:24.842+0000 7f8557400700 1 -- 172.21.15.43:0/196497262 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f855000e670 con 0x7f8560106120 2024-03-26T08:22:24.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:24.842+0000 7f855fe00700 1 -- 172.21.15.43:0/196497262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8560199d60 con 0x7f8560106120 2024-03-26T08:22:24.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:24.842+0000 7f855fe00700 1 -- 172.21.15.43:0/196497262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f856019de60 con 0x7f8560106120 2024-03-26T08:22:24.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:24.843+0000 7f855fe00700 1 -- 172.21.15.43:0/196497262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8560101740 con 0x7f8560106120 2024-03-26T08:22:24.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:24.843+0000 7f8557400700 1 -- 172.21.15.43:0/196497262 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f8550016a40 con 0x7f8560106120 2024-03-26T08:22:24.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:24.844+0000 7f8557400700 1 --2- 172.21.15.43:0/196497262 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8544039020 0x7f854403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:22:24.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:24.844+0000 7f8557400700 1 -- 172.21.15.43:0/196497262 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f855001e070 con 0x7f8560106120 2024-03-26T08:22:24.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:24.844+0000 7f855e000700 1 --2- 172.21.15.43:0/196497262 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8544039020 0x7f854403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:22:24.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:24.844+0000 7f855e000700 1 --2- 172.21.15.43:0/196497262 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8544039020 0x7f854403b4e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f854c006fd0 tx=0x7f854c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:22:24.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:24.850+0000 7f8557400700 1 -- 172.21.15.43:0/196497262 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f85500153d0 con 0x7f8560106120 2024-03-26T08:22:25.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:25.085+0000 7f855fe00700 1 -- 172.21.15.43:0/196497262 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f856019a8f0 con 0x7f8544039020 2024-03-26T08:22:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:25.087+0000 7f8557400700 1 -- 172.21.15.43:0/196497262 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+239 (secure 0 0 0) 0x7f856019a8f0 con 0x7f8544039020 2024-03-26T08:22:25.082 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:22:25.082 INFO:teuthology.orchestra.run.smithi043.stdout:[{"addr": "172.21.15.43", "hostname": "smithi043", "labels": [], "status": ""}, {"addr": "172.21.15.52", "hostname": "smithi052", "labels": [], "status": ""}, {"addr": "172.21.15.183", "hostname": "smithi183", "labels": [], "status": ""}] 2024-03-26T08:22:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:25.089+0000 7f855fe00700 1 -- 172.21.15.43:0/196497262 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8544039020 msgr2=0x7f854403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:25.089+0000 7f855fe00700 1 --2- 172.21.15.43:0/196497262 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8544039020 0x7f854403b4e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f854c006fd0 tx=0x7f854c006e40 comp rx=0 tx=0).stop 2024-03-26T08:22:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:25.090+0000 7f855fe00700 1 -- 172.21.15.43:0/196497262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8560106120 msgr2=0x7f8560199590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:25.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:25.090+0000 7f855fe00700 1 --2- 172.21.15.43:0/196497262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8560106120 0x7f8560199590 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f855000c6d0 tx=0x7f8550004e20 comp rx=0 tx=0).stop 2024-03-26T08:22:25.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:25.090+0000 7f855fe00700 1 -- 172.21.15.43:0/196497262 shutdown_connections 2024-03-26T08:22:25.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:25.090+0000 7f855fe00700 1 --2- 172.21.15.43:0/196497262 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8544039020 0x7f854403b4e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:25.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:25.090+0000 7f855fe00700 1 --2- 172.21.15.43:0/196497262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8560106120 0x7f8560199590 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:25.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:25.090+0000 7f855fe00700 1 -- 172.21.15.43:0/196497262 >> 172.21.15.43:0/196497262 conn(0x7f85600fb6e0 msgr2=0x7f85600fbac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:22:25.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:25.090+0000 7f855fe00700 1 -- 172.21.15.43:0/196497262 shutdown_connections 2024-03-26T08:22:25.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:25.090+0000 7f855fe00700 1 -- 172.21.15.43:0/196497262 wait complete. 2024-03-26T08:22:25.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:24 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:25.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:24 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:25.879 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-26T08:22:25.880 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph osd crush tunables default 2024-03-26T08:22:26.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:25 smithi043 ceph-mon[31929]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:26.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:25 smithi043 ceph-mon[31929]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:22:26.328 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:22:28.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:27 smithi043 ceph-mon[31929]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:28.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:27 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:28.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.366+0000 7f3759e00700 1 -- 172.21.15.43:0/2789484514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3754102d20 msgr2=0x7f3754103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:28.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.366+0000 7f3759e00700 1 --2- 172.21.15.43:0/2789484514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3754102d20 0x7f3754103100 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3744009910 tx=0x7f3744009c20 comp rx=0 tx=0).stop 2024-03-26T08:22:28.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.367+0000 7f3759e00700 1 -- 172.21.15.43:0/2789484514 shutdown_connections 2024-03-26T08:22:28.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.367+0000 7f3759e00700 1 --2- 172.21.15.43:0/2789484514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3754102d20 0x7f3754103100 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:28.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.367+0000 7f3759e00700 1 -- 172.21.15.43:0/2789484514 >> 172.21.15.43:0/2789484514 conn(0x7f37540776d0 msgr2=0x7f37540fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:22:28.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.367+0000 7f3759e00700 1 -- 172.21.15.43:0/2789484514 shutdown_connections 2024-03-26T08:22:28.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.367+0000 7f3759e00700 1 -- 172.21.15.43:0/2789484514 wait complete. 2024-03-26T08:22:28.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.368+0000 7f3759e00700 1 Processor -- start 2024-03-26T08:22:28.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.368+0000 7f3759e00700 1 -- start start 2024-03-26T08:22:28.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.369+0000 7f3759e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3754102d20 0x7f375419f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:22:28.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.369+0000 7f3759e00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375410c0c0 con 0x7f3754102d20 2024-03-26T08:22:28.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.369+0000 7f3752a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3754102d20 0x7f375419f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:22:28.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.369+0000 7f3752a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3754102d20 0x7f375419f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42258/0 (socket says 172.21.15.43:42258) 2024-03-26T08:22:28.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.369+0000 7f3752a00700 1 -- 172.21.15.43:0/4062527560 learned_addr learned my addr 172.21.15.43:0/4062527560 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:22:28.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.369+0000 7f3752a00700 1 -- 172.21.15.43:0/4062527560 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375419fab0 con 0x7f3754102d20 2024-03-26T08:22:28.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.370+0000 7f3752a00700 1 --2- 172.21.15.43:0/4062527560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3754102d20 0x7f375419f570 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3744006010 tx=0x7f3744005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:22:28.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.370+0000 7f3758a00700 1 -- 172.21.15.43:0/4062527560 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3744026070 con 0x7f3754102d20 2024-03-26T08:22:28.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.370+0000 7f3759e00700 1 -- 172.21.15.43:0/4062527560 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f375419fd40 con 0x7f3754102d20 2024-03-26T08:22:28.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.370+0000 7f3758a00700 1 -- 172.21.15.43:0/4062527560 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f374400bc00 con 0x7f3754102d20 2024-03-26T08:22:28.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.370+0000 7f3758a00700 1 -- 172.21.15.43:0/4062527560 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3744003910 con 0x7f3754102d20 2024-03-26T08:22:28.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.370+0000 7f3759e00700 1 -- 172.21.15.43:0/4062527560 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3754199640 con 0x7f3754102d20 2024-03-26T08:22:28.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.371+0000 7f3758a00700 1 -- 172.21.15.43:0/4062527560 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f3744029070 con 0x7f3754102d20 2024-03-26T08:22:28.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.371+0000 7f3759e00700 1 -- 172.21.15.43:0/4062527560 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3754069e60 con 0x7f3754102d20 2024-03-26T08:22:28.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.372+0000 7f3758a00700 1 --2- 172.21.15.43:0/4062527560 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3738038fc0 0x7f373803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:22:28.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.372+0000 7f3758a00700 1 -- 172.21.15.43:0/4062527560 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f374401e070 con 0x7f3754102d20 2024-03-26T08:22:28.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.372+0000 7f3752000700 1 --2- 172.21.15.43:0/4062527560 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3738038fc0 0x7f373803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:22:28.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.373+0000 7f3752000700 1 --2- 172.21.15.43:0/4062527560 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3738038fc0 0x7f373803b480 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3740006fd0 tx=0x7f3740006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:22:28.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.377+0000 7f3758a00700 1 -- 172.21.15.43:0/4062527560 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3744013070 con 0x7f3754102d20 2024-03-26T08:22:28.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.605+0000 7f3759e00700 1 -- 172.21.15.43:0/4062527560 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f37540637c0 con 0x7f3754102d20 2024-03-26T08:22:28.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.895+0000 7f3758a00700 1 -- 172.21.15.43:0/4062527560 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3744030090 con 0x7f3754102d20 2024-03-26T08:22:28.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.899+0000 7f3759e00700 1 -- 172.21.15.43:0/4062527560 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3738038fc0 msgr2=0x7f373803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:28.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.899+0000 7f3759e00700 1 --2- 172.21.15.43:0/4062527560 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3738038fc0 0x7f373803b480 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3740006fd0 tx=0x7f3740006e40 comp rx=0 tx=0).stop 2024-03-26T08:22:28.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.899+0000 7f3759e00700 1 -- 172.21.15.43:0/4062527560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3754102d20 msgr2=0x7f375419f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:28.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.899+0000 7f3759e00700 1 --2- 172.21.15.43:0/4062527560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3754102d20 0x7f375419f570 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3744006010 tx=0x7f3744005780 comp rx=0 tx=0).stop 2024-03-26T08:22:28.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.899+0000 7f3759e00700 1 -- 172.21.15.43:0/4062527560 shutdown_connections 2024-03-26T08:22:28.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.899+0000 7f3759e00700 1 --2- 172.21.15.43:0/4062527560 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3738038fc0 0x7f373803b480 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:28.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.899+0000 7f3759e00700 1 --2- 172.21.15.43:0/4062527560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3754102d20 0x7f375419f570 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:28.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.899+0000 7f3759e00700 1 -- 172.21.15.43:0/4062527560 >> 172.21.15.43:0/4062527560 conn(0x7f37540776d0 msgr2=0x7f3754108bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:22:28.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.899+0000 7f3759e00700 1 -- 172.21.15.43:0/4062527560 shutdown_connections 2024-03-26T08:22:28.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:22:28.899+0000 7f3759e00700 1 -- 172.21.15.43:0/4062527560 wait complete. 2024-03-26T08:22:28.895 INFO:teuthology.orchestra.run.smithi043.stderr:adjusted tunables profile to default 2024-03-26T08:22:29.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:28 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/4062527560' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-26T08:22:29.671 INFO:tasks.cephadm:Adding mon.a on smithi043 2024-03-26T08:22:29.671 INFO:tasks.cephadm:Adding mon.b on smithi052 2024-03-26T08:22:29.671 INFO:tasks.cephadm:Adding mon.c on smithi183 2024-03-26T08:22:29.672 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph orch apply mon '3;smithi043:172.21.15.43=a;smithi052:172.21.15.52=b;smithi183:172.21.15.183=c' 2024-03-26T08:22:30.084 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-26T08:22:30.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:29 smithi043 ceph-mon[31929]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:30.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:29 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/4062527560' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-26T08:22:30.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:29 smithi043 ceph-mon[31929]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T08:22:30.187 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-26T08:22:32.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:31 smithi043 ceph-mon[31929]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:32.483 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.488+0000 7fdd04600700 1 -- 172.21.15.183:0/4212749458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcfc101bc0 msgr2=0x7fdcfc101fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:32.554 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.488+0000 7fdd04600700 1 --2- 172.21.15.183:0/4212749458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcfc101bc0 0x7fdcfc101fe0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0009910 tx=0x7fdcf0009c20 comp rx=0 tx=0).stop 2024-03-26T08:22:32.554 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.488+0000 7fdd04600700 1 -- 172.21.15.183:0/4212749458 shutdown_connections 2024-03-26T08:22:32.554 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.488+0000 7fdd04600700 1 --2- 172.21.15.183:0/4212749458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcfc101bc0 0x7fdcfc101fe0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:32.554 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.488+0000 7fdd04600700 1 -- 172.21.15.183:0/4212749458 >> 172.21.15.183:0/4212749458 conn(0x7fdcfc0fd160 msgr2=0x7fdcfc0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:22:32.557 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.489+0000 7fdd04600700 1 -- 172.21.15.183:0/4212749458 shutdown_connections 2024-03-26T08:22:32.557 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.489+0000 7fdd04600700 1 -- 172.21.15.183:0/4212749458 wait complete. 2024-03-26T08:22:32.557 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.490+0000 7fdd04600700 1 Processor -- start 2024-03-26T08:22:32.557 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.490+0000 7fdd04600700 1 -- start start 2024-03-26T08:22:32.557 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.490+0000 7fdd04600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcfc101bc0 0x7fdcfc075e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:22:32.557 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.490+0000 7fdd04600700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc107b00 con 0x7fdcfc101bc0 2024-03-26T08:22:32.557 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.491+0000 7fdd01c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcfc101bc0 0x7fdcfc075e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:22:32.557 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.491+0000 7fdd01c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcfc101bc0 0x7fdcfc075e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.183:60486/0 (socket says 172.21.15.183:60486) 2024-03-26T08:22:32.557 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.491+0000 7fdd01c00700 1 -- 172.21.15.183:0/3112469266 learned_addr learned my addr 172.21.15.183:0/3112469266 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-03-26T08:22:32.557 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.492+0000 7fdd01c00700 1 -- 172.21.15.183:0/3112469266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcfc076390 con 0x7fdcfc101bc0 2024-03-26T08:22:32.557 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.492+0000 7fdd01c00700 1 --2- 172.21.15.183:0/3112469266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcfc101bc0 0x7fdcfc075e50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0005800 tx=0x7fdcf0009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:22:32.558 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.493+0000 7fdcfa000700 1 -- 172.21.15.183:0/3112469266 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdcf0027070 con 0x7fdcfc101bc0 2024-03-26T08:22:32.558 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.493+0000 7fdd04600700 1 -- 172.21.15.183:0/3112469266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcfc074580 con 0x7fdcfc101bc0 2024-03-26T08:22:32.558 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.493+0000 7fdd04600700 1 -- 172.21.15.183:0/3112469266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcfc074ac0 con 0x7fdcfc101bc0 2024-03-26T08:22:32.558 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.493+0000 7fdcfa000700 1 -- 172.21.15.183:0/3112469266 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdcf000be10 con 0x7fdcfc101bc0 2024-03-26T08:22:32.558 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.493+0000 7fdcfa000700 1 -- 172.21.15.183:0/3112469266 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdcf0022e20 con 0x7fdcfc101bc0 2024-03-26T08:22:32.558 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.494+0000 7fdd04600700 1 -- 172.21.15.183:0/3112469266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcdc005360 con 0x7fdcfc101bc0 2024-03-26T08:22:32.558 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.494+0000 7fdcfa000700 1 -- 172.21.15.183:0/3112469266 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fdcf002a070 con 0x7fdcfc101bc0 2024-03-26T08:22:32.558 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.495+0000 7fdcfa000700 1 --2- 172.21.15.183:0/3112469266 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fdce4039020 0x7fdce403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:22:32.558 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.495+0000 7fdcfa000700 1 -- 172.21.15.183:0/3112469266 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdcf001e030 con 0x7fdcfc101bc0 2024-03-26T08:22:32.558 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.495+0000 7fdd01200700 1 --2- 172.21.15.183:0/3112469266 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fdce4039020 0x7fdce403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:22:32.558 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.497+0000 7fdd01200700 1 --2- 172.21.15.183:0/3112469266 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fdce4039020 0x7fdce403b4e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdce8006fd0 tx=0x7fdce8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:22:32.558 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.511+0000 7fdcfa000700 1 -- 172.21.15.183:0/3112469266 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdcf0013070 con 0x7fdcfc101bc0 2024-03-26T08:22:32.742 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.747+0000 7fdd04600700 1 -- 172.21.15.183:0/3112469266 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi052:172.21.15.52=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fdcdc000cb0 con 0x7fdce4039020 2024-03-26T08:22:32.757 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.763+0000 7fdcfa000700 1 -- 172.21.15.183:0/3112469266 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fdcdc000cb0 con 0x7fdce4039020 2024-03-26T08:22:32.758 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled mon update... 2024-03-26T08:22:32.761 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.766+0000 7fdd04600700 1 -- 172.21.15.183:0/3112469266 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fdce4039020 msgr2=0x7fdce403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:32.761 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.766+0000 7fdd04600700 1 --2- 172.21.15.183:0/3112469266 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fdce4039020 0x7fdce403b4e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdce8006fd0 tx=0x7fdce8006e40 comp rx=0 tx=0).stop 2024-03-26T08:22:32.761 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.766+0000 7fdd04600700 1 -- 172.21.15.183:0/3112469266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcfc101bc0 msgr2=0x7fdcfc075e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:32.761 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.766+0000 7fdd04600700 1 --2- 172.21.15.183:0/3112469266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcfc101bc0 0x7fdcfc075e50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0005800 tx=0x7fdcf0009f60 comp rx=0 tx=0).stop 2024-03-26T08:22:32.761 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.766+0000 7fdd04600700 1 -- 172.21.15.183:0/3112469266 shutdown_connections 2024-03-26T08:22:32.761 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.766+0000 7fdd04600700 1 --2- 172.21.15.183:0/3112469266 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fdce4039020 0x7fdce403b4e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:32.761 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.766+0000 7fdd04600700 1 --2- 172.21.15.183:0/3112469266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcfc101bc0 0x7fdcfc075e50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:32.761 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.766+0000 7fdd04600700 1 -- 172.21.15.183:0/3112469266 >> 172.21.15.183:0/3112469266 conn(0x7fdcfc0fd160 msgr2=0x7fdcfc0fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:22:32.761 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.766+0000 7fdd04600700 1 -- 172.21.15.183:0/3112469266 shutdown_connections 2024-03-26T08:22:32.761 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:32.766+0000 7fdd04600700 1 -- 172.21.15.183:0/3112469266 wait complete. 2024-03-26T08:22:33.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:33 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:33.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:33 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:33.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:33 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:33.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:33 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:33.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:33 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:22:33.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:33 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:22:33.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:33 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:22:33.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:33 smithi043 ceph-mon[31929]: Updating smithi183:/etc/ceph/ceph.conf 2024-03-26T08:22:33.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:33 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:33.430 DEBUG:teuthology.orchestra.run.smithi052:mon.b> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mon.b.service 2024-03-26T08:22:33.438 DEBUG:teuthology.orchestra.run.smithi183:mon.c> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mon.c.service 2024-03-26T08:22:33.442 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-26T08:22:33.443 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-26T08:22:33.865 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:22:34.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:34 smithi043 ceph-mon[31929]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:34.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:34 smithi043 ceph-mon[31929]: Updating smithi183:/var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:22:34.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:34 smithi043 ceph-mon[31929]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi052:172.21.15.52=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:22:34.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:34 smithi043 ceph-mon[31929]: Saving service mon spec with placement smithi043:172.21.15.43=a;smithi052:172.21.15.52=b;smithi183:172.21.15.183=c;count:3 2024-03-26T08:22:35.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:35 smithi043 ceph-mon[31929]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-03-26T08:22:35.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:35 smithi043 ceph-mon[31929]: Updating smithi183:/var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T08:22:35.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:35 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:35.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:35 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:35.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:35 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:35.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:35 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:22:35.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:35 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:22:35.762 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:35.767+0000 7f61f5c00700 1 -- 172.21.15.183:0/1933396007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61f0100910 msgr2=0x7f61f0100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:35.762 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:35.767+0000 7f61f5c00700 1 --2- 172.21.15.183:0/1933396007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61f0100910 0x7f61f0100cf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f61e0009910 tx=0x7f61e0009c20 comp rx=0 tx=0).stop 2024-03-26T08:22:35.762 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:35.767+0000 7f61f5c00700 1 -- 172.21.15.183:0/1933396007 shutdown_connections 2024-03-26T08:22:35.762 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:35.767+0000 7f61f5c00700 1 --2- 172.21.15.183:0/1933396007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61f0100910 0x7f61f0100cf0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:35.762 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:35.767+0000 7f61f5c00700 1 -- 172.21.15.183:0/1933396007 >> 172.21.15.183:0/1933396007 conn(0x7f61f00776d0 msgr2=0x7f61f00fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:22:35.762 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:35.767+0000 7f61f5c00700 1 -- 172.21.15.183:0/1933396007 shutdown_connections 2024-03-26T08:22:35.763 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:35.768+0000 7f61f5c00700 1 -- 172.21.15.183:0/1933396007 wait complete. 2024-03-26T08:22:35.763 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:35.769+0000 7f61f5c00700 1 Processor -- start 2024-03-26T08:22:35.763 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:35.769+0000 7f61f5c00700 1 -- start start 2024-03-26T08:22:35.764 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:35.770+0000 7f61f5c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61f0100910 0x7f61f019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:22:35.764 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:35.770+0000 7f61f5c00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f010e2a0 con 0x7f61f0100910 2024-03-26T08:22:35.764 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:35.770+0000 7f61eea00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61f0100910 0x7f61f019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:22:35.764 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:35.770+0000 7f61eea00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61f0100910 0x7f61f019f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.183:60502/0 (socket says 172.21.15.183:60502) 2024-03-26T08:22:35.764 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:35.770+0000 7f61eea00700 1 -- 172.21.15.183:0/2767279802 learned_addr learned my addr 172.21.15.183:0/2767279802 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-03-26T08:22:35.765 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:35.771+0000 7f61eea00700 1 -- 172.21.15.183:0/2767279802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f019fab0 con 0x7f61f0100910 2024-03-26T08:22:35.765 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:35.771+0000 7f61eea00700 1 --2- 172.21.15.183:0/2767279802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61f0100910 0x7f61f019f570 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f61e0006010 tx=0x7f61e0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:22:35.766 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:35.772+0000 7f61e7400700 1 -- 172.21.15.183:0/2767279802 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61e0026070 con 0x7f61f0100910 2024-03-26T08:22:35.766 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:35.772+0000 7f61f5c00700 1 -- 172.21.15.183:0/2767279802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f019fd40 con 0x7f61f0100910 2024-03-26T08:22:35.766 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:35.772+0000 7f61e7400700 1 -- 172.21.15.183:0/2767279802 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f61e000bc00 con 0x7f61f0100910 2024-03-26T08:22:35.766 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:35.772+0000 7f61e7400700 1 -- 172.21.15.183:0/2767279802 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61e0003930 con 0x7f61f0100910 2024-03-26T08:22:35.766 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:35.772+0000 7f61f5c00700 1 -- 172.21.15.183:0/2767279802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f0199640 con 0x7f61f0100910 2024-03-26T08:22:35.767 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:35.773+0000 7f61f5c00700 1 -- 172.21.15.183:0/2767279802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f0069e00 con 0x7f61f0100910 2024-03-26T08:22:35.768 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:35.774+0000 7f61e7400700 1 -- 172.21.15.183:0/2767279802 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f61e0029070 con 0x7f61f0100910 2024-03-26T08:22:35.768 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:35.774+0000 7f61e7400700 1 --2- 172.21.15.183:0/2767279802 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f61d4038fc0 0x7f61d403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:22:35.769 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:35.775+0000 7f61e7400700 1 -- 172.21.15.183:0/2767279802 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f61e001e070 con 0x7f61f0100910 2024-03-26T08:22:35.769 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:35.775+0000 7f61ee000700 1 --2- 172.21.15.183:0/2767279802 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f61d4038fc0 0x7f61d403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:22:35.770 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:35.776+0000 7f61ee000700 1 --2- 172.21.15.183:0/2767279802 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f61d4038fc0 0x7f61d403b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f61dc006fd0 tx=0x7f61dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:22:35.785 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:35.790+0000 7f61e7400700 1 -- 172.21.15.183:0/2767279802 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f61e0013070 con 0x7f61f0100910 2024-03-26T08:22:36.105 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:36.110+0000 7f61f5c00700 1 -- 172.21.15.183:0/2767279802 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f61f019ab70 con 0x7f61f0100910 2024-03-26T08:22:36.106 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:36.111+0000 7f61e7400700 1 -- 172.21.15.183:0/2767279802 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+745 (secure 0 0 0) 0x7f61e0030020 con 0x7f61f0100910 2024-03-26T08:22:36.106 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-26T08:22:36.107 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":1,"fsid":"88759420-eb49-11ee-95d0-87774f69a715","modified":"2024-03-26T08:20:23.553863Z","created":"2024-03-26T08:20:23.553863Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-26T08:22:36.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:36.114+0000 7f61f5c00700 1 -- 172.21.15.183:0/2767279802 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f61d4038fc0 msgr2=0x7f61d403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:36.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:36.114+0000 7f61f5c00700 1 --2- 172.21.15.183:0/2767279802 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f61d4038fc0 0x7f61d403b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f61dc006fd0 tx=0x7f61dc006e40 comp rx=0 tx=0).stop 2024-03-26T08:22:36.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:36.114+0000 7f61f5c00700 1 -- 172.21.15.183:0/2767279802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61f0100910 msgr2=0x7f61f019f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:36.109 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:36.114+0000 7f61f5c00700 1 --2- 172.21.15.183:0/2767279802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61f0100910 0x7f61f019f570 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f61e0006010 tx=0x7f61e0005780 comp rx=0 tx=0).stop 2024-03-26T08:22:36.110 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:36.114+0000 7f61f5c00700 1 -- 172.21.15.183:0/2767279802 shutdown_connections 2024-03-26T08:22:36.110 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:36.114+0000 7f61f5c00700 1 --2- 172.21.15.183:0/2767279802 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f61d4038fc0 0x7f61d403b480 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:36.110 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:36.114+0000 7f61f5c00700 1 --2- 172.21.15.183:0/2767279802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61f0100910 0x7f61f019f570 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:36.110 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:36.114+0000 7f61f5c00700 1 -- 172.21.15.183:0/2767279802 >> 172.21.15.183:0/2767279802 conn(0x7f61f00776d0 msgr2=0x7f61f00fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:22:36.110 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:36.114+0000 7f61f5c00700 1 -- 172.21.15.183:0/2767279802 shutdown_connections 2024-03-26T08:22:36.110 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:36.115+0000 7f61f5c00700 1 -- 172.21.15.183:0/2767279802 wait complete. 2024-03-26T08:22:36.111 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 1 2024-03-26T08:22:36.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:36 smithi043 ceph-mon[31929]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:36.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:36 smithi043 ceph-mon[31929]: Deploying daemon mon.c on smithi183 2024-03-26T08:22:37.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:37 smithi043 ceph-mon[31929]: from='client.? 172.21.15.183:0/2767279802' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T08:22:38.099 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-26T08:22:38.099 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-26T08:22:38.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:38 smithi043 ceph-mon[31929]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:38.663 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:22:40.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:40 smithi043 ceph-mon[31929]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:41.226 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 podman[35790]: 2024-03-26 08:22:41.230114931 +0000 UTC m=+0.694692995 container init fc6a11acaf9f95d107e7bf0c94c13f337e6d46e9944411d7415380b9a318575a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-c, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T08:22:41.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.290+0000 7fc6b3c00700 1 -- 172.21.15.183:0/3063145791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6ac100a00 msgr2=0x7fc6ac100de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:41.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.290+0000 7fc6b3c00700 1 --2- 172.21.15.183:0/3063145791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6ac100a00 0x7fc6ac100de0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009910 tx=0x7fc6a0009c20 comp rx=0 tx=0).stop 2024-03-26T08:22:41.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.290+0000 7fc6b3c00700 1 -- 172.21.15.183:0/3063145791 shutdown_connections 2024-03-26T08:22:41.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.290+0000 7fc6b3c00700 1 --2- 172.21.15.183:0/3063145791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6ac100a00 0x7fc6ac100de0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:41.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.290+0000 7fc6b3c00700 1 -- 172.21.15.183:0/3063145791 >> 172.21.15.183:0/3063145791 conn(0x7fc6ac0fc640 msgr2=0x7fc6ac0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:22:41.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.290+0000 7fc6b3c00700 1 -- 172.21.15.183:0/3063145791 shutdown_connections 2024-03-26T08:22:41.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.290+0000 7fc6b3c00700 1 -- 172.21.15.183:0/3063145791 wait complete. 2024-03-26T08:22:41.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.291+0000 7fc6b3c00700 1 Processor -- start 2024-03-26T08:22:41.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.291+0000 7fc6b3c00700 1 -- start start 2024-03-26T08:22:41.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.292+0000 7fc6b3c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6ac100a00 0x7fc6ac19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:22:41.356 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.292+0000 7fc6b3c00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac10e390 con 0x7fc6ac100a00 2024-03-26T08:22:41.357 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.292+0000 7fc6b1200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6ac100a00 0x7fc6ac19f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:22:41.357 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.293+0000 7fc6b1200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6ac100a00 0x7fc6ac19f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.183:41470/0 (socket says 172.21.15.183:41470) 2024-03-26T08:22:41.357 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.293+0000 7fc6b1200700 1 -- 172.21.15.183:0/1633033327 learned_addr learned my addr 172.21.15.183:0/1633033327 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-03-26T08:22:41.357 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.293+0000 7fc6b1200700 1 -- 172.21.15.183:0/1633033327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ac19fb90 con 0x7fc6ac100a00 2024-03-26T08:22:41.357 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.294+0000 7fc6b1200700 1 --2- 172.21.15.183:0/1633033327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6ac100a00 0x7fc6ac19f650 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0006010 tx=0x7fc6a0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:22:41.357 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.294+0000 7fc6a9600700 1 -- 172.21.15.183:0/1633033327 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6a0026070 con 0x7fc6ac100a00 2024-03-26T08:22:41.357 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.294+0000 7fc6b3c00700 1 -- 172.21.15.183:0/1633033327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6ac19fe20 con 0x7fc6ac100a00 2024-03-26T08:22:41.357 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.294+0000 7fc6b3c00700 1 -- 172.21.15.183:0/1633033327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6ac199720 con 0x7fc6ac100a00 2024-03-26T08:22:41.357 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.294+0000 7fc6a9600700 1 -- 172.21.15.183:0/1633033327 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc6a000bc00 con 0x7fc6ac100a00 2024-03-26T08:22:41.357 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.294+0000 7fc6a9600700 1 -- 172.21.15.183:0/1633033327 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6a0003930 con 0x7fc6ac100a00 2024-03-26T08:22:41.357 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.296+0000 7fc6b3c00700 1 -- 172.21.15.183:0/1633033327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6ac06a360 con 0x7fc6ac100a00 2024-03-26T08:22:41.357 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.296+0000 7fc6a9600700 1 -- 172.21.15.183:0/1633033327 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fc6a0029070 con 0x7fc6ac100a00 2024-03-26T08:22:41.357 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.297+0000 7fc6a9600700 1 --2- 172.21.15.183:0/1633033327 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fc694039020 0x7fc69403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:22:41.358 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.297+0000 7fc6a9600700 1 -- 172.21.15.183:0/1633033327 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc6a001e070 con 0x7fc6ac100a00 2024-03-26T08:22:41.358 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.297+0000 7fc6abe00700 1 --2- 172.21.15.183:0/1633033327 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fc694039020 0x7fc69403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:22:41.358 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.298+0000 7fc6abe00700 1 --2- 172.21.15.183:0/1633033327 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fc694039020 0x7fc69403b4e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc69c006fd0 tx=0x7fc69c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:22:41.358 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.313+0000 7fc6a9600700 1 -- 172.21.15.183:0/1633033327 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc6a0013070 con 0x7fc6ac100a00 2024-03-26T08:22:41.358 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.353+0000 7fc6a9600700 1 -- 172.21.15.183:0/1633033327 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fc6a002ac40 con 0x7fc6ac100a00 2024-03-26T08:22:41.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 podman[35790]: 2024-03-26 08:22:41.240700247 +0000 UTC m=+0.705278274 container start fc6a11acaf9f95d107e7bf0c94c13f337e6d46e9944411d7415380b9a318575a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240319) 2024-03-26T08:22:41.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: set uid:gid to 167:167 (ceph:ceph) 2024-03-26T08:22:41.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev), process ceph-mon, pid 2 2024-03-26T08:22:41.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: pidfile_write: ignore empty --pid-file 2024-03-26T08:22:41.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: load: jerasure load: lrc 2024-03-26T08:22:41.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: RocksDB version: 7.9.2 2024-03-26T08:22:41.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Git sha 0 2024-03-26T08:22:41.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Compile date 2024-03-22 04:58:55 2024-03-26T08:22:41.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: DB SUMMARY 2024-03-26T08:22:41.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: DB Session ID: T847L44A28IETQ7AMVMW 2024-03-26T08:22:41.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: CURRENT file: CURRENT 2024-03-26T08:22:41.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: IDENTITY file: IDENTITY 2024-03-26T08:22:41.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-26T08:22:41.553 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-26T08:22:41.553 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-03-26T08:22:41.553 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.error_if_exists: 0 2024-03-26T08:22:41.553 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.create_if_missing: 0 2024-03-26T08:22:41.553 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.paranoid_checks: 1 2024-03-26T08:22:41.553 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-26T08:22:41.553 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-26T08:22:41.553 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-26T08:22:41.553 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.env: 0x5613c725eb20 2024-03-26T08:22:41.553 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.fs: PosixFileSystem 2024-03-26T08:22:41.553 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.info_log: 0x5613c95c3960 2024-03-26T08:22:41.553 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.max_file_opening_threads: 16 2024-03-26T08:22:41.553 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.statistics: (nil) 2024-03-26T08:22:41.553 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.use_fsync: 0 2024-03-26T08:22:41.554 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.max_log_file_size: 0 2024-03-26T08:22:41.554 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-26T08:22:41.554 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-26T08:22:41.554 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.keep_log_file_num: 1000 2024-03-26T08:22:41.554 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.recycle_log_file_num: 0 2024-03-26T08:22:41.554 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.allow_fallocate: 1 2024-03-26T08:22:41.554 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.allow_mmap_reads: 0 2024-03-26T08:22:41.554 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.allow_mmap_writes: 0 2024-03-26T08:22:41.555 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.use_direct_reads: 0 2024-03-26T08:22:41.555 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-26T08:22:41.555 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.create_missing_column_families: 0 2024-03-26T08:22:41.555 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.db_log_dir: 2024-03-26T08:22:41.555 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.wal_dir: 2024-03-26T08:22:41.555 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-26T08:22:41.555 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-26T08:22:41.555 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-26T08:22:41.555 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-26T08:22:41.555 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-26T08:22:41.555 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-26T08:22:41.555 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.advise_random_on_open: 1 2024-03-26T08:22:41.555 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.db_write_buffer_size: 0 2024-03-26T08:22:41.555 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.write_buffer_manager: 0x5613c95961e0 2024-03-26T08:22:41.556 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-26T08:22:41.556 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-26T08:22:41.556 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-26T08:22:41.556 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.rate_limiter: (nil) 2024-03-26T08:22:41.556 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-26T08:22:41.556 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.wal_recovery_mode: 2 2024-03-26T08:22:41.556 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.enable_thread_tracking: 0 2024-03-26T08:22:41.556 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.enable_pipelined_write: 0 2024-03-26T08:22:41.556 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.unordered_write: 0 2024-03-26T08:22:41.556 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-26T08:22:41.556 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-26T08:22:41.556 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-26T08:22:41.556 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-26T08:22:41.556 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.row_cache: None 2024-03-26T08:22:41.556 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.wal_filter: None 2024-03-26T08:22:41.557 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-26T08:22:41.557 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.allow_ingest_behind: 0 2024-03-26T08:22:41.557 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.two_write_queues: 0 2024-03-26T08:22:41.557 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.manual_wal_flush: 0 2024-03-26T08:22:41.557 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.wal_compression: 0 2024-03-26T08:22:41.557 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.atomic_flush: 0 2024-03-26T08:22:41.557 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-26T08:22:41.557 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-26T08:22:41.557 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-26T08:22:41.557 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.log_readahead_size: 0 2024-03-26T08:22:41.557 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-26T08:22:41.557 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.best_efforts_recovery: 0 2024-03-26T08:22:41.557 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-26T08:22:41.557 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-26T08:22:41.557 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.allow_data_in_errors: 0 2024-03-26T08:22:41.557 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.db_host_id: __hostname__ 2024-03-26T08:22:41.558 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-26T08:22:41.558 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.max_background_jobs: 2 2024-03-26T08:22:41.558 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.max_background_compactions: -1 2024-03-26T08:22:41.558 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.max_subcompactions: 1 2024-03-26T08:22:41.558 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-26T08:22:41.558 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-26T08:22:41.558 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-26T08:22:41.558 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.max_total_wal_size: 0 2024-03-26T08:22:41.558 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-26T08:22:41.558 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-26T08:22:41.558 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-26T08:22:41.558 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-26T08:22:41.558 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.max_open_files: -1 2024-03-26T08:22:41.558 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.bytes_per_sync: 0 2024-03-26T08:22:41.558 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-26T08:22:41.559 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-26T08:22:41.559 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.compaction_readahead_size: 0 2024-03-26T08:22:41.559 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.max_background_flushes: -1 2024-03-26T08:22:41.559 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Compression algorithms supported: 2024-03-26T08:22:41.559 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-26T08:22:41.559 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: kZSTD supported: 0 2024-03-26T08:22:41.559 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: kXpressCompression supported: 0 2024-03-26T08:22:41.559 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: kLZ4HCCompression supported: 1 2024-03-26T08:22:41.559 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: kZlibCompression supported: 1 2024-03-26T08:22:41.559 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: kSnappyCompression supported: 1 2024-03-26T08:22:41.559 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: kLZ4Compression supported: 1 2024-03-26T08:22:41.559 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: kBZip2Compression supported: 0 2024-03-26T08:22:41.559 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-26T08:22:41.559 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-26T08:22:41.559 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-26T08:22:41.560 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-26T08:22:41.560 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-26T08:22:41.560 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.merge_operator: 2024-03-26T08:22:41.561 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.compaction_filter: None 2024-03-26T08:22:41.561 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.compaction_filter_factory: None 2024-03-26T08:22:41.561 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.sst_partitioner_factory: None 2024-03-26T08:22:41.561 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-26T08:22:41.561 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-26T08:22:41.561 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5613c95c3a60) 2024-03-26T08:22:41.561 INFO:journalctl@ceph.mon.c.smithi183.stdout: cache_index_and_filter_blocks: 1 2024-03-26T08:22:41.561 INFO:journalctl@ceph.mon.c.smithi183.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-26T08:22:41.561 INFO:journalctl@ceph.mon.c.smithi183.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-26T08:22:41.562 INFO:journalctl@ceph.mon.c.smithi183.stdout: pin_top_level_index_and_filter: 1 2024-03-26T08:22:41.562 INFO:journalctl@ceph.mon.c.smithi183.stdout: index_type: 0 2024-03-26T08:22:41.562 INFO:journalctl@ceph.mon.c.smithi183.stdout: data_block_index_type: 0 2024-03-26T08:22:41.562 INFO:journalctl@ceph.mon.c.smithi183.stdout: index_shortening: 1 2024-03-26T08:22:41.562 INFO:journalctl@ceph.mon.c.smithi183.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-26T08:22:41.562 INFO:journalctl@ceph.mon.c.smithi183.stdout: checksum: 4 2024-03-26T08:22:41.562 INFO:journalctl@ceph.mon.c.smithi183.stdout: no_block_cache: 0 2024-03-26T08:22:41.562 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_cache: 0x5613c95c9090 2024-03-26T08:22:41.562 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_cache_name: BinnedLRUCache 2024-03-26T08:22:41.562 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_cache_options: 2024-03-26T08:22:41.562 INFO:journalctl@ceph.mon.c.smithi183.stdout: capacity : 536870912 2024-03-26T08:22:41.562 INFO:journalctl@ceph.mon.c.smithi183.stdout: num_shard_bits : 4 2024-03-26T08:22:41.562 INFO:journalctl@ceph.mon.c.smithi183.stdout: strict_capacity_limit : 0 2024-03-26T08:22:41.562 INFO:journalctl@ceph.mon.c.smithi183.stdout: high_pri_pool_ratio: 0.000 2024-03-26T08:22:41.563 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_cache_compressed: (nil) 2024-03-26T08:22:41.563 INFO:journalctl@ceph.mon.c.smithi183.stdout: persistent_cache: (nil) 2024-03-26T08:22:41.563 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_size: 4096 2024-03-26T08:22:41.563 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_size_deviation: 10 2024-03-26T08:22:41.563 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_restart_interval: 16 2024-03-26T08:22:41.563 INFO:journalctl@ceph.mon.c.smithi183.stdout: index_block_restart_interval: 1 2024-03-26T08:22:41.563 INFO:journalctl@ceph.mon.c.smithi183.stdout: metadata_block_size: 4096 2024-03-26T08:22:41.563 INFO:journalctl@ceph.mon.c.smithi183.stdout: partition_filters: 0 2024-03-26T08:22:41.563 INFO:journalctl@ceph.mon.c.smithi183.stdout: use_delta_encoding: 1 2024-03-26T08:22:41.563 INFO:journalctl@ceph.mon.c.smithi183.stdout: filter_policy: bloomfilter 2024-03-26T08:22:41.563 INFO:journalctl@ceph.mon.c.smithi183.stdout: whole_key_filtering: 1 2024-03-26T08:22:41.563 INFO:journalctl@ceph.mon.c.smithi183.stdout: verify_compression: 0 2024-03-26T08:22:41.563 INFO:journalctl@ceph.mon.c.smithi183.stdout: read_amp_bytes_per_bit: 0 2024-03-26T08:22:41.563 INFO:journalctl@ceph.mon.c.smithi183.stdout: format_version: 5 2024-03-26T08:22:41.563 INFO:journalctl@ceph.mon.c.smithi183.stdout: enable_index_compression: 1 2024-03-26T08:22:41.564 INFO:journalctl@ceph.mon.c.smithi183.stdout: block_align: 0 2024-03-26T08:22:41.564 INFO:journalctl@ceph.mon.c.smithi183.stdout: max_auto_readahead_size: 262144 2024-03-26T08:22:41.564 INFO:journalctl@ceph.mon.c.smithi183.stdout: prepopulate_block_cache: 0 2024-03-26T08:22:41.564 INFO:journalctl@ceph.mon.c.smithi183.stdout: initial_auto_readahead_size: 8192 2024-03-26T08:22:41.564 INFO:journalctl@ceph.mon.c.smithi183.stdout: num_file_reads_for_auto_readahead: 2 2024-03-26T08:22:41.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.write_buffer_size: 33554432 2024-03-26T08:22:41.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.max_write_buffer_number: 2 2024-03-26T08:22:41.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.compression: NoCompression 2024-03-26T08:22:41.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.bottommost_compression: Disabled 2024-03-26T08:22:41.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.prefix_extractor: nullptr 2024-03-26T08:22:41.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-26T08:22:41.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.num_levels: 7 2024-03-26T08:22:41.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-26T08:22:41.564 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-26T08:22:41.565 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-26T08:22:41.565 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-26T08:22:41.565 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-26T08:22:41.565 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-26T08:22:41.565 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-26T08:22:41.565 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-26T08:22:41.565 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-26T08:22:41.565 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-26T08:22:41.565 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-26T08:22:41.565 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-26T08:22:41.565 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-26T08:22:41.565 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.compression_opts.level: 32767 2024-03-26T08:22:41.565 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.compression_opts.strategy: 0 2024-03-26T08:22:41.565 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-26T08:22:41.565 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-26T08:22:41.565 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-26T08:22:41.566 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-26T08:22:41.566 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.compression_opts.enabled: false 2024-03-26T08:22:41.566 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-26T08:22:41.566 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-26T08:22:41.566 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-26T08:22:41.566 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-26T08:22:41.566 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.target_file_size_base: 67108864 2024-03-26T08:22:41.566 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-26T08:22:41.566 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-26T08:22:41.566 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-26T08:22:41.566 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-26T08:22:41.566 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-26T08:22:41.566 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-26T08:22:41.567 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-26T08:22:41.567 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-26T08:22:41.567 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-26T08:22:41.567 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-26T08:22:41.567 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-26T08:22:41.567 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-26T08:22:41.567 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-26T08:22:41.567 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-26T08:22:41.567 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.arena_block_size: 1048576 2024-03-26T08:22:41.567 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-26T08:22:41.567 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-26T08:22:41.567 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.disable_auto_compactions: 0 2024-03-26T08:22:41.567 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-26T08:22:41.567 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-26T08:22:41.568 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-26T08:22:41.568 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-26T08:22:41.568 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-26T08:22:41.568 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-26T08:22:41.568 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-26T08:22:41.568 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-26T08:22:41.568 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-26T08:22:41.568 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-26T08:22:41.568 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.table_properties_collectors: 2024-03-26T08:22:41.568 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.inplace_update_support: 0 2024-03-26T08:22:41.568 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-26T08:22:41.568 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-26T08:22:41.568 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-26T08:22:41.568 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-26T08:22:41.568 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.bloom_locality: 0 2024-03-26T08:22:41.569 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.max_successive_merges: 0 2024-03-26T08:22:41.569 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-26T08:22:41.569 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.paranoid_file_checks: 0 2024-03-26T08:22:41.569 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.force_consistency_checks: 1 2024-03-26T08:22:41.569 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.report_bg_io_stats: 0 2024-03-26T08:22:41.569 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.ttl: 2592000 2024-03-26T08:22:41.569 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-26T08:22:41.569 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-26T08:22:41.569 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-26T08:22:41.569 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.enable_blob_files: false 2024-03-26T08:22:41.569 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.min_blob_size: 0 2024-03-26T08:22:41.569 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.blob_file_size: 268435456 2024-03-26T08:22:41.569 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-26T08:22:41.569 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-26T08:22:41.569 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-26T08:22:41.570 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-26T08:22:41.570 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-26T08:22:41.570 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.blob_file_starting_level: 0 2024-03-26T08:22:41.570 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-26T08:22:41.570 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-26T08:22:41.570 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-26T08:22:41.570 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 4a1bd2bd-224e-4942-a04f-bfb7ba6486db 2024-03-26T08:22:41.570 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711441361297347, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-26T08:22:41.570 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-26T08:22:41.570 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711441361298814, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711441361, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "4a1bd2bd-224e-4942-a04f-bfb7ba6486db", "db_session_id": "T847L44A28IETQ7AMVMW", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-26T08:22:41.570 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711441361299052, "job": 1, "event": "recovery_finished"} 2024-03-26T08:22:41.570 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-26T08:22:41.570 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-26T08:22:41.570 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5613c96bc000 2024-03-26T08:22:41.571 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: DB pointer 0x5613c96a6000 2024-03-26T08:22:41.571 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-26T08:22:41.571 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-26T08:22:41.571 INFO:journalctl@ceph.mon.c.smithi183.stdout: ** DB Stats ** 2024-03-26T08:22:41.571 INFO:journalctl@ceph.mon.c.smithi183.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T08:22:41.571 INFO:journalctl@ceph.mon.c.smithi183.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-26T08:22:41.571 INFO:journalctl@ceph.mon.c.smithi183.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T08:22:41.571 INFO:journalctl@ceph.mon.c.smithi183.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T08:22:41.571 INFO:journalctl@ceph.mon.c.smithi183.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-26T08:22:41.571 INFO:journalctl@ceph.mon.c.smithi183.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T08:22:41.571 INFO:journalctl@ceph.mon.c.smithi183.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T08:22:41.571 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-03-26T08:22:41.571 INFO:journalctl@ceph.mon.c.smithi183.stdout: ** Compaction Stats [default] ** 2024-03-26T08:22:41.571 INFO:journalctl@ceph.mon.c.smithi183.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T08:22:41.571 INFO:journalctl@ceph.mon.c.smithi183.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-26T08:22:41.571 INFO:journalctl@ceph.mon.c.smithi183.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T08:22:41.572 INFO:journalctl@ceph.mon.c.smithi183.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T08:22:41.572 INFO:journalctl@ceph.mon.c.smithi183.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T08:22:41.572 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-03-26T08:22:41.572 INFO:journalctl@ceph.mon.c.smithi183.stdout: ** Compaction Stats [default] ** 2024-03-26T08:22:41.572 INFO:journalctl@ceph.mon.c.smithi183.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T08:22:41.572 INFO:journalctl@ceph.mon.c.smithi183.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-26T08:22:41.572 INFO:journalctl@ceph.mon.c.smithi183.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T08:22:41.572 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-03-26T08:22:41.572 INFO:journalctl@ceph.mon.c.smithi183.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-26T08:22:41.572 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-03-26T08:22:41.572 INFO:journalctl@ceph.mon.c.smithi183.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T08:22:41.572 INFO:journalctl@ceph.mon.c.smithi183.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-26T08:22:41.572 INFO:journalctl@ceph.mon.c.smithi183.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-26T08:22:41.572 INFO:journalctl@ceph.mon.c.smithi183.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-26T08:22:41.573 INFO:journalctl@ceph.mon.c.smithi183.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-26T08:22:41.573 INFO:journalctl@ceph.mon.c.smithi183.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-26T08:22:41.573 INFO:journalctl@ceph.mon.c.smithi183.stdout: Cumulative compaction: 0.00 GB write, 0.22 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T08:22:41.573 INFO:journalctl@ceph.mon.c.smithi183.stdout: Interval compaction: 0.00 GB write, 0.22 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T08:22:41.573 INFO:journalctl@ceph.mon.c.smithi183.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-26T08:22:41.573 INFO:journalctl@ceph.mon.c.smithi183.stdout: Block cache BinnedLRUCache@0x5613c95c9090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-03-26T08:22:41.573 INFO:journalctl@ceph.mon.c.smithi183.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-26T08:22:41.573 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-03-26T08:22:41.573 INFO:journalctl@ceph.mon.c.smithi183.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-26T08:22:41.573 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-26T08:22:41.573 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: using public_addr v2:172.21.15.183:0/0 -> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] 2024-03-26T08:22:41.573 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: starting mon.c rank -1 at public addrs [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] at bind addrs [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 88759420-eb49-11ee-95d0-87774f69a715 2024-03-26T08:22:41.573 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: mon.c@-1(???) e0 preinit fsid 88759420-eb49-11ee-95d0-87774f69a715 2024-03-26T08:22:41.573 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: mon.c@-1(synchronizing).mds e1 new map 2024-03-26T08:22:41.573 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-26T08:22:41.574 INFO:journalctl@ceph.mon.c.smithi183.stdout: e1 2024-03-26T08:22:41.574 INFO:journalctl@ceph.mon.c.smithi183.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-26T08:22:41.574 INFO:journalctl@ceph.mon.c.smithi183.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-26T08:22:41.574 INFO:journalctl@ceph.mon.c.smithi183.stdout: legacy client fscid: -1 2024-03-26T08:22:41.574 INFO:journalctl@ceph.mon.c.smithi183.stdout: 2024-03-26T08:22:41.574 INFO:journalctl@ceph.mon.c.smithi183.stdout: No filesystems configured 2024-03-26T08:22:41.574 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-26T08:22:41.574 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-26T08:22:41.574 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-26T08:22:41.574 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-26T08:22:41.574 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-26T08:22:41.574 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-26T08:22:41.574 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-26T08:22:41.574 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T08:22:41.575 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T08:22:41.575 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T08:22:41.575 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:41.575 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:41.575 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:41.575 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:41.575 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:22:41.575 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: Added host smithi052 2024-03-26T08:22:41.575 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:41.575 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:41.575 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:41.575 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:41.575 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:41.575 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:22:41.575 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:41.576 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:41.576 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:22:41.576 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:41.576 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: Deploying cephadm binary to smithi183 2024-03-26T08:22:41.576 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:41.576 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:41.576 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:41.576 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:41.576 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi052", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:22:41.576 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:22:41.576 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:22:41.576 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: Updating smithi052:/etc/ceph/ceph.conf 2024-03-26T08:22:41.576 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: Updating smithi052:/var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:22:41.576 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:41.576 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:41.577 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: Updating smithi052:/etc/ceph/ceph.client.admin.keyring 2024-03-26T08:22:41.577 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:41.577 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: Added host smithi183 2024-03-26T08:22:41.577 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: Updating smithi052:/var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T08:22:41.577 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:41.577 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:41.577 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:41.577 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:41.577 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:22:41.577 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:41.577 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:41.577 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:41.577 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:22:41.577 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:41.577 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:41.578 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/4062527560' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-26T08:22:41.578 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:41.578 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/4062527560' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-26T08:22:41.578 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T08:22:41.578 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:41.578 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:41.578 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:41.578 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:41.578 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:41.579 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:22:41.579 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:22:41.579 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:22:41.579 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: Updating smithi183:/etc/ceph/ceph.conf 2024-03-26T08:22:41.579 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:41.579 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:41.579 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: Updating smithi183:/var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:22:41.579 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi052:172.21.15.52=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:22:41.579 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: Saving service mon spec with placement smithi043:172.21.15.43=a;smithi052:172.21.15.52=b;smithi183:172.21.15.183=c;count:3 2024-03-26T08:22:41.579 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-03-26T08:22:41.579 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: Updating smithi183:/var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T08:22:41.579 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:41.579 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:41.579 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:41.579 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:22:41.580 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:22:41.580 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:41.580 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: Deploying daemon mon.c on smithi183 2024-03-26T08:22:41.580 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: from='client.? 172.21.15.183:0/2767279802' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T08:22:41.580 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:41.580 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:41.580 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 ceph-mon[35818]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-26T08:22:41.580 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 bash[35790]: fc6a11acaf9f95d107e7bf0c94c13f337e6d46e9944411d7415380b9a318575a 2024-03-26T08:22:41.580 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:41 smithi183 systemd[1]: Started Ceph mon.c for 88759420-eb49-11ee-95d0-87774f69a715. 2024-03-26T08:22:41.619 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:41.625+0000 7fc6b3c00700 1 -- 172.21.15.183:0/1633033327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc6ac0637c0 con 0x7fc6ac100a00 2024-03-26T08:22:46.375 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:46.379+0000 7fc6a9600700 1 -- 172.21.15.183:0/1633033327 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7fc6a0030090 con 0x7fc6ac100a00 2024-03-26T08:22:46.375 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-26T08:22:46.375 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":2,"fsid":"88759420-eb49-11ee-95d0-87774f69a715","modified":"2024-03-26T08:22:41.351386Z","created":"2024-03-26T08:20:23.553863Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-26T08:22:46.379 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:46.384+0000 7fc6b3c00700 1 -- 172.21.15.183:0/1633033327 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fc694039020 msgr2=0x7fc69403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:46.379 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:46.384+0000 7fc6b3c00700 1 --2- 172.21.15.183:0/1633033327 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fc694039020 0x7fc69403b4e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc69c006fd0 tx=0x7fc69c006e40 comp rx=0 tx=0).stop 2024-03-26T08:22:46.380 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:46.384+0000 7fc6b3c00700 1 -- 172.21.15.183:0/1633033327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6ac100a00 msgr2=0x7fc6ac19f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:46.380 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:46.384+0000 7fc6b3c00700 1 --2- 172.21.15.183:0/1633033327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6ac100a00 0x7fc6ac19f650 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0006010 tx=0x7fc6a0005780 comp rx=0 tx=0).stop 2024-03-26T08:22:46.380 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:46.384+0000 7fc6b3c00700 1 -- 172.21.15.183:0/1633033327 shutdown_connections 2024-03-26T08:22:46.380 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:46.384+0000 7fc6b3c00700 1 --2- 172.21.15.183:0/1633033327 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fc694039020 0x7fc69403b4e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:46.380 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:46.384+0000 7fc6b3c00700 1 --2- 172.21.15.183:0/1633033327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6ac100a00 0x7fc6ac19f650 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:46.380 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:46.384+0000 7fc6b3c00700 1 -- 172.21.15.183:0/1633033327 >> 172.21.15.183:0/1633033327 conn(0x7fc6ac0fc640 msgr2=0x7fc6ac0fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:22:46.380 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:46.384+0000 7fc6b3c00700 1 -- 172.21.15.183:0/1633033327 shutdown_connections 2024-03-26T08:22:46.380 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:46.384+0000 7fc6b3c00700 1 -- 172.21.15.183:0/1633033327 wait complete. 2024-03-26T08:22:46.381 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 2 2024-03-26T08:22:46.769 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:46 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:22:46.769 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:46 smithi183 ceph-mon[35818]: mon.a calling monitor election 2024-03-26T08:22:46.769 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:46 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:22:46.769 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:46 smithi183 ceph-mon[35818]: from='client.? 172.21.15.183:0/1633033327' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T08:22:46.769 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:46 smithi183 ceph-mon[35818]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:46.769 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:46 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:22:46.769 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:46 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:22:46.769 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:46 smithi183 ceph-mon[35818]: mon.c calling monitor election 2024-03-26T08:22:46.769 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:46 smithi183 ceph-mon[35818]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:46.769 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:46 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:22:46.769 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:46 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:22:46.769 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:46 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:22:46.770 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:46 smithi183 ceph-mon[35818]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-26T08:22:46.770 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:46 smithi183 ceph-mon[35818]: monmap e2: 2 mons at {a=[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0],c=[v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T08:22:46.770 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:46 smithi183 ceph-mon[35818]: fsmap 2024-03-26T08:22:46.770 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:46 smithi183 ceph-mon[35818]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T08:22:46.770 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:46 smithi183 ceph-mon[35818]: mgrmap e13: x(active, since 70s) 2024-03-26T08:22:46.770 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:46 smithi183 ceph-mon[35818]: overall HEALTH_OK 2024-03-26T08:22:46.770 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:46 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:46.770 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:46 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:46.770 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:46 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:46.770 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:46 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:22:46.770 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:46 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:22:46.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:46 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:22:46.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:46 smithi043 ceph-mon[31929]: mon.a calling monitor election 2024-03-26T08:22:46.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:46 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:22:46.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:46 smithi043 ceph-mon[31929]: from='client.? 172.21.15.183:0/1633033327' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T08:22:46.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:46 smithi043 ceph-mon[31929]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:46.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:46 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:22:46.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:46 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:22:46.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:46 smithi043 ceph-mon[31929]: mon.c calling monitor election 2024-03-26T08:22:46.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:46 smithi043 ceph-mon[31929]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:46.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:46 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:22:46.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:46 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:22:46.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:46 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:22:46.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:46 smithi043 ceph-mon[31929]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-26T08:22:46.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:46 smithi043 ceph-mon[31929]: monmap e2: 2 mons at {a=[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0],c=[v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T08:22:46.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:46 smithi043 ceph-mon[31929]: fsmap 2024-03-26T08:22:46.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:46 smithi043 ceph-mon[31929]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T08:22:46.906 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:46 smithi043 ceph-mon[31929]: mgrmap e13: x(active, since 70s) 2024-03-26T08:22:46.906 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:46 smithi043 ceph-mon[31929]: overall HEALTH_OK 2024-03-26T08:22:46.906 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:46 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:46.906 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:46 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:46.906 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:46 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:46.906 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:46 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:22:46.906 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:46 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:22:47.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:47 smithi183 ceph-mon[35818]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:47.801 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:47 smithi183 ceph-mon[35818]: Deploying daemon mon.b on smithi052 2024-03-26T08:22:47.801 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:47 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:22:47.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:47 smithi043 ceph-mon[31929]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:47.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:47 smithi043 ceph-mon[31929]: Deploying daemon mon.b on smithi052 2024-03-26T08:22:47.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:47 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:22:48.084 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-26T08:22:48.084 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-26T08:22:48.518 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:22:49.698 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:49 smithi183 ceph-mon[35818]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:49.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:49 smithi043 ceph-mon[31929]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:50.486 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.489+0000 7fd4ae200700 1 -- 172.21.15.183:0/3312961954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4a81040e0 msgr2=0x7fd4a81044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:50.487 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.489+0000 7fd4ae200700 1 --2- 172.21.15.183:0/3312961954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4a81040e0 0x7fd4a81044c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd498009910 tx=0x7fd498009c20 comp rx=0 tx=0).stop 2024-03-26T08:22:50.487 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.490+0000 7fd4ae200700 1 -- 172.21.15.183:0/3312961954 shutdown_connections 2024-03-26T08:22:50.487 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.490+0000 7fd4ae200700 1 --2- 172.21.15.183:0/3312961954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4a81040e0 0x7fd4a81044c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:50.487 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.490+0000 7fd4ae200700 1 -- 172.21.15.183:0/3312961954 >> 172.21.15.183:0/3312961954 conn(0x7fd4a80ff990 msgr2=0x7fd4a8101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:22:50.487 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.490+0000 7fd4ae200700 1 -- 172.21.15.183:0/3312961954 shutdown_connections 2024-03-26T08:22:50.487 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.490+0000 7fd4ae200700 1 -- 172.21.15.183:0/3312961954 wait complete. 2024-03-26T08:22:50.487 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.491+0000 7fd4ae200700 1 Processor -- start 2024-03-26T08:22:50.487 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.492+0000 7fd4ae200700 1 -- start start 2024-03-26T08:22:50.488 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.492+0000 7fd4ae200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4a81040e0 0x7fd4a819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:22:50.488 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.492+0000 7fd4ae200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4a819fc00 0x7fd4a8199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:22:50.488 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.492+0000 7fd4ae200700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a810e360 con 0x7fd4a81040e0 2024-03-26T08:22:50.488 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.492+0000 7fd4ae200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a810e1e0 con 0x7fd4a819fc00 2024-03-26T08:22:50.488 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.492+0000 7fd49ea00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4a819fc00 0x7fd4a8199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:22:50.488 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.493+0000 7fd49ea00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4a819fc00 0x7fd4a8199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:40680/0 (socket says 172.21.15.183:40680) 2024-03-26T08:22:50.488 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.493+0000 7fd49ea00700 1 -- 172.21.15.183:0/1298783043 learned_addr learned my addr 172.21.15.183:0/1298783043 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-03-26T08:22:50.488 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.493+0000 7fd4a7400700 1 --2- 172.21.15.183:0/1298783043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4a81040e0 0x7fd4a819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:22:50.488 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.493+0000 7fd49ea00700 1 -- 172.21.15.183:0/1298783043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4a819fc00 msgr2=0x7fd4a8199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-26T08:22:50.488 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.493+0000 7fd49ea00700 1 -- 172.21.15.183:0/1298783043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4a819fc00 msgr2=0x7fd4a8199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-26T08:22:50.489 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.493+0000 7fd49ea00700 1 --2- 172.21.15.183:0/1298783043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4a819fc00 0x7fd4a8199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-26T08:22:50.489 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.493+0000 7fd49ea00700 1 --2- 172.21.15.183:0/1298783043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4a819fc00 0x7fd4a8199840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T08:22:50.489 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.493+0000 7fd4a7400700 1 -- 172.21.15.183:0/1298783043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4a819fc00 msgr2=0x7fd4a8199840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T08:22:50.490 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.493+0000 7fd4a7400700 1 --2- 172.21.15.183:0/1298783043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4a819fc00 0x7fd4a8199840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:50.490 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.493+0000 7fd4a7400700 1 -- 172.21.15.183:0/1298783043 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a81a0240 con 0x7fd4a81040e0 2024-03-26T08:22:50.490 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.494+0000 7fd4a7400700 1 --2- 172.21.15.183:0/1298783043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4a81040e0 0x7fd4a819f6c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd498014000 tx=0x7fd498005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:22:50.490 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.494+0000 7fd4a4c00700 1 -- 172.21.15.183:0/1298783043 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fd498005dc0 con 0x7fd4a81040e0 2024-03-26T08:22:50.490 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.494+0000 7fd4ae200700 1 -- 172.21.15.183:0/1298783043 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a8199de0 con 0x7fd4a81040e0 2024-03-26T08:22:50.490 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.494+0000 7fd4ae200700 1 -- 172.21.15.183:0/1298783043 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a819a340 con 0x7fd4a81040e0 2024-03-26T08:22:50.490 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.494+0000 7fd4a4c00700 1 -- 172.21.15.183:0/1298783043 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd498004710 con 0x7fd4a81040e0 2024-03-26T08:22:50.490 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.494+0000 7fd4a4c00700 1 -- 172.21.15.183:0/1298783043 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fd49800bc60 con 0x7fd4a81040e0 2024-03-26T08:22:50.490 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.495+0000 7fd4ae200700 1 -- 172.21.15.183:0/1298783043 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd484005330 con 0x7fd4a81040e0 2024-03-26T08:22:50.491 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.496+0000 7fd4a4c00700 1 -- 172.21.15.183:0/1298783043 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fd498029070 con 0x7fd4a81040e0 2024-03-26T08:22:50.491 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.496+0000 7fd4a4c00700 1 --2- 172.21.15.183:0/1298783043 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fd48c038db0 0x7fd48c03b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:22:50.491 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.496+0000 7fd4a4c00700 1 -- 172.21.15.183:0/1298783043 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd49801e070 con 0x7fd4a81040e0 2024-03-26T08:22:50.492 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.497+0000 7fd49ea00700 1 --2- 172.21.15.183:0/1298783043 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fd48c038db0 0x7fd48c03b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:22:50.493 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.498+0000 7fd49ea00700 1 --2- 172.21.15.183:0/1298783043 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fd48c038db0 0x7fd48c03b270 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd490009400 tx=0x7fd490007040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:22:50.508 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.514+0000 7fd4a4c00700 1 -- 172.21.15.183:0/1298783043 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd498013070 con 0x7fd4a81040e0 2024-03-26T08:22:50.823 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.828+0000 7fd4ae200700 1 -- 172.21.15.183:0/1298783043 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd484005620 con 0x7fd4a81040e0 2024-03-26T08:22:50.824 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.829+0000 7fd4a4c00700 1 -- 172.21.15.183:0/1298783043 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7fd49800ec70 con 0x7fd4a81040e0 2024-03-26T08:22:50.824 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-26T08:22:50.824 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":2,"fsid":"88759420-eb49-11ee-95d0-87774f69a715","modified":"2024-03-26T08:22:41.351386Z","created":"2024-03-26T08:20:23.553863Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-26T08:22:50.827 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.832+0000 7fd4ae200700 1 -- 172.21.15.183:0/1298783043 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fd48c038db0 msgr2=0x7fd48c03b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:50.827 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.832+0000 7fd4ae200700 1 --2- 172.21.15.183:0/1298783043 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fd48c038db0 0x7fd48c03b270 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd490009400 tx=0x7fd490007040 comp rx=0 tx=0).stop 2024-03-26T08:22:50.827 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.832+0000 7fd4ae200700 1 -- 172.21.15.183:0/1298783043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4a81040e0 msgr2=0x7fd4a819f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:50.828 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.832+0000 7fd4ae200700 1 --2- 172.21.15.183:0/1298783043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4a81040e0 0x7fd4a819f6c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd498014000 tx=0x7fd498005030 comp rx=0 tx=0).stop 2024-03-26T08:22:50.828 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.832+0000 7fd4ae200700 1 -- 172.21.15.183:0/1298783043 shutdown_connections 2024-03-26T08:22:50.828 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.832+0000 7fd4ae200700 1 --2- 172.21.15.183:0/1298783043 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fd48c038db0 0x7fd48c03b270 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:50.828 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.832+0000 7fd4ae200700 1 --2- 172.21.15.183:0/1298783043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4a81040e0 0x7fd4a819f6c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:50.828 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.832+0000 7fd4ae200700 1 --2- 172.21.15.183:0/1298783043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4a819fc00 0x7fd4a8199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:50.828 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.832+0000 7fd4ae200700 1 -- 172.21.15.183:0/1298783043 >> 172.21.15.183:0/1298783043 conn(0x7fd4a80ff990 msgr2=0x7fd4a8105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:22:50.828 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.832+0000 7fd4ae200700 1 -- 172.21.15.183:0/1298783043 shutdown_connections 2024-03-26T08:22:50.828 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:50.832+0000 7fd4ae200700 1 -- 172.21.15.183:0/1298783043 wait complete. 2024-03-26T08:22:50.828 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 2 2024-03-26T08:22:51.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:51 smithi183 ceph-mon[35818]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:51.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:51 smithi183 ceph-mon[35818]: from='client.? 172.21.15.183:0/1298783043' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T08:22:51.883 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 podman[35052]: 2024-03-26 08:22:51.887371936 +0000 UTC m=+0.387543225 container init 02f8c34e593dcc8d9af24481d097371b060bc446b1bf1671b837eb1a0e460b66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-b, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T08:22:51.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:51 smithi043 ceph-mon[31929]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:51.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:51 smithi043 ceph-mon[31929]: from='client.? 172.21.15.183:0/1298783043' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T08:22:52.326 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 podman[35052]: 2024-03-26 08:22:51.897653156 +0000 UTC m=+0.397824481 container start 02f8c34e593dcc8d9af24481d097371b060bc446b1bf1671b837eb1a0e460b66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-26T08:22:52.326 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: set uid:gid to 167:167 (ceph:ceph) 2024-03-26T08:22:52.326 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev), process ceph-mon, pid 2 2024-03-26T08:22:52.326 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: pidfile_write: ignore empty --pid-file 2024-03-26T08:22:52.326 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: load: jerasure load: lrc 2024-03-26T08:22:52.326 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: RocksDB version: 7.9.2 2024-03-26T08:22:52.327 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Git sha 0 2024-03-26T08:22:52.327 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Compile date 2024-03-22 04:58:55 2024-03-26T08:22:52.327 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: DB SUMMARY 2024-03-26T08:22:52.327 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: DB Session ID: BTH7DTOMP9N48YRICJMU 2024-03-26T08:22:52.327 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: CURRENT file: CURRENT 2024-03-26T08:22:52.327 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: IDENTITY file: IDENTITY 2024-03-26T08:22:52.327 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-26T08:22:52.327 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-03-26T08:22:52.327 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-03-26T08:22:52.327 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.error_if_exists: 0 2024-03-26T08:22:52.327 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.create_if_missing: 0 2024-03-26T08:22:52.327 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.paranoid_checks: 1 2024-03-26T08:22:52.327 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-26T08:22:52.327 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-26T08:22:52.328 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-26T08:22:52.328 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.env: 0x55ef1e37ab20 2024-03-26T08:22:52.328 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.fs: PosixFileSystem 2024-03-26T08:22:52.328 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.info_log: 0x55ef20b19960 2024-03-26T08:22:52.328 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.max_file_opening_threads: 16 2024-03-26T08:22:52.328 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.statistics: (nil) 2024-03-26T08:22:52.328 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.use_fsync: 0 2024-03-26T08:22:52.328 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.max_log_file_size: 0 2024-03-26T08:22:52.328 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-26T08:22:52.328 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-26T08:22:52.328 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.keep_log_file_num: 1000 2024-03-26T08:22:52.328 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.recycle_log_file_num: 0 2024-03-26T08:22:52.328 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.allow_fallocate: 1 2024-03-26T08:22:52.329 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.allow_mmap_reads: 0 2024-03-26T08:22:52.329 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.allow_mmap_writes: 0 2024-03-26T08:22:52.329 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.use_direct_reads: 0 2024-03-26T08:22:52.329 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-26T08:22:52.329 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.create_missing_column_families: 0 2024-03-26T08:22:52.329 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.db_log_dir: 2024-03-26T08:22:52.329 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.wal_dir: 2024-03-26T08:22:52.329 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-26T08:22:52.329 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-26T08:22:52.330 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-26T08:22:52.330 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-26T08:22:52.330 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-26T08:22:52.330 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-26T08:22:52.330 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.advise_random_on_open: 1 2024-03-26T08:22:52.330 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.db_write_buffer_size: 0 2024-03-26T08:22:52.330 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.write_buffer_manager: 0x55ef20aec1e0 2024-03-26T08:22:52.330 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-26T08:22:52.330 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-26T08:22:52.330 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-26T08:22:52.330 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.rate_limiter: (nil) 2024-03-26T08:22:52.330 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-26T08:22:52.330 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.wal_recovery_mode: 2 2024-03-26T08:22:52.331 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.enable_thread_tracking: 0 2024-03-26T08:22:52.331 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.enable_pipelined_write: 0 2024-03-26T08:22:52.331 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.unordered_write: 0 2024-03-26T08:22:52.331 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-26T08:22:52.331 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-26T08:22:52.331 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-26T08:22:52.331 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-26T08:22:52.331 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.row_cache: None 2024-03-26T08:22:52.331 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.wal_filter: None 2024-03-26T08:22:52.331 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-26T08:22:52.331 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.allow_ingest_behind: 0 2024-03-26T08:22:52.331 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.two_write_queues: 0 2024-03-26T08:22:52.331 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.manual_wal_flush: 0 2024-03-26T08:22:52.331 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.wal_compression: 0 2024-03-26T08:22:52.332 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.atomic_flush: 0 2024-03-26T08:22:52.332 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-26T08:22:52.332 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-26T08:22:52.332 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-26T08:22:52.332 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.log_readahead_size: 0 2024-03-26T08:22:52.332 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-26T08:22:52.332 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.best_efforts_recovery: 0 2024-03-26T08:22:52.332 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-26T08:22:52.332 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-26T08:22:52.332 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.allow_data_in_errors: 0 2024-03-26T08:22:52.332 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.db_host_id: __hostname__ 2024-03-26T08:22:52.332 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-26T08:22:52.332 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.max_background_jobs: 2 2024-03-26T08:22:52.332 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.max_background_compactions: -1 2024-03-26T08:22:52.333 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.max_subcompactions: 1 2024-03-26T08:22:52.333 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-26T08:22:52.333 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-26T08:22:52.333 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-26T08:22:52.333 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.max_total_wal_size: 0 2024-03-26T08:22:52.333 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-26T08:22:52.333 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-26T08:22:52.333 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-26T08:22:52.333 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-26T08:22:52.333 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.max_open_files: -1 2024-03-26T08:22:52.333 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.bytes_per_sync: 0 2024-03-26T08:22:52.333 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-26T08:22:52.333 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-26T08:22:52.333 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.compaction_readahead_size: 0 2024-03-26T08:22:52.333 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.max_background_flushes: -1 2024-03-26T08:22:52.334 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Compression algorithms supported: 2024-03-26T08:22:52.334 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-26T08:22:52.334 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: kZSTD supported: 0 2024-03-26T08:22:52.334 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: kXpressCompression supported: 0 2024-03-26T08:22:52.334 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: kLZ4HCCompression supported: 1 2024-03-26T08:22:52.334 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: kZlibCompression supported: 1 2024-03-26T08:22:52.334 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: kSnappyCompression supported: 1 2024-03-26T08:22:52.334 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: kLZ4Compression supported: 1 2024-03-26T08:22:52.334 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: kBZip2Compression supported: 0 2024-03-26T08:22:52.334 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-26T08:22:52.334 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-26T08:22:52.334 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-03-26T08:22:52.334 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-26T08:22:52.334 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-26T08:22:52.334 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.merge_operator: 2024-03-26T08:22:52.336 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.compaction_filter: None 2024-03-26T08:22:52.336 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.compaction_filter_factory: None 2024-03-26T08:22:52.336 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.sst_partitioner_factory: None 2024-03-26T08:22:52.336 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-26T08:22:52.336 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-26T08:22:52.336 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ef20b19a60) 2024-03-26T08:22:52.336 INFO:journalctl@ceph.mon.b.smithi052.stdout: cache_index_and_filter_blocks: 1 2024-03-26T08:22:52.336 INFO:journalctl@ceph.mon.b.smithi052.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-26T08:22:52.336 INFO:journalctl@ceph.mon.b.smithi052.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-26T08:22:52.336 INFO:journalctl@ceph.mon.b.smithi052.stdout: pin_top_level_index_and_filter: 1 2024-03-26T08:22:52.336 INFO:journalctl@ceph.mon.b.smithi052.stdout: index_type: 0 2024-03-26T08:22:52.337 INFO:journalctl@ceph.mon.b.smithi052.stdout: data_block_index_type: 0 2024-03-26T08:22:52.337 INFO:journalctl@ceph.mon.b.smithi052.stdout: index_shortening: 1 2024-03-26T08:22:52.337 INFO:journalctl@ceph.mon.b.smithi052.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-26T08:22:52.337 INFO:journalctl@ceph.mon.b.smithi052.stdout: checksum: 4 2024-03-26T08:22:52.337 INFO:journalctl@ceph.mon.b.smithi052.stdout: no_block_cache: 0 2024-03-26T08:22:52.337 INFO:journalctl@ceph.mon.b.smithi052.stdout: block_cache: 0x55ef20b1f090 2024-03-26T08:22:52.337 INFO:journalctl@ceph.mon.b.smithi052.stdout: block_cache_name: BinnedLRUCache 2024-03-26T08:22:52.337 INFO:journalctl@ceph.mon.b.smithi052.stdout: block_cache_options: 2024-03-26T08:22:52.337 INFO:journalctl@ceph.mon.b.smithi052.stdout: capacity : 536870912 2024-03-26T08:22:52.337 INFO:journalctl@ceph.mon.b.smithi052.stdout: num_shard_bits : 4 2024-03-26T08:22:52.337 INFO:journalctl@ceph.mon.b.smithi052.stdout: strict_capacity_limit : 0 2024-03-26T08:22:52.337 INFO:journalctl@ceph.mon.b.smithi052.stdout: high_pri_pool_ratio: 0.000 2024-03-26T08:22:52.337 INFO:journalctl@ceph.mon.b.smithi052.stdout: block_cache_compressed: (nil) 2024-03-26T08:22:52.337 INFO:journalctl@ceph.mon.b.smithi052.stdout: persistent_cache: (nil) 2024-03-26T08:22:52.337 INFO:journalctl@ceph.mon.b.smithi052.stdout: block_size: 4096 2024-03-26T08:22:52.338 INFO:journalctl@ceph.mon.b.smithi052.stdout: block_size_deviation: 10 2024-03-26T08:22:52.338 INFO:journalctl@ceph.mon.b.smithi052.stdout: block_restart_interval: 16 2024-03-26T08:22:52.338 INFO:journalctl@ceph.mon.b.smithi052.stdout: index_block_restart_interval: 1 2024-03-26T08:22:52.338 INFO:journalctl@ceph.mon.b.smithi052.stdout: metadata_block_size: 4096 2024-03-26T08:22:52.338 INFO:journalctl@ceph.mon.b.smithi052.stdout: partition_filters: 0 2024-03-26T08:22:52.338 INFO:journalctl@ceph.mon.b.smithi052.stdout: use_delta_encoding: 1 2024-03-26T08:22:52.338 INFO:journalctl@ceph.mon.b.smithi052.stdout: filter_policy: bloomfilter 2024-03-26T08:22:52.338 INFO:journalctl@ceph.mon.b.smithi052.stdout: whole_key_filtering: 1 2024-03-26T08:22:52.338 INFO:journalctl@ceph.mon.b.smithi052.stdout: verify_compression: 0 2024-03-26T08:22:52.338 INFO:journalctl@ceph.mon.b.smithi052.stdout: read_amp_bytes_per_bit: 0 2024-03-26T08:22:52.338 INFO:journalctl@ceph.mon.b.smithi052.stdout: format_version: 5 2024-03-26T08:22:52.338 INFO:journalctl@ceph.mon.b.smithi052.stdout: enable_index_compression: 1 2024-03-26T08:22:52.338 INFO:journalctl@ceph.mon.b.smithi052.stdout: block_align: 0 2024-03-26T08:22:52.338 INFO:journalctl@ceph.mon.b.smithi052.stdout: max_auto_readahead_size: 262144 2024-03-26T08:22:52.338 INFO:journalctl@ceph.mon.b.smithi052.stdout: prepopulate_block_cache: 0 2024-03-26T08:22:52.339 INFO:journalctl@ceph.mon.b.smithi052.stdout: initial_auto_readahead_size: 8192 2024-03-26T08:22:52.339 INFO:journalctl@ceph.mon.b.smithi052.stdout: num_file_reads_for_auto_readahead: 2 2024-03-26T08:22:52.339 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.write_buffer_size: 33554432 2024-03-26T08:22:52.339 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.max_write_buffer_number: 2 2024-03-26T08:22:52.339 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.compression: NoCompression 2024-03-26T08:22:52.339 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.bottommost_compression: Disabled 2024-03-26T08:22:52.339 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.prefix_extractor: nullptr 2024-03-26T08:22:52.339 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-26T08:22:52.339 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.num_levels: 7 2024-03-26T08:22:52.339 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-26T08:22:52.339 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-26T08:22:52.339 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-26T08:22:52.339 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-26T08:22:52.339 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-26T08:22:52.339 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-26T08:22:52.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-26T08:22:52.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-26T08:22:52.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-26T08:22:52.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-26T08:22:52.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-26T08:22:52.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-26T08:22:52.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-26T08:22:52.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.compression_opts.level: 32767 2024-03-26T08:22:52.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.compression_opts.strategy: 0 2024-03-26T08:22:52.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-26T08:22:52.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-26T08:22:52.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-26T08:22:52.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-26T08:22:52.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.compression_opts.enabled: false 2024-03-26T08:22:52.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-26T08:22:52.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-26T08:22:52.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-26T08:22:52.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-26T08:22:52.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.target_file_size_base: 67108864 2024-03-26T08:22:52.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-26T08:22:52.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-26T08:22:52.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-26T08:22:52.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-26T08:22:52.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-26T08:22:52.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-26T08:22:52.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-26T08:22:52.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-26T08:22:52.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-26T08:22:52.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-26T08:22:52.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-26T08:22:52.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-26T08:22:52.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-26T08:22:52.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-26T08:22:52.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.arena_block_size: 1048576 2024-03-26T08:22:52.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-26T08:22:52.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-26T08:22:52.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.disable_auto_compactions: 0 2024-03-26T08:22:52.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-26T08:22:52.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-26T08:22:52.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-26T08:22:52.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-26T08:22:52.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-26T08:22:52.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-26T08:22:52.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-26T08:22:52.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-26T08:22:52.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-26T08:22:52.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-26T08:22:52.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.table_properties_collectors: 2024-03-26T08:22:52.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.inplace_update_support: 0 2024-03-26T08:22:52.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-26T08:22:52.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-26T08:22:52.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-26T08:22:52.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-26T08:22:52.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.bloom_locality: 0 2024-03-26T08:22:52.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.max_successive_merges: 0 2024-03-26T08:22:52.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-26T08:22:52.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.paranoid_file_checks: 0 2024-03-26T08:22:52.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.force_consistency_checks: 1 2024-03-26T08:22:52.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.report_bg_io_stats: 0 2024-03-26T08:22:52.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.ttl: 2592000 2024-03-26T08:22:52.344 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-26T08:22:52.344 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-26T08:22:52.344 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-26T08:22:52.344 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.enable_blob_files: false 2024-03-26T08:22:52.344 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.min_blob_size: 0 2024-03-26T08:22:52.344 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.blob_file_size: 268435456 2024-03-26T08:22:52.344 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-26T08:22:52.344 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-26T08:22:52.344 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-26T08:22:52.344 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-26T08:22:52.344 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-26T08:22:52.344 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.blob_file_starting_level: 0 2024-03-26T08:22:52.344 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-26T08:22:52.344 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-26T08:22:52.345 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-26T08:22:52.345 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 4ed17225-c016-4c9c-9a09-6ecfc848a166 2024-03-26T08:22:52.345 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711441371961470, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-26T08:22:52.345 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-26T08:22:52.345 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711441371962393, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711441371, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "4ed17225-c016-4c9c-9a09-6ecfc848a166", "db_session_id": "BTH7DTOMP9N48YRICJMU", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-26T08:22:52.345 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711441371962666, "job": 1, "event": "recovery_finished"} 2024-03-26T08:22:52.345 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-26T08:22:52.345 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-26T08:22:52.345 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ef20c12000 2024-03-26T08:22:52.345 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: DB pointer 0x55ef20bfc000 2024-03-26T08:22:52.345 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-26T08:22:52.345 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-26T08:22:52.345 INFO:journalctl@ceph.mon.b.smithi052.stdout: ** DB Stats ** 2024-03-26T08:22:52.345 INFO:journalctl@ceph.mon.b.smithi052.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T08:22:52.345 INFO:journalctl@ceph.mon.b.smithi052.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-26T08:22:52.345 INFO:journalctl@ceph.mon.b.smithi052.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T08:22:52.346 INFO:journalctl@ceph.mon.b.smithi052.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T08:22:52.346 INFO:journalctl@ceph.mon.b.smithi052.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-26T08:22:52.346 INFO:journalctl@ceph.mon.b.smithi052.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T08:22:52.346 INFO:journalctl@ceph.mon.b.smithi052.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T08:22:52.346 INFO:journalctl@ceph.mon.b.smithi052.stdout: 2024-03-26T08:22:52.346 INFO:journalctl@ceph.mon.b.smithi052.stdout: ** Compaction Stats [default] ** 2024-03-26T08:22:52.346 INFO:journalctl@ceph.mon.b.smithi052.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T08:22:52.346 INFO:journalctl@ceph.mon.b.smithi052.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-26T08:22:52.346 INFO:journalctl@ceph.mon.b.smithi052.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T08:22:52.346 INFO:journalctl@ceph.mon.b.smithi052.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T08:22:52.346 INFO:journalctl@ceph.mon.b.smithi052.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T08:22:52.346 INFO:journalctl@ceph.mon.b.smithi052.stdout: 2024-03-26T08:22:52.346 INFO:journalctl@ceph.mon.b.smithi052.stdout: ** Compaction Stats [default] ** 2024-03-26T08:22:52.346 INFO:journalctl@ceph.mon.b.smithi052.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T08:22:52.347 INFO:journalctl@ceph.mon.b.smithi052.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-26T08:22:52.347 INFO:journalctl@ceph.mon.b.smithi052.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T08:22:52.347 INFO:journalctl@ceph.mon.b.smithi052.stdout: 2024-03-26T08:22:52.347 INFO:journalctl@ceph.mon.b.smithi052.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-26T08:22:52.347 INFO:journalctl@ceph.mon.b.smithi052.stdout: 2024-03-26T08:22:52.347 INFO:journalctl@ceph.mon.b.smithi052.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T08:22:52.347 INFO:journalctl@ceph.mon.b.smithi052.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-26T08:22:52.347 INFO:journalctl@ceph.mon.b.smithi052.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-26T08:22:52.347 INFO:journalctl@ceph.mon.b.smithi052.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-26T08:22:52.347 INFO:journalctl@ceph.mon.b.smithi052.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-26T08:22:52.347 INFO:journalctl@ceph.mon.b.smithi052.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-26T08:22:52.347 INFO:journalctl@ceph.mon.b.smithi052.stdout: Cumulative compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T08:22:52.347 INFO:journalctl@ceph.mon.b.smithi052.stdout: Interval compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T08:22:52.347 INFO:journalctl@ceph.mon.b.smithi052.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-26T08:22:52.347 INFO:journalctl@ceph.mon.b.smithi052.stdout: Block cache BinnedLRUCache@0x55ef20b1f090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-03-26T08:22:52.348 INFO:journalctl@ceph.mon.b.smithi052.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-26T08:22:52.348 INFO:journalctl@ceph.mon.b.smithi052.stdout: 2024-03-26T08:22:52.348 INFO:journalctl@ceph.mon.b.smithi052.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-26T08:22:52.348 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-03-26T08:22:52.348 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: using public_addr v2:172.21.15.52:0/0 -> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] 2024-03-26T08:22:52.348 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: starting mon.b rank -1 at public addrs [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] at bind addrs [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 88759420-eb49-11ee-95d0-87774f69a715 2024-03-26T08:22:52.348 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 ceph-mon[35071]: mon.b@-1(???) e0 preinit fsid 88759420-eb49-11ee-95d0-87774f69a715 2024-03-26T08:22:52.348 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 bash[35052]: 02f8c34e593dcc8d9af24481d097371b060bc446b1bf1671b837eb1a0e460b66 2024-03-26T08:22:52.348 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:51 smithi052 systemd[1]: Started Ceph mon.b for 88759420-eb49-11ee-95d0-87774f69a715. 2024-03-26T08:22:52.348 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: mon.b@-1(synchronizing).mds e1 new map 2024-03-26T08:22:52.348 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: mon.b@-1(synchronizing).mds e1 print_map 2024-03-26T08:22:52.348 INFO:journalctl@ceph.mon.b.smithi052.stdout: e1 2024-03-26T08:22:52.348 INFO:journalctl@ceph.mon.b.smithi052.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-26T08:22:52.348 INFO:journalctl@ceph.mon.b.smithi052.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-26T08:22:52.348 INFO:journalctl@ceph.mon.b.smithi052.stdout: legacy client fscid: -1 2024-03-26T08:22:52.349 INFO:journalctl@ceph.mon.b.smithi052.stdout: 2024-03-26T08:22:52.349 INFO:journalctl@ceph.mon.b.smithi052.stdout: No filesystems configured 2024-03-26T08:22:52.349 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-26T08:22:52.349 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-26T08:22:52.349 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-26T08:22:52.349 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-26T08:22:52.349 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-26T08:22:52.349 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-26T08:22:52.349 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-26T08:22:52.349 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T08:22:52.349 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T08:22:52.349 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T08:22:52.349 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:52.349 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:52.349 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:52.349 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:52.350 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:22:52.350 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: Added host smithi052 2024-03-26T08:22:52.350 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:52.350 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:52.350 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:52.350 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:52.350 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:52.350 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:22:52.350 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:52.350 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:52.350 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:22:52.350 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:52.350 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: Deploying cephadm binary to smithi183 2024-03-26T08:22:52.350 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:52.351 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:52.351 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:52.351 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:52.351 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi052", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:22:52.351 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:22:52.351 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:22:52.351 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: Updating smithi052:/etc/ceph/ceph.conf 2024-03-26T08:22:52.351 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: Updating smithi052:/var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:22:52.351 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:52.351 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:52.351 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: Updating smithi052:/etc/ceph/ceph.client.admin.keyring 2024-03-26T08:22:52.351 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:52.351 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: Added host smithi183 2024-03-26T08:22:52.351 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: Updating smithi052:/var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T08:22:52.351 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:52.351 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:52.352 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:52.352 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:52.352 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:22:52.352 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:52.352 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:52.352 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:52.352 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:22:52.352 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:52.352 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:52.352 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/4062527560' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-26T08:22:52.352 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:52.353 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/4062527560' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-26T08:22:52.353 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T08:22:52.353 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:52.353 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:52.353 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:52.353 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:52.353 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:52.353 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:22:52.353 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:22:52.353 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:22:52.353 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: Updating smithi183:/etc/ceph/ceph.conf 2024-03-26T08:22:52.353 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:52.353 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:52.354 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: Updating smithi183:/var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:22:52.354 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi052:172.21.15.52=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:22:52.354 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: Saving service mon spec with placement smithi043:172.21.15.43=a;smithi052:172.21.15.52=b;smithi183:172.21.15.183=c;count:3 2024-03-26T08:22:52.354 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-03-26T08:22:52.354 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: Updating smithi183:/var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T08:22:52.354 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:52.354 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:52.354 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:52.354 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:22:52.354 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:22:52.354 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:52.354 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: Deploying daemon mon.c on smithi183 2024-03-26T08:22:52.354 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='client.? 172.21.15.183:0/2767279802' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T08:22:52.354 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:52.354 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:52.355 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:22:52.355 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: mon.a calling monitor election 2024-03-26T08:22:52.355 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:22:52.355 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='client.? 172.21.15.183:0/1633033327' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T08:22:52.355 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:52.355 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:22:52.355 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:22:52.355 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: mon.c calling monitor election 2024-03-26T08:22:52.355 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:52.355 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:22:52.355 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:22:52.355 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:22:52.355 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-26T08:22:52.355 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: monmap e2: 2 mons at {a=[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0],c=[v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T08:22:52.356 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: fsmap 2024-03-26T08:22:52.356 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T08:22:52.356 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: mgrmap e13: x(active, since 70s) 2024-03-26T08:22:52.356 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: overall HEALTH_OK 2024-03-26T08:22:52.356 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:52.356 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:52.356 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:52.356 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:22:52.356 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:22:52.356 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:52.356 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: Deploying daemon mon.b on smithi052 2024-03-26T08:22:52.356 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:22:52.356 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:52.356 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:52.356 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: from='client.? 172.21.15.183:0/1298783043' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T08:22:52.357 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:52 smithi052 ceph-mon[35071]: mon.b@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-03-26T08:22:52.555 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-26T08:22:52.556 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-26T08:22:52.965 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:22:57.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:57 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:22:57.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:57 smithi043 ceph-mon[31929]: mon.a calling monitor election 2024-03-26T08:22:57.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:57 smithi043 ceph-mon[31929]: mon.c calling monitor election 2024-03-26T08:22:57.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:57 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:22:57.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:57 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:22:57.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:57 smithi043 ceph-mon[31929]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:57.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:57 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:22:57.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:57 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:22:57.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:57 smithi043 ceph-mon[31929]: mon.b calling monitor election 2024-03-26T08:22:57.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:57 smithi043 ceph-mon[31929]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:57.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:57 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:22:57.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:57 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:22:57.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:57 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:22:57.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:57 smithi043 ceph-mon[31929]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-26T08:22:57.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:57 smithi043 ceph-mon[31929]: monmap e3: 3 mons at {a=[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0],b=[v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0],c=[v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T08:22:57.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:57 smithi043 ceph-mon[31929]: fsmap 2024-03-26T08:22:57.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:57 smithi043 ceph-mon[31929]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T08:22:57.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:57 smithi043 ceph-mon[31929]: mgrmap e13: x(active, since 80s) 2024-03-26T08:22:57.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:57 smithi043 ceph-mon[31929]: overall HEALTH_OK 2024-03-26T08:22:57.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:57 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:57.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:57 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:57.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:57 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:57.406 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:57 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:57.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:57 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:22:57.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:57 smithi183 ceph-mon[35818]: mon.a calling monitor election 2024-03-26T08:22:57.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:57 smithi183 ceph-mon[35818]: mon.c calling monitor election 2024-03-26T08:22:57.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:57 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:22:57.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:57 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:22:57.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:57 smithi183 ceph-mon[35818]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:57.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:57 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:22:57.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:57 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:22:57.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:57 smithi183 ceph-mon[35818]: mon.b calling monitor election 2024-03-26T08:22:57.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:57 smithi183 ceph-mon[35818]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:57.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:57 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:22:57.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:57 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:22:57.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:57 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:22:57.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:57 smithi183 ceph-mon[35818]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-26T08:22:57.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:57 smithi183 ceph-mon[35818]: monmap e3: 3 mons at {a=[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0],b=[v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0],c=[v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T08:22:57.553 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:57 smithi183 ceph-mon[35818]: fsmap 2024-03-26T08:22:57.553 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:57 smithi183 ceph-mon[35818]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T08:22:57.553 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:57 smithi183 ceph-mon[35818]: mgrmap e13: x(active, since 80s) 2024-03-26T08:22:57.553 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:57 smithi183 ceph-mon[35818]: overall HEALTH_OK 2024-03-26T08:22:57.553 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:57 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:57.553 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:57 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:57.553 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:57 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:57.553 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:57 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:57.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:57 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:22:57.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:57 smithi052 ceph-mon[35071]: mon.a calling monitor election 2024-03-26T08:22:57.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:57 smithi052 ceph-mon[35071]: mon.c calling monitor election 2024-03-26T08:22:57.576 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:57 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:22:57.576 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:57 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:22:57.576 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:57 smithi052 ceph-mon[35071]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:57.576 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:57 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:22:57.576 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:57 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:22:57.576 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:57 smithi052 ceph-mon[35071]: mon.b calling monitor election 2024-03-26T08:22:57.576 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:57 smithi052 ceph-mon[35071]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:57.576 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:57 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:22:57.576 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:57 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:22:57.576 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:57 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:22:57.576 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:57 smithi052 ceph-mon[35071]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-26T08:22:57.576 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:57 smithi052 ceph-mon[35071]: monmap e3: 3 mons at {a=[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0],b=[v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0],c=[v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T08:22:57.576 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:57 smithi052 ceph-mon[35071]: fsmap 2024-03-26T08:22:57.576 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:57 smithi052 ceph-mon[35071]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T08:22:57.576 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:57 smithi052 ceph-mon[35071]: mgrmap e13: x(active, since 80s) 2024-03-26T08:22:57.577 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:57 smithi052 ceph-mon[35071]: overall HEALTH_OK 2024-03-26T08:22:57.577 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:57 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:57.577 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:57 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:57.577 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:57 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:57.577 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:57 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:22:57.924 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.927+0000 7fa0d3c00700 1 -- 172.21.15.183:0/3246266109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0cc101eb0 msgr2=0x7fa0b4005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:57.924 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.927+0000 7fa0d3c00700 1 --2- 172.21.15.183:0/3246266109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0cc101eb0 0x7fa0b4005610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0005fa0 tx=0x7fa0c000ffc0 comp rx=0 tx=0).stop 2024-03-26T08:22:57.924 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.928+0000 7fa0d3c00700 1 -- 172.21.15.183:0/3246266109 shutdown_connections 2024-03-26T08:22:57.924 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.928+0000 7fa0d3c00700 1 --2- 172.21.15.183:0/3246266109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0cc101eb0 0x7fa0b4005610 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:57.924 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.928+0000 7fa0d3c00700 1 -- 172.21.15.183:0/3246266109 >> 172.21.15.183:0/3246266109 conn(0x7fa0cc0fd720 msgr2=0x7fa0cc0ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:22:57.924 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.928+0000 7fa0d3c00700 1 -- 172.21.15.183:0/3246266109 shutdown_connections 2024-03-26T08:22:57.925 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.928+0000 7fa0d3c00700 1 -- 172.21.15.183:0/3246266109 wait complete. 2024-03-26T08:22:57.926 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.931+0000 7fa0d3c00700 1 Processor -- start 2024-03-26T08:22:57.926 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.931+0000 7fa0d3c00700 1 -- start start 2024-03-26T08:22:57.927 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.932+0000 7fa0d3c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0cc101eb0 0x7fa0cc19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:22:57.927 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.932+0000 7fa0d3c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0cc19fb90 0x7fa0cc1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:22:57.927 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.932+0000 7fa0d3c00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0cc199d10 0x7fa0cc19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:22:57.927 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.932+0000 7fa0d3c00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc109b40 con 0x7fa0cc19fb90 2024-03-26T08:22:57.928 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.932+0000 7fa0d3c00700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc1099c0 con 0x7fa0cc199d10 2024-03-26T08:22:57.928 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.932+0000 7fa0d3c00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc109cc0 con 0x7fa0cc101eb0 2024-03-26T08:22:57.928 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.933+0000 7fa0cbe00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0cc19fb90 0x7fa0cc1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:22:57.928 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.933+0000 7fa0cbe00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0cc19fb90 0x7fa0cc1997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.183:48786/0 (socket says 172.21.15.183:48786) 2024-03-26T08:22:57.928 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.933+0000 7fa0cbe00700 1 -- 172.21.15.183:0/4101915636 learned_addr learned my addr 172.21.15.183:0/4101915636 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-03-26T08:22:57.928 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.933+0000 7fa0d1200700 1 --2- 172.21.15.183:0/4101915636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0cc101eb0 0x7fa0cc19f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:22:57.929 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.933+0000 7fa0cbe00700 1 -- 172.21.15.183:0/4101915636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0cc101eb0 msgr2=0x7fa0cc19f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:57.929 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.933+0000 7fa0cbe00700 1 --2- 172.21.15.183:0/4101915636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0cc101eb0 0x7fa0cc19f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:57.929 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.933+0000 7fa0cbe00700 1 -- 172.21.15.183:0/4101915636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0cc199d10 msgr2=0x7fa0cc19a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:57.929 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.933+0000 7fa0cbe00700 1 --2- 172.21.15.183:0/4101915636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0cc199d10 0x7fa0cc19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:57.929 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.933+0000 7fa0cbe00700 1 -- 172.21.15.183:0/4101915636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc1a64b0 con 0x7fa0cc19fb90 2024-03-26T08:22:57.929 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.934+0000 7fa0cbe00700 1 --2- 172.21.15.183:0/4101915636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0cc19fb90 0x7fa0cc1997d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800b780 tx=0x7fa0b800ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:22:57.930 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.934+0000 7fa0c9600700 1 -- 172.21.15.183:0/4101915636 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8004230 con 0x7fa0cc19fb90 2024-03-26T08:22:57.930 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.934+0000 7fa0d3c00700 1 -- 172.21.15.183:0/4101915636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0cc1a6740 con 0x7fa0cc19fb90 2024-03-26T08:22:57.930 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.935+0000 7fa0c9600700 1 -- 172.21.15.183:0/4101915636 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa0b80043d0 con 0x7fa0cc19fb90 2024-03-26T08:22:57.930 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.935+0000 7fa0c9600700 1 -- 172.21.15.183:0/4101915636 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8011af0 con 0x7fa0cc19fb90 2024-03-26T08:22:57.930 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.935+0000 7fa0d3c00700 1 -- 172.21.15.183:0/4101915636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0cc1a6ca0 con 0x7fa0cc19fb90 2024-03-26T08:22:57.931 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.936+0000 7fa0d3c00700 1 -- 172.21.15.183:0/4101915636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0cc103390 con 0x7fa0cc19fb90 2024-03-26T08:22:57.931 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.936+0000 7fa0c9600700 1 -- 172.21.15.183:0/4101915636 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7fa0b80036a0 con 0x7fa0cc19fb90 2024-03-26T08:22:57.932 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.937+0000 7fa0c9600700 1 --2- 172.21.15.183:0/4101915636 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa0ac039450 0x7fa0ac03b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:22:57.932 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.937+0000 7fa0c9600700 1 -- 172.21.15.183:0/4101915636 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa0b804bba0 con 0x7fa0cc19fb90 2024-03-26T08:22:57.932 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.937+0000 7fa0d1200700 1 --2- 172.21.15.183:0/4101915636 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa0ac039450 0x7fa0ac03b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:22:57.934 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.939+0000 7fa0d1200700 1 --2- 172.21.15.183:0/4101915636 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa0ac039450 0x7fa0ac03b910 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0005fd0 tx=0x7fa0c001d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:22:57.948 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:57.953+0000 7fa0c9600700 1 -- 172.21.15.183:0/4101915636 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa0b8014160 con 0x7fa0cc19fb90 2024-03-26T08:22:58.267 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:58.272+0000 7fa0d3c00700 1 -- 172.21.15.183:0/4101915636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa0cc02da30 con 0x7fa0cc19fb90 2024-03-26T08:22:58.268 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:58 smithi183 ceph-mon[35818]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:58.268 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:58 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:22:58.268 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:58 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:22:58.268 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:58.273+0000 7fa0c9600700 1 -- 172.21.15.183:0/4101915636 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1281 (secure 0 0 0) 0x7fa0b80189a0 con 0x7fa0cc19fb90 2024-03-26T08:22:58.269 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-03-26T08:22:58.269 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":3,"fsid":"88759420-eb49-11ee-95d0-87774f69a715","modified":"2024-03-26T08:22:52.012154Z","created":"2024-03-26T08:20:23.553863Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:3300","nonce":0},{"type":"v1","addr":"172.21.15.52:6789","nonce":0}]},"addr":"172.21.15.52:6789/0","public_addr":"172.21.15.52:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-03-26T08:22:58.272 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:58.277+0000 7fa0d3c00700 1 -- 172.21.15.183:0/4101915636 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa0ac039450 msgr2=0x7fa0ac03b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:58.272 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:58.277+0000 7fa0d3c00700 1 --2- 172.21.15.183:0/4101915636 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa0ac039450 0x7fa0ac03b910 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0005fd0 tx=0x7fa0c001d040 comp rx=0 tx=0).stop 2024-03-26T08:22:58.272 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:58.277+0000 7fa0d3c00700 1 -- 172.21.15.183:0/4101915636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0cc19fb90 msgr2=0x7fa0cc1997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:22:58.273 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:58.277+0000 7fa0d3c00700 1 --2- 172.21.15.183:0/4101915636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0cc19fb90 0x7fa0cc1997d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800b780 tx=0x7fa0b800ba90 comp rx=0 tx=0).stop 2024-03-26T08:22:58.273 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:58.277+0000 7fa0d3c00700 1 -- 172.21.15.183:0/4101915636 shutdown_connections 2024-03-26T08:22:58.273 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:58.277+0000 7fa0d3c00700 1 --2- 172.21.15.183:0/4101915636 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa0ac039450 0x7fa0ac03b910 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:58.273 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:58.277+0000 7fa0d3c00700 1 --2- 172.21.15.183:0/4101915636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0cc199d10 0x7fa0cc19a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:58.273 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:58.277+0000 7fa0d3c00700 1 --2- 172.21.15.183:0/4101915636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0cc101eb0 0x7fa0cc19f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:58.273 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:58.277+0000 7fa0d3c00700 1 --2- 172.21.15.183:0/4101915636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0cc19fb90 0x7fa0cc1997d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:22:58.274 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:58.277+0000 7fa0d3c00700 1 -- 172.21.15.183:0/4101915636 >> 172.21.15.183:0/4101915636 conn(0x7fa0cc0fd720 msgr2=0x7fa0cc0fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:22:58.274 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:58.278+0000 7fa0d3c00700 1 -- 172.21.15.183:0/4101915636 shutdown_connections 2024-03-26T08:22:58.274 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:22:58.278+0000 7fa0d3c00700 1 -- 172.21.15.183:0/4101915636 wait complete. 2024-03-26T08:22:58.274 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 3 2024-03-26T08:22:58.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:58 smithi043 ceph-mon[31929]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:58.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:58 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:22:58.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:58 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:22:58.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:58 smithi052 ceph-mon[35071]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:22:58.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:58 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:22:58.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:58 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:22:59.083 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-26T08:22:59.083 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph config generate-minimal-conf 2024-03-26T08:22:59.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:22:59 smithi183 ceph-mon[35818]: from='client.? 172.21.15.183:0/4101915636' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T08:22:59.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:22:59 smithi052 ceph-mon[35071]: from='client.? 172.21.15.183:0/4101915636' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T08:22:59.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:22:59 smithi043 ceph-mon[31929]: from='client.? 172.21.15.183:0/4101915636' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T08:22:59.557 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:23:00.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:00 smithi183 ceph-mon[35818]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:00.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:00 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:00.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:00 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:00.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:00 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:00.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:00 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:00.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:00 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:00.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:00 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:23:00.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:00 smithi183 ceph-mon[35818]: Updating smithi043:/etc/ceph/ceph.conf 2024-03-26T08:23:00.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:00 smithi183 ceph-mon[35818]: Updating smithi052:/etc/ceph/ceph.conf 2024-03-26T08:23:00.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:00 smithi183 ceph-mon[35818]: Updating smithi183:/etc/ceph/ceph.conf 2024-03-26T08:23:00.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:00 smithi043 ceph-mon[31929]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:00.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:00 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:00.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:00 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:00.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:00 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:00.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:00 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:00.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:00 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:00.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:00 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:23:00.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:00 smithi043 ceph-mon[31929]: Updating smithi043:/etc/ceph/ceph.conf 2024-03-26T08:23:00.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:00 smithi043 ceph-mon[31929]: Updating smithi052:/etc/ceph/ceph.conf 2024-03-26T08:23:00.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:00 smithi043 ceph-mon[31929]: Updating smithi183:/etc/ceph/ceph.conf 2024-03-26T08:23:00.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:00 smithi052 ceph-mon[35071]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:00.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:00 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:00.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:00 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:00.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:00 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:00.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:00 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:00.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:00 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:00.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:00 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:23:00.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:00 smithi052 ceph-mon[35071]: Updating smithi043:/etc/ceph/ceph.conf 2024-03-26T08:23:00.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:00 smithi052 ceph-mon[35071]: Updating smithi052:/etc/ceph/ceph.conf 2024-03-26T08:23:00.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:00 smithi052 ceph-mon[35071]: Updating smithi183:/etc/ceph/ceph.conf 2024-03-26T08:23:01.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:01 smithi043 ceph-mon[31929]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:01.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:01 smithi043 ceph-mon[31929]: Updating smithi043:/var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:23:01.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:01 smithi043 ceph-mon[31929]: Updating smithi183:/var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:23:01.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:01 smithi043 ceph-mon[31929]: Updating smithi052:/var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:23:01.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:01 smithi183 ceph-mon[35818]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:01.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:01 smithi183 ceph-mon[35818]: Updating smithi043:/var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:23:01.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:01 smithi183 ceph-mon[35818]: Updating smithi183:/var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:23:01.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:01 smithi183 ceph-mon[35818]: Updating smithi052:/var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:23:01.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:01 smithi052 ceph-mon[35071]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:01.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:01 smithi052 ceph-mon[35071]: Updating smithi043:/var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:23:01.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:01 smithi052 ceph-mon[35071]: Updating smithi183:/var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:23:01.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:01 smithi052 ceph-mon[35071]: Updating smithi052:/var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:23:01.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.752+0000 7f2f3ac00700 1 -- 172.21.15.43:0/1239435393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f34103ff0 msgr2=0x7f2f341043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:23:01.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.752+0000 7f2f3ac00700 1 --2- 172.21.15.43:0/1239435393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f34103ff0 0x7f2f341043d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2f24009910 tx=0x7f2f24009c20 comp rx=0 tx=0).stop 2024-03-26T08:23:01.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.752+0000 7f2f3ac00700 1 -- 172.21.15.43:0/1239435393 shutdown_connections 2024-03-26T08:23:01.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.752+0000 7f2f3ac00700 1 --2- 172.21.15.43:0/1239435393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f34103ff0 0x7f2f341043d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:23:01.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.752+0000 7f2f3ac00700 1 -- 172.21.15.43:0/1239435393 >> 172.21.15.43:0/1239435393 conn(0x7f2f340ff880 msgr2=0x7f2f34101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:23:01.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.752+0000 7f2f3ac00700 1 -- 172.21.15.43:0/1239435393 shutdown_connections 2024-03-26T08:23:01.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.752+0000 7f2f3ac00700 1 -- 172.21.15.43:0/1239435393 wait complete. 2024-03-26T08:23:01.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.753+0000 7f2f3ac00700 1 Processor -- start 2024-03-26T08:23:01.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.753+0000 7f2f3ac00700 1 -- start start 2024-03-26T08:23:01.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.754+0000 7f2f3ac00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f34103ff0 0x7f2f3419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:23:01.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.754+0000 7f2f3ac00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f3419fbf0 0x7f2f341998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:23:01.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.754+0000 7f2f3ac00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f34199e10 0x7f2f3419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:23:01.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.754+0000 7f2f3ac00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3410e270 con 0x7f2f34199e10 2024-03-26T08:23:01.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.754+0000 7f2f3ac00700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3410e0f0 con 0x7f2f34103ff0 2024-03-26T08:23:01.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.754+0000 7f2f3ac00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3410e3f0 con 0x7f2f3419fbf0 2024-03-26T08:23:01.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.754+0000 7f2f38c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f34199e10 0x7f2f3419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:23:01.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.754+0000 7f2f38c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f34199e10 0x7f2f3419a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33924/0 (socket says 172.21.15.43:33924) 2024-03-26T08:23:01.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.754+0000 7f2f38c00700 1 -- 172.21.15.43:0/2434414947 learned_addr learned my addr 172.21.15.43:0/2434414947 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:23:01.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.754+0000 7f2f33e00700 1 --2- 172.21.15.43:0/2434414947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f34103ff0 0x7f2f3419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:23:01.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.755+0000 7f2f33e00700 1 -- 172.21.15.43:0/2434414947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f34103ff0 msgr2=0x7f2f3419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-26T08:23:01.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.755+0000 7f2f33e00700 1 -- 172.21.15.43:0/2434414947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f34103ff0 msgr2=0x7f2f3419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-26T08:23:01.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.755+0000 7f2f33e00700 1 --2- 172.21.15.43:0/2434414947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f34103ff0 0x7f2f3419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-26T08:23:01.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.755+0000 7f2f33e00700 1 --2- 172.21.15.43:0/2434414947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f34103ff0 0x7f2f3419f6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T08:23:01.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.755+0000 7f2f33400700 1 --2- 172.21.15.43:0/2434414947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f3419fbf0 0x7f2f341998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:23:01.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.755+0000 7f2f38c00700 1 -- 172.21.15.43:0/2434414947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f3419fbf0 msgr2=0x7f2f341998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:23:01.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.755+0000 7f2f38c00700 1 --2- 172.21.15.43:0/2434414947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f3419fbf0 0x7f2f341998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:23:01.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.755+0000 7f2f38c00700 1 -- 172.21.15.43:0/2434414947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f34103ff0 msgr2=0x7f2f3419f6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T08:23:01.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.755+0000 7f2f38c00700 1 --2- 172.21.15.43:0/2434414947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f34103ff0 0x7f2f3419f6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:23:01.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.755+0000 7f2f38c00700 1 -- 172.21.15.43:0/2434414947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f341a6510 con 0x7f2f34199e10 2024-03-26T08:23:01.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.755+0000 7f2f33400700 1 --2- 172.21.15.43:0/2434414947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f3419fbf0 0x7f2f341998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:23:01.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.755+0000 7f2f38c00700 1 --2- 172.21.15.43:0/2434414947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f34199e10 0x7f2f3419a2c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00ba50 tx=0x7f2f2c00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:23:01.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.756+0000 7f2f30c00700 1 -- 172.21.15.43:0/2434414947 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c019070 con 0x7f2f34199e10 2024-03-26T08:23:01.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.756+0000 7f2f30c00700 1 -- 172.21.15.43:0/2434414947 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2f2c014440 con 0x7f2f34199e10 2024-03-26T08:23:01.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.756+0000 7f2f30c00700 1 -- 172.21.15.43:0/2434414947 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c007590 con 0x7f2f34199e10 2024-03-26T08:23:01.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.756+0000 7f2f3ac00700 1 -- 172.21.15.43:0/2434414947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f341a67a0 con 0x7f2f34199e10 2024-03-26T08:23:01.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.756+0000 7f2f3ac00700 1 -- 172.21.15.43:0/2434414947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f341a6cb0 con 0x7f2f34199e10 2024-03-26T08:23:01.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.757+0000 7f2f30c00700 1 -- 172.21.15.43:0/2434414947 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f2f2c01c070 con 0x7f2f34199e10 2024-03-26T08:23:01.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.757+0000 7f2f3ac00700 1 -- 172.21.15.43:0/2434414947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f341054d0 con 0x7f2f34199e10 2024-03-26T08:23:01.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.758+0000 7f2f30c00700 1 --2- 172.21.15.43:0/2434414947 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2f18039450 0x7f2f1803b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:23:01.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.758+0000 7f2f30c00700 1 -- 172.21.15.43:0/2434414947 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2f2c04b720 con 0x7f2f34199e10 2024-03-26T08:23:01.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.758+0000 7f2f33e00700 1 --2- 172.21.15.43:0/2434414947 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2f18039450 0x7f2f1803b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:23:01.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.759+0000 7f2f33e00700 1 --2- 172.21.15.43:0/2434414947 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2f18039450 0x7f2f1803b910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2f24009750 tx=0x7f2f2401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:23:01.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.764+0000 7f2f30c00700 1 -- 172.21.15.43:0/2434414947 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2f2c014980 con 0x7f2f34199e10 2024-03-26T08:23:01.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.994+0000 7f2f3ac00700 1 -- 172.21.15.43:0/2434414947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f2f341a6fc0 con 0x7f2f34199e10 2024-03-26T08:23:01.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.994+0000 7f2f30c00700 1 -- 172.21.15.43:0/2434414947 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7f2f2c007730 con 0x7f2f34199e10 2024-03-26T08:23:01.989 INFO:teuthology.orchestra.run.smithi043.stdout:# minimal ceph.conf for 88759420-eb49-11ee-95d0-87774f69a715 2024-03-26T08:23:01.989 INFO:teuthology.orchestra.run.smithi043.stdout:[global] 2024-03-26T08:23:01.990 INFO:teuthology.orchestra.run.smithi043.stdout: fsid = 88759420-eb49-11ee-95d0-87774f69a715 2024-03-26T08:23:01.990 INFO:teuthology.orchestra.run.smithi043.stdout: mon_host = [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] 2024-03-26T08:23:01.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.997+0000 7f2f3ac00700 1 -- 172.21.15.43:0/2434414947 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2f18039450 msgr2=0x7f2f1803b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:23:01.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.997+0000 7f2f3ac00700 1 --2- 172.21.15.43:0/2434414947 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2f18039450 0x7f2f1803b910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2f24009750 tx=0x7f2f2401a040 comp rx=0 tx=0).stop 2024-03-26T08:23:01.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.997+0000 7f2f3ac00700 1 -- 172.21.15.43:0/2434414947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f34199e10 msgr2=0x7f2f3419a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:23:01.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.997+0000 7f2f3ac00700 1 --2- 172.21.15.43:0/2434414947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f34199e10 0x7f2f3419a2c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00ba50 tx=0x7f2f2c00be10 comp rx=0 tx=0).stop 2024-03-26T08:23:01.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.998+0000 7f2f3ac00700 1 -- 172.21.15.43:0/2434414947 shutdown_connections 2024-03-26T08:23:01.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.998+0000 7f2f3ac00700 1 --2- 172.21.15.43:0/2434414947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f34199e10 0x7f2f3419a2c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:23:01.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.998+0000 7f2f3ac00700 1 --2- 172.21.15.43:0/2434414947 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2f18039450 0x7f2f1803b910 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:23:01.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.998+0000 7f2f3ac00700 1 --2- 172.21.15.43:0/2434414947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f34103ff0 0x7f2f3419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:23:01.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.998+0000 7f2f3ac00700 1 --2- 172.21.15.43:0/2434414947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f3419fbf0 0x7f2f341998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:23:01.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.998+0000 7f2f3ac00700 1 -- 172.21.15.43:0/2434414947 >> 172.21.15.43:0/2434414947 conn(0x7f2f340ff880 msgr2=0x7f2f341059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:23:01.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.998+0000 7f2f3ac00700 1 -- 172.21.15.43:0/2434414947 shutdown_connections 2024-03-26T08:23:01.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:01.998+0000 7f2f3ac00700 1 -- 172.21.15.43:0/2434414947 wait complete. 2024-03-26T08:23:02.318 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:02 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.318 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:02 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.318 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:02 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.319 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:02 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.319 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:02 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.319 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:02 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.319 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:02 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.319 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:02 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.319 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:02 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2434414947' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:02.319 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:02 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.319 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:02 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.319 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:02 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.319 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:02 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.319 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:02 smithi043 ceph-mon[31929]: Reconfiguring mon.a (unknown last config time)... 2024-03-26T08:23:02.319 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:02 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:23:02.319 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:02 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T08:23:02.319 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:02 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:02.319 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:02 smithi043 ceph-mon[31929]: Reconfiguring daemon mon.a on smithi043 2024-03-26T08:23:02.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:02 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:02 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:02 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:02 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:02 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:02 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:02 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:02 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:02 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2434414947' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:02.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:02 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:02 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:02 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:02 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:02 smithi183 ceph-mon[35818]: Reconfiguring mon.a (unknown last config time)... 2024-03-26T08:23:02.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:02 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:23:02.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:02 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T08:23:02.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:02 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:02.552 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:02 smithi183 ceph-mon[35818]: Reconfiguring daemon mon.a on smithi043 2024-03-26T08:23:02.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:02 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:02 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:02 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:02 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:02 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:02 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:02 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:02 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:02 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2434414947' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:02.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:02 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:02 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:02 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:02 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:02.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:02 smithi052 ceph-mon[35071]: Reconfiguring mon.a (unknown last config time)... 2024-03-26T08:23:02.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:02 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:23:02.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:02 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T08:23:02.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:02 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:02.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:02 smithi052 ceph-mon[35071]: Reconfiguring daemon mon.a on smithi043 2024-03-26T08:23:02.726 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-26T08:23:02.726 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-26T08:23:02.726 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.conf 2024-03-26T08:23:02.798 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-26T08:23:02.798 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T08:23:02.919 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-26T08:23:02.919 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/ceph/ceph.conf 2024-03-26T08:23:03.021 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-26T08:23:03.021 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T08:23:03.168 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-03-26T08:23:03.168 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.conf 2024-03-26T08:23:03.268 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-03-26T08:23:03.270 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T08:23:03.408 INFO:tasks.cephadm:Adding mgr.x on smithi043 2024-03-26T08:23:03.408 INFO:tasks.cephadm:Adding mgr.y on smithi052 2024-03-26T08:23:03.408 INFO:tasks.cephadm:Adding mgr.z on smithi183 2024-03-26T08:23:03.409 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph orch apply mgr '3;smithi043=x;smithi052=y;smithi183=z' 2024-03-26T08:23:03.887 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:23:04.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:04 smithi043 ceph-mon[31929]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:04.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:04 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:04.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:04 smithi183 ceph-mon[35818]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:04.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:04 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:04.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:04 smithi052 ceph-mon[35071]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:04.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:04 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:05.264 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:05 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:05.264 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:05 smithi052 ceph-mon[35071]: Reconfiguring mon.b (monmap changed)... 2024-03-26T08:23:05.264 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:05 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:23:05.264 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:05 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T08:23:05.264 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:05 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:05.264 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:05 smithi052 ceph-mon[35071]: Reconfiguring daemon mon.b on smithi052 2024-03-26T08:23:05.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:05 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:05.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:05 smithi183 ceph-mon[35818]: Reconfiguring mon.b (monmap changed)... 2024-03-26T08:23:05.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:05 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:23:05.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:05 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T08:23:05.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:05 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:05.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:05 smithi183 ceph-mon[35818]: Reconfiguring daemon mon.b on smithi052 2024-03-26T08:23:05.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:05 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:05.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:05 smithi043 ceph-mon[31929]: Reconfiguring mon.b (monmap changed)... 2024-03-26T08:23:05.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:05 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:23:05.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:05 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T08:23:05.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:05 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:05.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:05 smithi043 ceph-mon[31929]: Reconfiguring daemon mon.b on smithi052 2024-03-26T08:23:05.723 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.728+0000 7f9e8e000700 1 -- 172.21.15.183:0/4126723633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e88105560 msgr2=0x7f9e88105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:23:05.723 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.728+0000 7f9e8e000700 1 --2- 172.21.15.183:0/4126723633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e88105560 0x7f9e88105980 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009910 tx=0x7f9e78009c20 comp rx=0 tx=0).stop 2024-03-26T08:23:05.724 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.728+0000 7f9e8e000700 1 -- 172.21.15.183:0/4126723633 shutdown_connections 2024-03-26T08:23:05.724 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.728+0000 7f9e8e000700 1 --2- 172.21.15.183:0/4126723633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e88105560 0x7f9e88105980 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:23:05.724 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.728+0000 7f9e8e000700 1 -- 172.21.15.183:0/4126723633 >> 172.21.15.183:0/4126723633 conn(0x7f9e880ff4c0 msgr2=0x7f9e88101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:23:05.724 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.728+0000 7f9e8e000700 1 -- 172.21.15.183:0/4126723633 shutdown_connections 2024-03-26T08:23:05.724 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.729+0000 7f9e8e000700 1 -- 172.21.15.183:0/4126723633 wait complete. 2024-03-26T08:23:05.726 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.730+0000 7f9e8e000700 1 Processor -- start 2024-03-26T08:23:05.726 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.730+0000 7f9e8e000700 1 -- start start 2024-03-26T08:23:05.726 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.730+0000 7f9e8e000700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e88199910 0x7f9e88199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:23:05.726 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.730+0000 7f9e8e000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e8819a270 0x7f9e8819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:23:05.727 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.730+0000 7f9e8e000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e8819f440 0x7f9e881a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:23:05.727 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.730+0000 7f9e8e000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e88109ee0 con 0x7f9e8819a270 2024-03-26T08:23:05.727 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.730+0000 7f9e8e000700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e88109d60 con 0x7f9e88199910 2024-03-26T08:23:05.727 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.730+0000 7f9e8e000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8810a060 con 0x7f9e8819f440 2024-03-26T08:23:05.727 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.731+0000 7f9e87e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e8819f440 0x7f9e881a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:23:05.727 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.731+0000 7f9e87e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e8819f440 0x7f9e881a1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:50896/0 (socket says 172.21.15.183:50896) 2024-03-26T08:23:05.727 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.731+0000 7f9e87e00700 1 -- 172.21.15.183:0/1742924383 learned_addr learned my addr 172.21.15.183:0/1742924383 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-03-26T08:23:05.727 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.731+0000 7f9e86a00700 1 --2- 172.21.15.183:0/1742924383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e8819a270 0x7f9e8819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:23:05.727 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.731+0000 7f9e87400700 1 --2- 172.21.15.183:0/1742924383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e88199910 0x7f9e88199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:23:05.727 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.731+0000 7f9e87e00700 1 -- 172.21.15.183:0/1742924383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e88199910 msgr2=0x7f9e88199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:23:05.727 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.731+0000 7f9e87e00700 1 --2- 172.21.15.183:0/1742924383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e88199910 0x7f9e88199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:23:05.727 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.731+0000 7f9e87e00700 1 -- 172.21.15.183:0/1742924383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e8819a270 msgr2=0x7f9e8819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:23:05.727 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.731+0000 7f9e87e00700 1 --2- 172.21.15.183:0/1742924383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e8819a270 0x7f9e8819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:23:05.728 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.731+0000 7f9e87e00700 1 -- 172.21.15.183:0/1742924383 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e881a2160 con 0x7f9e8819f440 2024-03-26T08:23:05.728 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.731+0000 7f9e86a00700 1 --2- 172.21.15.183:0/1742924383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e8819a270 0x7f9e8819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:23:05.728 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.732+0000 7f9e87e00700 1 --2- 172.21.15.183:0/1742924383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e8819f440 0x7f9e881a1890 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9e8000ba50 tx=0x7f9e8000be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:23:05.728 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.732+0000 7f9e7fe00700 1 -- 172.21.15.183:0/1742924383 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e80019070 con 0x7f9e8819f440 2024-03-26T08:23:05.728 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.732+0000 7f9e7fe00700 1 -- 172.21.15.183:0/1742924383 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9e80014440 con 0x7f9e8819f440 2024-03-26T08:23:05.728 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.732+0000 7f9e8e000700 1 -- 172.21.15.183:0/1742924383 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e881a2450 con 0x7f9e8819f440 2024-03-26T08:23:05.728 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.732+0000 7f9e7fe00700 1 -- 172.21.15.183:0/1742924383 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e80007590 con 0x7f9e8819f440 2024-03-26T08:23:05.728 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.732+0000 7f9e8e000700 1 -- 172.21.15.183:0/1742924383 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e881a2a60 con 0x7f9e8819f440 2024-03-26T08:23:05.729 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.733+0000 7f9e8e000700 1 -- 172.21.15.183:0/1742924383 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e880637c0 con 0x7f9e8819f440 2024-03-26T08:23:05.729 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.733+0000 7f9e7fe00700 1 -- 172.21.15.183:0/1742924383 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f9e8001c070 con 0x7f9e8819f440 2024-03-26T08:23:05.729 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.734+0000 7f9e7fe00700 1 --2- 172.21.15.183:0/1742924383 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f9e6c039450 0x7f9e6c03b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:23:05.729 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.734+0000 7f9e7fe00700 1 -- 172.21.15.183:0/1742924383 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9e8004b710 con 0x7f9e8819f440 2024-03-26T08:23:05.729 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.735+0000 7f9e87400700 1 --2- 172.21.15.183:0/1742924383 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f9e6c039450 0x7f9e6c03b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:23:05.729 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.736+0000 7f9e87400700 1 --2- 172.21.15.183:0/1742924383 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f9e6c039450 0x7f9e6c03b910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9e78005e00 tx=0x7f9e7801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:23:05.734 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.740+0000 7f9e7fe00700 1 -- 172.21.15.183:0/1742924383 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9e8001b2c0 con 0x7f9e8819f440 2024-03-26T08:23:05.967 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.973+0000 7f9e8e000700 1 -- 172.21.15.183:0/1742924383 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi043=x;smithi052=y;smithi183=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e880625d0 con 0x7f9e6c039450 2024-03-26T08:23:05.981 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.987+0000 7f9e7fe00700 1 -- 172.21.15.183:0/1742924383 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f9e880625d0 con 0x7f9e6c039450 2024-03-26T08:23:05.981 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled mgr update... 2024-03-26T08:23:05.984 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.989+0000 7f9e8e000700 1 -- 172.21.15.183:0/1742924383 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f9e6c039450 msgr2=0x7f9e6c03b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:23:05.984 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.989+0000 7f9e8e000700 1 --2- 172.21.15.183:0/1742924383 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f9e6c039450 0x7f9e6c03b910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9e78005e00 tx=0x7f9e7801a040 comp rx=0 tx=0).stop 2024-03-26T08:23:05.984 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.989+0000 7f9e8e000700 1 -- 172.21.15.183:0/1742924383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e8819f440 msgr2=0x7f9e881a1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:23:05.984 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.989+0000 7f9e8e000700 1 --2- 172.21.15.183:0/1742924383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e8819f440 0x7f9e881a1890 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9e8000ba50 tx=0x7f9e8000be10 comp rx=0 tx=0).stop 2024-03-26T08:23:05.984 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.990+0000 7f9e8e000700 1 -- 172.21.15.183:0/1742924383 shutdown_connections 2024-03-26T08:23:05.984 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.990+0000 7f9e8e000700 1 --2- 172.21.15.183:0/1742924383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e8819f440 0x7f9e881a1890 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:23:05.984 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.990+0000 7f9e8e000700 1 --2- 172.21.15.183:0/1742924383 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f9e6c039450 0x7f9e6c03b910 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:23:05.984 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.990+0000 7f9e8e000700 1 --2- 172.21.15.183:0/1742924383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e88199910 0x7f9e88199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:23:05.985 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.990+0000 7f9e8e000700 1 --2- 172.21.15.183:0/1742924383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e8819a270 0x7f9e8819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:23:05.985 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.990+0000 7f9e8e000700 1 -- 172.21.15.183:0/1742924383 >> 172.21.15.183:0/1742924383 conn(0x7f9e880ff4c0 msgr2=0x7f9e880ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:23:05.985 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.990+0000 7f9e8e000700 1 -- 172.21.15.183:0/1742924383 shutdown_connections 2024-03-26T08:23:05.985 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:23:05.990+0000 7f9e8e000700 1 -- 172.21.15.183:0/1742924383 wait complete. 2024-03-26T08:23:06.190 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:06 smithi052 ceph-mon[35071]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:06.190 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:06 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:06.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:06 smithi183 ceph-mon[35818]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:06.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:06 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:06.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:06 smithi043 ceph-mon[31929]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:06.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:06 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:06.749 DEBUG:teuthology.orchestra.run.smithi052:mgr.y> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mgr.y.service 2024-03-26T08:23:06.756 DEBUG:teuthology.orchestra.run.smithi183:mgr.z> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mgr.z.service 2024-03-26T08:23:06.762 INFO:tasks.cephadm:Deploying OSDs... 2024-03-26T08:23:06.762 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-26T08:23:06.762 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/scratch_devs of=/dev/stdout 2024-03-26T08:23:06.823 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-26T08:23:06.823 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_1 2024-03-26T08:23:06.876 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-26T08:23:06.876 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T08:23:06.876 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 633 Links: 1 2024-03-26T08:23:06.876 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T08:23:06.876 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T08:23:06.876 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-03-26 08:21:50.275039937 +0000 2024-03-26T08:23:06.876 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-03-26 08:16:09.680664054 +0000 2024-03-26T08:23:06.876 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-03-26 08:16:09.680664054 +0000 2024-03-26T08:23:06.876 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-03-26 08:16:09.680664054 +0000 2024-03-26T08:23:06.877 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-26T08:23:06.969 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-03-26T08:23:06.969 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-03-26T08:23:06.969 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.00024175 s, 2.1 MB/s 2024-03-26T08:23:06.974 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-26T08:23:07.036 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_2 2024-03-26T08:23:07.091 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-26T08:23:07.091 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T08:23:07.091 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 652 Links: 1 2024-03-26T08:23:07.091 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T08:23:07.091 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T08:23:07.091 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-03-26 08:21:50.375038622 +0000 2024-03-26T08:23:07.091 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-03-26 08:16:10.275656318 +0000 2024-03-26T08:23:07.091 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-03-26 08:16:10.275656318 +0000 2024-03-26T08:23:07.091 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-03-26 08:16:10.275656318 +0000 2024-03-26T08:23:07.092 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-26T08:23:07.186 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-03-26T08:23:07.186 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-03-26T08:23:07.186 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000277357 s, 1.8 MB/s 2024-03-26T08:23:07.190 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-26T08:23:07.247 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_3 2024-03-26T08:23:07.298 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-26T08:23:07.298 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T08:23:07.298 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 672 Links: 1 2024-03-26T08:23:07.298 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T08:23:07.298 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T08:23:07.298 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-03-26 08:21:50.460037503 +0000 2024-03-26T08:23:07.298 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-03-26 08:16:10.845648906 +0000 2024-03-26T08:23:07.298 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-03-26 08:16:10.845648906 +0000 2024-03-26T08:23:07.298 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-03-26 08:16:10.845648906 +0000 2024-03-26T08:23:07.299 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-26T08:23:07.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:07 smithi183 ceph-mon[35818]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi043=x;smithi052=y;smithi183=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:23:07.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:07 smithi183 ceph-mon[35818]: Saving service mgr spec with placement smithi043=x;smithi052=y;smithi183=z;count:3 2024-03-26T08:23:07.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:07 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:07.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:07 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:07.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:07 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:23:07.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:07 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T08:23:07.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:07 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:07.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:07 smithi052 ceph-mon[35071]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi043=x;smithi052=y;smithi183=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:23:07.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:07 smithi052 ceph-mon[35071]: Saving service mgr spec with placement smithi043=x;smithi052=y;smithi183=z;count:3 2024-03-26T08:23:07.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:07 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:07.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:07 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:07.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:07 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:23:07.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:07 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T08:23:07.325 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:07 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:07.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:07 smithi043 ceph-mon[31929]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi043=x;smithi052=y;smithi183=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:23:07.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:07 smithi043 ceph-mon[31929]: Saving service mgr spec with placement smithi043=x;smithi052=y;smithi183=z;count:3 2024-03-26T08:23:07.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:07 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:07.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:07 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:07.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:07 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:23:07.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:07 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T08:23:07.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:07 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:07.394 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-03-26T08:23:07.394 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-03-26T08:23:07.394 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000242725 s, 2.1 MB/s 2024-03-26T08:23:07.399 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-26T08:23:07.462 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_4 2024-03-26T08:23:07.517 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-26T08:23:07.517 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T08:23:07.518 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 690 Links: 1 2024-03-26T08:23:07.518 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T08:23:07.518 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T08:23:07.518 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-03-26 08:21:50.528036609 +0000 2024-03-26T08:23:07.518 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-03-26 08:16:11.435641235 +0000 2024-03-26T08:23:07.518 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-03-26 08:16:11.435641235 +0000 2024-03-26T08:23:07.518 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-03-26 08:16:11.435641235 +0000 2024-03-26T08:23:07.518 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-26T08:23:07.614 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-03-26T08:23:07.614 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-03-26T08:23:07.614 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000276965 s, 1.8 MB/s 2024-03-26T08:23:07.619 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-26T08:23:07.678 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-26T08:23:07.678 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/scratch_devs of=/dev/stdout 2024-03-26T08:23:07.737 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-26T08:23:07.737 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_1 2024-03-26T08:23:07.785 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-26T08:23:07.785 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T08:23:07.785 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 632 Links: 1 2024-03-26T08:23:07.786 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T08:23:07.786 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T08:23:07.786 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-03-26 08:22:51.116777301 +0000 2024-03-26T08:23:07.786 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-03-26 08:16:11.733668573 +0000 2024-03-26T08:23:07.786 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-03-26 08:16:11.733668573 +0000 2024-03-26T08:23:07.786 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-03-26 08:16:11.733668573 +0000 2024-03-26T08:23:07.786 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-26T08:23:07.867 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-03-26T08:23:07.867 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-03-26T08:23:07.868 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000265138 s, 1.9 MB/s 2024-03-26T08:23:07.871 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-26T08:23:07.929 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_2 2024-03-26T08:23:07.983 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-26T08:23:07.984 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T08:23:07.984 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 653 Links: 1 2024-03-26T08:23:07.984 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T08:23:07.984 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T08:23:07.984 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-03-26 08:22:51.117777288 +0000 2024-03-26T08:23:07.984 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-03-26 08:16:12.302661319 +0000 2024-03-26T08:23:07.984 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-03-26 08:16:12.302661319 +0000 2024-03-26T08:23:07.984 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-03-26 08:16:12.302661319 +0000 2024-03-26T08:23:07.985 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-26T08:23:08.075 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:08 smithi183 ceph-mon[35818]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:08.075 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:08 smithi183 ceph-mon[35818]: Reconfiguring mon.c (monmap changed)... 2024-03-26T08:23:08.075 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:08 smithi183 ceph-mon[35818]: Reconfiguring daemon mon.c on smithi183 2024-03-26T08:23:08.082 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-03-26T08:23:08.082 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-03-26T08:23:08.082 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000330566 s, 1.5 MB/s 2024-03-26T08:23:08.087 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-26T08:23:08.144 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_3 2024-03-26T08:23:08.194 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-26T08:23:08.194 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T08:23:08.194 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 669 Links: 1 2024-03-26T08:23:08.194 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T08:23:08.194 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T08:23:08.195 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-03-26 08:22:51.118777275 +0000 2024-03-26T08:23:08.195 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-03-26 08:16:12.870654078 +0000 2024-03-26T08:23:08.195 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-03-26 08:16:12.870654078 +0000 2024-03-26T08:23:08.195 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-03-26 08:16:12.870654078 +0000 2024-03-26T08:23:08.195 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-26T08:23:08.284 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-03-26T08:23:08.284 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-03-26T08:23:08.285 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000273977 s, 1.9 MB/s 2024-03-26T08:23:08.289 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-26T08:23:08.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:08 smithi052 ceph-mon[35071]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:08.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:08 smithi052 ceph-mon[35071]: Reconfiguring mon.c (monmap changed)... 2024-03-26T08:23:08.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:08 smithi052 ceph-mon[35071]: Reconfiguring daemon mon.c on smithi183 2024-03-26T08:23:08.348 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_4 2024-03-26T08:23:08.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:08 smithi043 ceph-mon[31929]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:08.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:08 smithi043 ceph-mon[31929]: Reconfiguring mon.c (monmap changed)... 2024-03-26T08:23:08.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:08 smithi043 ceph-mon[31929]: Reconfiguring daemon mon.c on smithi183 2024-03-26T08:23:08.449 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-26T08:23:08.449 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T08:23:08.450 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 690 Links: 1 2024-03-26T08:23:08.450 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T08:23:08.450 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T08:23:08.451 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-03-26 08:22:51.118777275 +0000 2024-03-26T08:23:08.451 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-03-26 08:16:13.455646621 +0000 2024-03-26T08:23:08.451 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-03-26 08:16:13.455646621 +0000 2024-03-26T08:23:08.451 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-03-26 08:16:13.455646621 +0000 2024-03-26T08:23:08.451 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-26T08:23:08.545 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-03-26T08:23:08.546 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-03-26T08:23:08.546 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000289144 s, 1.8 MB/s 2024-03-26T08:23:08.550 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-26T08:23:08.609 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-03-26T08:23:08.610 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/scratch_devs of=/dev/stdout 2024-03-26T08:23:08.663 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-26T08:23:08.663 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_1 2024-03-26T08:23:08.709 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-26T08:23:08.709 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T08:23:08.709 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 632 Links: 1 2024-03-26T08:23:08.709 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T08:23:08.709 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T08:23:08.709 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-03-26 08:22:40.163139505 +0000 2024-03-26T08:23:08.709 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-03-26 08:16:06.551638921 +0000 2024-03-26T08:23:08.709 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-03-26 08:16:06.551638921 +0000 2024-03-26T08:23:08.709 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-03-26 08:16:06.551638921 +0000 2024-03-26T08:23:08.710 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-26T08:23:08.838 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-03-26T08:23:08.838 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-03-26T08:23:08.838 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.00037614 s, 1.4 MB/s 2024-03-26T08:23:08.842 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-26T08:23:08.891 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_2 2024-03-26T08:23:08.934 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-26T08:23:08.934 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T08:23:08.935 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 652 Links: 1 2024-03-26T08:23:08.935 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T08:23:08.935 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T08:23:08.935 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-03-26 08:22:40.164139478 +0000 2024-03-26T08:23:08.935 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-03-26 08:16:07.143623023 +0000 2024-03-26T08:23:08.935 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-03-26 08:16:07.143623023 +0000 2024-03-26T08:23:08.935 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-03-26 08:16:07.143623023 +0000 2024-03-26T08:23:08.936 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-26T08:23:09.052 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-03-26T08:23:09.052 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-03-26T08:23:09.052 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000324905 s, 1.6 MB/s 2024-03-26T08:23:09.055 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-26T08:23:09.095 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_3 2024-03-26T08:23:09.178 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-26T08:23:09.178 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T08:23:09.178 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 671 Links: 1 2024-03-26T08:23:09.179 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T08:23:09.179 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T08:23:09.179 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-03-26 08:22:40.165139451 +0000 2024-03-26T08:23:09.179 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-03-26 08:16:07.716607636 +0000 2024-03-26T08:23:09.179 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-03-26 08:16:07.716607636 +0000 2024-03-26T08:23:09.179 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-03-26 08:16:07.716607636 +0000 2024-03-26T08:23:09.179 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-26T08:23:09.297 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-03-26T08:23:09.297 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-03-26T08:23:09.297 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.00033695 s, 1.5 MB/s 2024-03-26T08:23:09.300 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-26T08:23:09.353 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_4 2024-03-26T08:23:09.400 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-26T08:23:09.400 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T08:23:09.400 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 690 Links: 1 2024-03-26T08:23:09.400 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T08:23:09.400 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T08:23:09.400 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-03-26 08:22:40.165139451 +0000 2024-03-26T08:23:09.400 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-03-26 08:16:08.295592088 +0000 2024-03-26T08:23:09.400 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-03-26 08:16:08.295592088 +0000 2024-03-26T08:23:09.400 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-03-26 08:16:08.295592088 +0000 2024-03-26T08:23:09.401 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-26T08:23:09.466 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:09 smithi183 ceph-mon[35818]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:09.466 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:09 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:09.466 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:09 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:09.466 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:09 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:23:09.466 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:09 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:09.466 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:09 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:23:09.466 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:09 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:09.467 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:09 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T08:23:09.467 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:09 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T08:23:09.467 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:09 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T08:23:09.467 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:09 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:09.467 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:09 smithi183 ceph-mon[35818]: Deploying daemon mgr.z on smithi183 2024-03-26T08:23:09.479 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-03-26T08:23:09.479 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-03-26T08:23:09.479 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000311519 s, 1.6 MB/s 2024-03-26T08:23:09.482 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-26T08:23:09.524 INFO:tasks.cephadm:Deploying osd.0 on smithi043 with /dev/vg_nvme/lv_4... 2024-03-26T08:23:09.524 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-26T08:23:09.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:09 smithi052 ceph-mon[35071]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:09.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:09 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:09.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:09 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:09.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:09 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:23:09.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:09 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:09.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:09 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:23:09.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:09 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:09.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:09 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T08:23:09.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:09 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T08:23:09.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:09 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T08:23:09.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:09 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:09.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:09 smithi052 ceph-mon[35071]: Deploying daemon mgr.z on smithi183 2024-03-26T08:23:09.618 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:09 smithi043 ceph-mon[31929]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:09.618 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:09 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:09.618 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:09 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:09.618 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:09 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:23:09.619 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:09 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:09.619 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:09 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:23:09.619 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:09 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:09.619 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:09 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T08:23:09.619 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:09 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T08:23:09.619 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:09 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T08:23:09.619 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:09 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:09.619 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:09 smithi043 ceph-mon[31929]: Deploying daemon mgr.z on smithi183 2024-03-26T08:23:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:23:11.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:11 smithi183 ceph-mon[35818]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:11.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:11 smithi052 ceph-mon[35071]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:11.619 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:11 smithi043 ceph-mon[31929]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:11.621 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:11 smithi183 podman[37687]: 2024-03-26 08:23:11.39774519 +0000 UTC m=+0.160337041 container create 5eb7852de1c7988488694a3d235bf39ba8a382b50065652d7d13118809e61c20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T08:23:11.621 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:11 smithi183 podman[37687]: 2024-03-26 08:23:11.626303534 +0000 UTC m=+0.388895411 container init 5eb7852de1c7988488694a3d235bf39ba8a382b50065652d7d13118809e61c20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-26T08:23:12.051 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:11 smithi183 podman[37687]: 2024-03-26 08:23:11.636023873 +0000 UTC m=+0.398615734 container start 5eb7852de1c7988488694a3d235bf39ba8a382b50065652d7d13118809e61c20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-26T08:23:12.051 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:11 smithi183 ceph-mgr[37706]: -- 172.21.15.183:0/1642001443 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5579507a65a0 con 0x55795077a800 2024-03-26T08:23:12.051 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:11 smithi183 bash[37687]: 5eb7852de1c7988488694a3d235bf39ba8a382b50065652d7d13118809e61c20 2024-03-26T08:23:12.051 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:11 smithi183 systemd[1]: Started Ceph mgr.z for 88759420-eb49-11ee-95d0-87774f69a715. 2024-03-26T08:23:12.800 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:12 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z[37702]: 2024-03-26T08:23:12.392+0000 7fac1b302200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T08:23:12.800 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:12 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z[37702]: 2024-03-26T08:23:12.510+0000 7fac1b302200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T08:23:12.931 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-26T08:23:12.931 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-26T08:23:12.931 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2024-03-26T08:23:12.931 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2024-03-26T08:23:12.931 INFO:teuthology.orchestra.run.smithi043.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0395665 s, 265 MB/s 2024-03-26T08:23:12.931 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2024-03-26T08:23:13.136 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:12 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:13.137 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:12 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:13.137 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:12 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:13.137 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:12 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T08:23:13.137 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:12 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T08:23:13.137 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:12 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T08:23:13.137 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:12 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:13.137 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:12 smithi052 ceph-mon[35071]: Deploying daemon mgr.y on smithi052 2024-03-26T08:23:13.155 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:12 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:13.155 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:12 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:13.155 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:12 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:13.155 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:12 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T08:23:13.155 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:12 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T08:23:13.155 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:12 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T08:23:13.155 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:12 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:13.156 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:12 smithi043 ceph-mon[31929]: Deploying daemon mgr.y on smithi052 2024-03-26T08:23:13.291 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:12 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:13.291 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:12 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:13.291 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:12 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:13.291 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:12 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T08:23:13.292 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:12 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T08:23:13.292 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:12 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T08:23:13.292 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:12 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:13.292 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:12 smithi183 ceph-mon[35818]: Deploying daemon mgr.y on smithi052 2024-03-26T08:23:13.550 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:13 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z[37702]: 2024-03-26T08:23:13.296+0000 7fac1b302200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T08:23:13.550 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:13 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z[37702]: 2024-03-26T08:23:13.416+0000 7fac1b302200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T08:23:13.750 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi043:vg_nvme/lv_4 2024-03-26T08:23:13.879 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:13 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z[37702]: 2024-03-26T08:23:13.596+0000 7fac1b302200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T08:23:13.880 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:13 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z[37702]: 2024-03-26T08:23:13.718+0000 7fac1b302200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T08:23:13.880 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:13 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z[37702]: 2024-03-26T08:23:13.826+0000 7fac1b302200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T08:23:14.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:13 smithi043 ceph-mon[31929]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:14.079 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:13 smithi052 ceph-mon[35071]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:14.151 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:13 smithi183 ceph-mon[35818]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:14.195 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:23:14.402 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:14 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z[37702]: 2024-03-26T08:23:14.156+0000 7fac1b302200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T08:23:14.402 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:14 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z[37702]: 2024-03-26T08:23:14.295+0000 7fac1b302200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T08:23:14.402 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:14 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z[37702]: 2024-03-26T08:23:14.407+0000 7fac1b302200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T08:23:14.739 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:14 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z[37702]: 2024-03-26T08:23:14.528+0000 7fac1b302200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T08:23:14.739 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:14 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z[37702]: 2024-03-26T08:23:14.744+0000 7fac1b302200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T08:23:15.265 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:14 smithi052 systemd[1]: Starting Ceph mgr.y for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:23:15.541 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:15 smithi052 podman[36425]: 2024-03-26 08:23:15.176109573 +0000 UTC m=+0.038872355 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:23:15.541 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:15 smithi052 podman[36425]: 2024-03-26 08:23:15.270817211 +0000 UTC m=+0.133580024 container create 1bdae692a3392dbad6199655b98007919cc8b0ee99a1266ecbce97cb0feefb43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-26T08:23:15.542 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:15 smithi052 podman[36425]: 2024-03-26 08:23:15.483102147 +0000 UTC m=+0.345864996 container init 1bdae692a3392dbad6199655b98007919cc8b0ee99a1266ecbce97cb0feefb43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-26T08:23:15.542 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:15 smithi052 podman[36425]: 2024-03-26 08:23:15.494004363 +0000 UTC m=+0.356767203 container start 1bdae692a3392dbad6199655b98007919cc8b0ee99a1266ecbce97cb0feefb43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-26T08:23:15.824 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:15 smithi052 ceph-mgr[36444]: -- 172.21.15.52:0/4179531832 <== mon.2 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563c865c25a0 con 0x563c86596c00 2024-03-26T08:23:15.824 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:15 smithi052 bash[36425]: 1bdae692a3392dbad6199655b98007919cc8b0ee99a1266ecbce97cb0feefb43 2024-03-26T08:23:15.824 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:15 smithi052 systemd[1]: Started Ceph mgr.y for 88759420-eb49-11ee-95d0-87774f69a715. 2024-03-26T08:23:16.115 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:15 smithi043 ceph-mon[31929]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:16.116 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:15 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:16.116 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:15 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:16.116 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:15 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:16.116 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:15 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:16.195 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:16 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z[37702]: 2024-03-26T08:23:16.027+0000 7fac1b302200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T08:23:16.196 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:16 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z[37702]: 2024-03-26T08:23:16.201+0000 7fac1b302200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T08:23:16.196 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:15 smithi183 ceph-mon[35818]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:16.196 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:15 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:16.196 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:15 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:16.197 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:15 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:16.197 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:15 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:16.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.299+0000 7f6762200700 1 -- 172.21.15.43:0/1279855829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f675c10e6e0 msgr2=0x7f675c110ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:23:16.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.299+0000 7f6762200700 1 --2- 172.21.15.43:0/1279855829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f675c10e6e0 0x7f675c110ad0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6754009990 tx=0x7f6754009ca0 comp rx=0 tx=0).stop 2024-03-26T08:23:16.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.299+0000 7f6762200700 1 -- 172.21.15.43:0/1279855829 shutdown_connections 2024-03-26T08:23:16.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.299+0000 7f6762200700 1 --2- 172.21.15.43:0/1279855829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f675c10e6e0 0x7f675c110ad0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:23:16.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.299+0000 7f6762200700 1 --2- 172.21.15.43:0/1279855829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f675c0761c0 0x7f675c074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:23:16.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.299+0000 7f6762200700 1 --2- 172.21.15.43:0/1279855829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f675c074bf0 0x7f675c075070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:23:16.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.299+0000 7f6762200700 1 -- 172.21.15.43:0/1279855829 >> 172.21.15.43:0/1279855829 conn(0x7f675c0fd6a0 msgr2=0x7f675c0ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:23:16.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.299+0000 7f6762200700 1 -- 172.21.15.43:0/1279855829 shutdown_connections 2024-03-26T08:23:16.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.300+0000 7f6762200700 1 -- 172.21.15.43:0/1279855829 wait complete. 2024-03-26T08:23:16.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.301+0000 7f6762200700 1 Processor -- start 2024-03-26T08:23:16.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.301+0000 7f6762200700 1 -- start start 2024-03-26T08:23:16.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.301+0000 7f6762200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f675c074bf0 0x7f675c19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:23:16.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.301+0000 7f6762200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f675c0761c0 0x7f675c19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:23:16.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.301+0000 7f6762200700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f675c10e6e0 0x7f675c1a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:23:16.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.301+0000 7f6762200700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c114200 con 0x7f675c074bf0 2024-03-26T08:23:16.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.301+0000 7f6762200700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c114080 con 0x7f675c10e6e0 2024-03-26T08:23:16.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.301+0000 7f6762200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c114380 con 0x7f675c0761c0 2024-03-26T08:23:16.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.302+0000 7f6760e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f675c074bf0 0x7f675c19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:23:16.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.302+0000 7f6760e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f675c074bf0 0x7f675c19e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60990/0 (socket says 172.21.15.43:60990) 2024-03-26T08:23:16.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.302+0000 7f6760e00700 1 -- 172.21.15.43:0/2746912977 learned_addr learned my addr 172.21.15.43:0/2746912977 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:23:16.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.302+0000 7f6761800700 1 --2- 172.21.15.43:0/2746912977 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f675c10e6e0 0x7f675c1a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:23:16.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.302+0000 7f6753e00700 1 --2- 172.21.15.43:0/2746912977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f675c0761c0 0x7f675c19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:23:16.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.302+0000 7f6760e00700 1 -- 172.21.15.43:0/2746912977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f675c0761c0 msgr2=0x7f675c19ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:23:16.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.302+0000 7f6760e00700 1 --2- 172.21.15.43:0/2746912977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f675c0761c0 0x7f675c19ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:23:16.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.302+0000 7f6760e00700 1 -- 172.21.15.43:0/2746912977 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f675c10e6e0 msgr2=0x7f675c1a5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:23:16.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.302+0000 7f6760e00700 1 --2- 172.21.15.43:0/2746912977 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f675c10e6e0 0x7f675c1a5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:23:16.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.302+0000 7f6760e00700 1 -- 172.21.15.43:0/2746912977 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f675c1a6420 con 0x7f675c074bf0 2024-03-26T08:23:16.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.302+0000 7f6760e00700 1 --2- 172.21.15.43:0/2746912977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f675c074bf0 0x7f675c19e7c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f674c007f40 tx=0x7f674c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:23:16.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.303+0000 7f675a000700 1 -- 172.21.15.43:0/2746912977 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c00dd20 con 0x7f675c074bf0 2024-03-26T08:23:16.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.303+0000 7f675a000700 1 -- 172.21.15.43:0/2746912977 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f674c018070 con 0x7f675c074bf0 2024-03-26T08:23:16.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.303+0000 7f6762200700 1 -- 172.21.15.43:0/2746912977 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f675c1a6710 con 0x7f675c074bf0 2024-03-26T08:23:16.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.303+0000 7f675a000700 1 -- 172.21.15.43:0/2746912977 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c0125b0 con 0x7f675c074bf0 2024-03-26T08:23:16.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.303+0000 7f6762200700 1 -- 172.21.15.43:0/2746912977 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f675c1a6ca0 con 0x7f675c074bf0 2024-03-26T08:23:16.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.304+0000 7f675a000700 1 -- 172.21.15.43:0/2746912977 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 46162+0+0 (secure 0 0 0) 0x7f674c012750 con 0x7f675c074bf0 2024-03-26T08:23:16.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.304+0000 7f6762200700 1 -- 172.21.15.43:0/2746912977 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f675c068e50 con 0x7f675c074bf0 2024-03-26T08:23:16.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.305+0000 7f675a000700 1 --2- 172.21.15.43:0/2746912977 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6740039200 0x7f674003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:23:16.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.305+0000 7f675a000700 1 -- 172.21.15.43:0/2746912977 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f674c04bf90 con 0x7f675c074bf0 2024-03-26T08:23:16.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.305+0000 7f6753e00700 1 --2- 172.21.15.43:0/2746912977 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6740039200 0x7f674003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:23:16.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.306+0000 7f6753e00700 1 --2- 172.21.15.43:0/2746912977 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6740039200 0x7f674003b6c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f675c075ef0 tx=0x7f6744009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:23:16.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.311+0000 7f675a000700 1 -- 172.21.15.43:0/2746912977 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f674c010070 con 0x7f675c074bf0 2024-03-26T08:23:16.314 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:15 smithi052 ceph-mon[35071]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:16.314 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:15 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:16.314 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:15 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:16.314 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:15 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:16.314 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:15 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:16.546+0000 7f6762200700 1 -- 172.21.15.43:0/2746912977 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f675c0625d0 con 0x7f6740039200 2024-03-26T08:23:16.574 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:16 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y[36440]: 2024-03-26T08:23:16.318+0000 7fbd1684b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T08:23:16.574 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:16 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y[36440]: 2024-03-26T08:23:16.448+0000 7fbd1684b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T08:23:16.887 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:16 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z[37702]: 2024-03-26T08:23:16.635+0000 7fac1b302200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T08:23:16.887 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:16 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z[37702]: 2024-03-26T08:23:16.757+0000 7fac1b302200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T08:23:17.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:16 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:23:17.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:16 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:17.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:16 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:23:17.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:16 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:23:17.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:16 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:17.270 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:17 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z[37702]: 2024-03-26T08:23:17.023+0000 7fac1b302200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T08:23:17.271 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:16 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:23:17.271 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:16 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:17.271 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:16 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:23:17.271 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:16 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:23:17.271 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:16 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:17.318 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:16 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:23:17.318 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:16 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:17.318 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:16 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:23:17.318 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:16 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:23:17.318 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:16 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:17.550 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:17 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z[37702]: 2024-03-26T08:23:17.276+0000 7fac1b302200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T08:23:17.574 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:17 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y[36440]: 2024-03-26T08:23:17.322+0000 7fbd1684b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T08:23:17.574 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:17 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y[36440]: 2024-03-26T08:23:17.454+0000 7fbd1684b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T08:23:17.896 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:17 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y[36440]: 2024-03-26T08:23:17.645+0000 7fbd1684b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T08:23:17.896 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:17 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y[36440]: 2024-03-26T08:23:17.781+0000 7fbd1684b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T08:23:17.984 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:17 smithi183 ceph-mon[35818]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:17.984 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:17 smithi183 ceph-mon[35818]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:23:17.984 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:17 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z[37702]: 2024-03-26T08:23:17.659+0000 7fac1b302200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T08:23:17.984 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:17 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z[37702]: 2024-03-26T08:23:17.989+0000 7fac1b302200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T08:23:18.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:17 smithi043 ceph-mon[31929]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:18.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:17 smithi043 ceph-mon[31929]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:23:18.267 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:17 smithi052 ceph-mon[35071]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:18.268 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:17 smithi052 ceph-mon[35071]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:23:18.268 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:17 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y[36440]: 2024-03-26T08:23:17.918+0000 7fbd1684b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T08:23:18.268 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:18 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y[36440]: 2024-03-26T08:23:18.272+0000 7fbd1684b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T08:23:18.538 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:18 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y[36440]: 2024-03-26T08:23:18.424+0000 7fbd1684b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T08:23:18.538 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:18 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y[36440]: 2024-03-26T08:23:18.543+0000 7fbd1684b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T08:23:18.550 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:18 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z[37702]: 2024-03-26T08:23:18.290+0000 7fac1b302200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T08:23:18.824 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:18 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y[36440]: 2024-03-26T08:23:18.676+0000 7fbd1684b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T08:23:19.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:18 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:19.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:18 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:19.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:18 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:19.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:18 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:19.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:18 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:19.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:18 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:19.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:18 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:19.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:18 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:19.324 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:18 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y[36440]: 2024-03-26T08:23:18.902+0000 7fbd1684b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T08:23:19.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:18 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:19.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:18 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:19.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:18 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:19.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:18 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:19.800 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:19 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z[37702]: 2024-03-26T08:23:19.305+0000 7fac1b302200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T08:23:20.300 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:20 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z[37702]: 2024-03-26T08:23:20.002+0000 7fac1b302200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T08:23:20.300 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:20 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z[37702]: 2024-03-26T08:23:20.131+0000 7fac1b302200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T08:23:20.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:20 smithi183 ceph-mon[35818]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:20.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:20 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:20.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:20 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:23:20.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:20 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:20.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:20 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T08:23:20.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:20 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T08:23:20.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:20 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:20.324 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:20 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y[36440]: 2024-03-26T08:23:20.257+0000 7fbd1684b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T08:23:20.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:20 smithi052 ceph-mon[35071]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:20.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:20 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:20.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:20 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:23:20.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:20 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:20.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:20 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T08:23:20.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:20 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T08:23:20.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:20 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:20.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:20 smithi043 ceph-mon[31929]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:20.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:20 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:20.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:20 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:23:20.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:20 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:20.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:20 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T08:23:20.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:20 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T08:23:20.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:20 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:20.659 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:23:20 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z[37702]: 2024-03-26T08:23:20.415+0000 7fac1b302200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T08:23:20.824 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:20 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y[36440]: 2024-03-26T08:23:20.434+0000 7fbd1684b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T08:23:21.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:21.028+0000 7f675a000700 1 -- 172.21.15.43:0/2746912977 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 14) v1 ==== 91698+0+0 (secure 0 0 0) 0x7f674c01d460 con 0x7f675c074bf0 2024-03-26T08:23:21.247 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:20 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y[36440]: 2024-03-26T08:23:20.862+0000 7fbd1684b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T08:23:21.247 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:20 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y[36440]: 2024-03-26T08:23:20.984+0000 7fbd1684b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T08:23:21.248 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:21 smithi052 ceph-mon[35071]: Reconfiguring mgr.x (unknown last config time)... 2024-03-26T08:23:21.248 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:21 smithi052 ceph-mon[35071]: Reconfiguring daemon mgr.x on smithi043 2024-03-26T08:23:21.248 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:21 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2929002037' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28670673-9193-4fa3-9cc0-c1a5a6b27284"}]: dispatch 2024-03-26T08:23:21.248 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:21 smithi052 ceph-mon[35071]: from='mgr.? 172.21.15.183:0/4019045375' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-26T08:23:21.248 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:21 smithi052 ceph-mon[35071]: from='mgr.? 172.21.15.183:0/4019045375' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T08:23:21.248 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:21 smithi052 ceph-mon[35071]: Standby manager daemon z started 2024-03-26T08:23:21.248 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:21 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2929002037' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28670673-9193-4fa3-9cc0-c1a5a6b27284"}]': finished 2024-03-26T08:23:21.248 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:21 smithi052 ceph-mon[35071]: from='mgr.? 172.21.15.183:0/4019045375' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-26T08:23:21.248 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:21 smithi052 ceph-mon[35071]: osdmap e5: 1 total, 0 up, 1 in 2024-03-26T08:23:21.249 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:21 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:23:21.249 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:21 smithi052 ceph-mon[35071]: from='mgr.? 172.21.15.183:0/4019045375' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T08:23:21.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:21 smithi183 ceph-mon[35818]: Reconfiguring mgr.x (unknown last config time)... 2024-03-26T08:23:21.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:21 smithi183 ceph-mon[35818]: Reconfiguring daemon mgr.x on smithi043 2024-03-26T08:23:21.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:21 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2929002037' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28670673-9193-4fa3-9cc0-c1a5a6b27284"}]: dispatch 2024-03-26T08:23:21.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:21 smithi183 ceph-mon[35818]: from='mgr.? 172.21.15.183:0/4019045375' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-26T08:23:21.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:21 smithi183 ceph-mon[35818]: from='mgr.? 172.21.15.183:0/4019045375' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T08:23:21.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:21 smithi183 ceph-mon[35818]: Standby manager daemon z started 2024-03-26T08:23:21.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:21 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2929002037' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28670673-9193-4fa3-9cc0-c1a5a6b27284"}]': finished 2024-03-26T08:23:21.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:21 smithi183 ceph-mon[35818]: from='mgr.? 172.21.15.183:0/4019045375' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-26T08:23:21.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:21 smithi183 ceph-mon[35818]: osdmap e5: 1 total, 0 up, 1 in 2024-03-26T08:23:21.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:21 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:23:21.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:21 smithi183 ceph-mon[35818]: from='mgr.? 172.21.15.183:0/4019045375' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T08:23:21.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:21 smithi043 ceph-mon[31929]: Reconfiguring mgr.x (unknown last config time)... 2024-03-26T08:23:21.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:21 smithi043 ceph-mon[31929]: Reconfiguring daemon mgr.x on smithi043 2024-03-26T08:23:21.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:21 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2929002037' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28670673-9193-4fa3-9cc0-c1a5a6b27284"}]: dispatch 2024-03-26T08:23:21.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:21 smithi043 ceph-mon[31929]: from='mgr.? 172.21.15.183:0/4019045375' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-26T08:23:21.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:21 smithi043 ceph-mon[31929]: from='mgr.? 172.21.15.183:0/4019045375' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T08:23:21.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:21 smithi043 ceph-mon[31929]: Standby manager daemon z started 2024-03-26T08:23:21.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:21 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2929002037' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28670673-9193-4fa3-9cc0-c1a5a6b27284"}]': finished 2024-03-26T08:23:21.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:21 smithi043 ceph-mon[31929]: from='mgr.? 172.21.15.183:0/4019045375' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-26T08:23:21.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:21 smithi043 ceph-mon[31929]: osdmap e5: 1 total, 0 up, 1 in 2024-03-26T08:23:21.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:21 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:23:21.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:21 smithi043 ceph-mon[31929]: from='mgr.? 172.21.15.183:0/4019045375' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T08:23:21.501 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:21 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y[36440]: 2024-03-26T08:23:21.252+0000 7fbd1684b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T08:23:21.824 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:21 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y[36440]: 2024-03-26T08:23:21.506+0000 7fbd1684b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T08:23:22.214 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:21 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y[36440]: 2024-03-26T08:23:21.894+0000 7fbd1684b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T08:23:22.214 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:22 smithi052 ceph-mon[35071]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:22.214 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:22 smithi052 ceph-mon[35071]: mgrmap e14: x(active, since 104s), standbys: z 2024-03-26T08:23:22.214 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:22 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-26T08:23:22.214 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:22 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/827263506' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:23:22.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:22 smithi183 ceph-mon[35818]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:22.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:22 smithi183 ceph-mon[35818]: mgrmap e14: x(active, since 104s), standbys: z 2024-03-26T08:23:22.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:22 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-26T08:23:22.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:22 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/827263506' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:23:22.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:22 smithi043 ceph-mon[31929]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:22.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:22 smithi043 ceph-mon[31929]: mgrmap e14: x(active, since 104s), standbys: z 2024-03-26T08:23:22.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:22 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-26T08:23:22.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:22 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/827263506' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:23:22.527 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:22 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y[36440]: 2024-03-26T08:23:22.219+0000 7fbd1684b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T08:23:22.824 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:22 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y[36440]: 2024-03-26T08:23:22.532+0000 7fbd1684b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T08:23:24.021 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:23 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y[36440]: 2024-03-26T08:23:23.601+0000 7fbd1684b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T08:23:24.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:24 smithi183 ceph-mon[35818]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:24.313 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:24 smithi052 ceph-mon[35071]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:24.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:24 smithi043 ceph-mon[31929]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:24.574 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:24 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y[36440]: 2024-03-26T08:23:24.318+0000 7fbd1684b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T08:23:24.574 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:24 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y[36440]: 2024-03-26T08:23:24.452+0000 7fbd1684b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T08:23:25.041 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:23:24 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y[36440]: 2024-03-26T08:23:24.745+0000 7fbd1684b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T08:23:25.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:25.063+0000 7f675a000700 1 -- 172.21.15.43:0/2746912977 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 15) v1 ==== 137234+0+0 (secure 0 0 0) 0x7f674c04ecd0 con 0x7f675c074bf0 2024-03-26T08:23:25.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:25 smithi183 ceph-mon[35818]: Standby manager daemon y started 2024-03-26T08:23:25.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:25 smithi183 ceph-mon[35818]: from='mgr.? 172.21.15.52:0/2595615499' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-26T08:23:25.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:25 smithi183 ceph-mon[35818]: from='mgr.? 172.21.15.52:0/2595615499' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T08:23:25.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:25 smithi183 ceph-mon[35818]: from='mgr.? 172.21.15.52:0/2595615499' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-26T08:23:25.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:25 smithi183 ceph-mon[35818]: from='mgr.? 172.21.15.52:0/2595615499' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T08:23:25.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:25 smithi052 ceph-mon[35071]: Standby manager daemon y started 2024-03-26T08:23:25.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:25 smithi052 ceph-mon[35071]: from='mgr.? 172.21.15.52:0/2595615499' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-26T08:23:25.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:25 smithi052 ceph-mon[35071]: from='mgr.? 172.21.15.52:0/2595615499' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T08:23:25.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:25 smithi052 ceph-mon[35071]: from='mgr.? 172.21.15.52:0/2595615499' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-26T08:23:25.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:25 smithi052 ceph-mon[35071]: from='mgr.? 172.21.15.52:0/2595615499' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T08:23:25.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:25 smithi043 ceph-mon[31929]: Standby manager daemon y started 2024-03-26T08:23:25.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:25 smithi043 ceph-mon[31929]: from='mgr.? 172.21.15.52:0/2595615499' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-26T08:23:25.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:25 smithi043 ceph-mon[31929]: from='mgr.? 172.21.15.52:0/2595615499' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T08:23:25.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:25 smithi043 ceph-mon[31929]: from='mgr.? 172.21.15.52:0/2595615499' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-26T08:23:25.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:25 smithi043 ceph-mon[31929]: from='mgr.? 172.21.15.52:0/2595615499' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T08:23:26.257 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:26 smithi043 ceph-mon[31929]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:26.257 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:26 smithi043 ceph-mon[31929]: mgrmap e15: x(active, since 108s), standbys: z, y 2024-03-26T08:23:26.257 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:26 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T08:23:26.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:26 smithi183 ceph-mon[35818]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:26.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:26 smithi183 ceph-mon[35818]: mgrmap e15: x(active, since 108s), standbys: z, y 2024-03-26T08:23:26.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:26 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T08:23:26.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:26 smithi052 ceph-mon[35071]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:26.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:26 smithi052 ceph-mon[35071]: mgrmap e15: x(active, since 108s), standbys: z, y 2024-03-26T08:23:26.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:26 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T08:23:28.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:28 smithi183 ceph-mon[35818]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:28.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:28 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:28.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:28 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:28.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:28 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:23:28.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:28 smithi052 ceph-mon[35071]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:28.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:28 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:28.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:28 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:28.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:28 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:23:28.341 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:28 smithi043 ceph-mon[31929]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:28.341 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:28 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:28.342 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:28 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:28.342 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:28 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:23:29.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:29 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:29.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:29 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:23:29.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:29 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:29.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:29 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:29.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:29 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:23:29.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:29 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:29.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:29 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:29.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:29 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:23:29.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:29 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:30.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:30 smithi043 ceph-mon[31929]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:30.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:30 smithi052 ceph-mon[35071]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:30.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:30 smithi183 ceph-mon[35818]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:31.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:31 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-26T08:23:31.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:31 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:31.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:31 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-26T08:23:31.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:31 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:31.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:31 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-26T08:23:31.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:31 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:32.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:32 smithi183 ceph-mon[35818]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:32.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:32 smithi183 ceph-mon[35818]: Deploying daemon osd.0 on smithi043 2024-03-26T08:23:32.318 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:32 smithi043 ceph-mon[31929]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:32.318 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:32 smithi043 ceph-mon[31929]: Deploying daemon osd.0 on smithi043 2024-03-26T08:23:32.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:32 smithi052 ceph-mon[35071]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:32.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:32 smithi052 ceph-mon[35071]: Deploying daemon osd.0 on smithi043 2024-03-26T08:23:34.211 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:34 smithi043 ceph-mon[31929]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:34.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:34 smithi052 ceph-mon[35071]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:34.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:34 smithi183 ceph-mon[35818]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:36.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:36 smithi043 ceph-mon[31929]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:36.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:36 smithi183 ceph-mon[35818]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:36.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:36 smithi052 ceph-mon[35071]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:38.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:38 smithi043 ceph-mon[31929]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:38.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:38 smithi183 ceph-mon[35818]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:38.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:38 smithi052 ceph-mon[35071]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:39.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:39 smithi052 ceph-mon[35071]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:39.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:39 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:39.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:39 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:39.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:39 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:23:39.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:39 smithi043 ceph-mon[31929]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:39.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:39 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:39.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:39 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:39.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:39 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:23:40.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:39 smithi183 ceph-mon[35818]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:40.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:39 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:40.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:39 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:40.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:39 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:23:41.677 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:41 smithi043 ceph-mon[31929]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:42.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:41 smithi183 ceph-mon[35818]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:42.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:41 smithi052 ceph-mon[35071]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:43.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:43 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:43.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:43 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:43.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:43 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:43.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:43 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:23:43.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:43 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:43.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:43 smithi183 ceph-mon[35818]: from='osd.0 [v2:172.21.15.43:6802/936601146,v1:172.21.15.43:6803/936601146]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-26T08:23:43.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:43 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:43.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:43 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:43.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:43 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:43.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:43 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:23:43.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:43 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:43.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:43 smithi052 ceph-mon[35071]: from='osd.0 [v2:172.21.15.43:6802/936601146,v1:172.21.15.43:6803/936601146]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-26T08:23:43.355 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:43 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:43.356 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:43 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:43.356 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:43 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:43.356 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:43 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:23:43.356 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:43 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:43.356 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:43 smithi043 ceph-mon[31929]: from='osd.0 [v2:172.21.15.43:6802/936601146,v1:172.21.15.43:6803/936601146]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-26T08:23:43.511 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 0 on host 'smithi043' 2024-03-26T08:23:43.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:43.516+0000 7f675a000700 1 -- 172.21.15.43:0/2746912977 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f675c0625d0 con 0x7f6740039200 2024-03-26T08:23:43.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:43.519+0000 7f6762200700 1 -- 172.21.15.43:0/2746912977 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6740039200 msgr2=0x7f674003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:23:43.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:43.519+0000 7f6762200700 1 --2- 172.21.15.43:0/2746912977 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6740039200 0x7f674003b6c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f675c075ef0 tx=0x7f6744009260 comp rx=0 tx=0).stop 2024-03-26T08:23:43.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:43.519+0000 7f6762200700 1 -- 172.21.15.43:0/2746912977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f675c074bf0 msgr2=0x7f675c19e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:23:43.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:43.519+0000 7f6762200700 1 --2- 172.21.15.43:0/2746912977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f675c074bf0 0x7f675c19e7c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f674c007f40 tx=0x7f674c00d3b0 comp rx=0 tx=0).stop 2024-03-26T08:23:43.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:43.520+0000 7f6762200700 1 -- 172.21.15.43:0/2746912977 shutdown_connections 2024-03-26T08:23:43.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:43.520+0000 7f6762200700 1 --2- 172.21.15.43:0/2746912977 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6740039200 0x7f674003b6c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:23:43.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:43.520+0000 7f6762200700 1 --2- 172.21.15.43:0/2746912977 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f675c10e6e0 0x7f675c1a5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:23:43.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:43.520+0000 7f6762200700 1 --2- 172.21.15.43:0/2746912977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f675c074bf0 0x7f675c19e7c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:23:43.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:43.520+0000 7f6762200700 1 --2- 172.21.15.43:0/2746912977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f675c0761c0 0x7f675c19ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:23:43.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:43.520+0000 7f6762200700 1 -- 172.21.15.43:0/2746912977 >> 172.21.15.43:0/2746912977 conn(0x7f675c0fd6a0 msgr2=0x7f675c101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:23:43.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:43.520+0000 7f6762200700 1 -- 172.21.15.43:0/2746912977 shutdown_connections 2024-03-26T08:23:43.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:23:43.520+0000 7f6762200700 1 -- 172.21.15.43:0/2746912977 wait complete. 2024-03-26T08:23:44.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:44 smithi043 ceph-mon[31929]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:44.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:44 smithi043 ceph-mon[31929]: from='osd.0 [v2:172.21.15.43:6802/936601146,v1:172.21.15.43:6803/936601146]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-26T08:23:44.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:44 smithi043 ceph-mon[31929]: osdmap e6: 1 total, 0 up, 1 in 2024-03-26T08:23:44.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:44 smithi043 ceph-mon[31929]: from='osd.0 [v2:172.21.15.43:6802/936601146,v1:172.21.15.43:6803/936601146]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-26T08:23:44.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:44 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:23:44.386 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:44 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:44.386 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:44 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:44.386 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:44 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:23:44.448 DEBUG:teuthology.orchestra.run.smithi043:osd.0> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.0.service 2024-03-26T08:23:44.459 INFO:tasks.cephadm:Deploying osd.1 on smithi052 with /dev/vg_nvme/lv_4... 2024-03-26T08:23:44.459 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-26T08:23:44.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:44 smithi183 ceph-mon[35818]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:44.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:44 smithi183 ceph-mon[35818]: from='osd.0 [v2:172.21.15.43:6802/936601146,v1:172.21.15.43:6803/936601146]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-26T08:23:44.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:44 smithi183 ceph-mon[35818]: osdmap e6: 1 total, 0 up, 1 in 2024-03-26T08:23:44.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:44 smithi183 ceph-mon[35818]: from='osd.0 [v2:172.21.15.43:6802/936601146,v1:172.21.15.43:6803/936601146]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-26T08:23:44.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:44 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:23:44.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:44 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:44.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:44 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:44.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:44 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:23:44.557 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:44 smithi052 ceph-mon[35071]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:44.557 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:44 smithi052 ceph-mon[35071]: from='osd.0 [v2:172.21.15.43:6802/936601146,v1:172.21.15.43:6803/936601146]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-26T08:23:44.557 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:44 smithi052 ceph-mon[35071]: osdmap e6: 1 total, 0 up, 1 in 2024-03-26T08:23:44.557 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:44 smithi052 ceph-mon[35071]: from='osd.0 [v2:172.21.15.43:6802/936601146,v1:172.21.15.43:6803/936601146]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-26T08:23:44.557 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:44 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:23:44.557 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:44 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:44.558 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:44 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:44.558 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:44 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:23:44.927 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T08:23:45.381 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:45 smithi052 ceph-mon[35071]: from='osd.0 [v2:172.21.15.43:6802/936601146,v1:172.21.15.43:6803/936601146]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-03-26T08:23:45.381 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:45 smithi052 ceph-mon[35071]: osdmap e7: 1 total, 0 up, 1 in 2024-03-26T08:23:45.381 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:45 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:23:45.381 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:45 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:23:45.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:45 smithi043 ceph-mon[31929]: from='osd.0 [v2:172.21.15.43:6802/936601146,v1:172.21.15.43:6803/936601146]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-03-26T08:23:45.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:45 smithi043 ceph-mon[31929]: osdmap e7: 1 total, 0 up, 1 in 2024-03-26T08:23:45.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:45 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:23:45.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:45 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:23:45.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:45 smithi183 ceph-mon[35818]: from='osd.0 [v2:172.21.15.43:6802/936601146,v1:172.21.15.43:6803/936601146]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-03-26T08:23:45.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:45 smithi183 ceph-mon[35818]: osdmap e7: 1 total, 0 up, 1 in 2024-03-26T08:23:45.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:45 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:23:45.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:45 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:23:46.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:46 smithi043 ceph-mon[31929]: purged_snaps scrub starts 2024-03-26T08:23:46.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:46 smithi043 ceph-mon[31929]: purged_snaps scrub ok 2024-03-26T08:23:46.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:46 smithi043 ceph-mon[31929]: pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:46.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:46 smithi043 ceph-mon[31929]: osd.0 [v2:172.21.15.43:6802/936601146,v1:172.21.15.43:6803/936601146] boot 2024-03-26T08:23:46.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:46 smithi043 ceph-mon[31929]: osdmap e8: 1 total, 1 up, 1 in 2024-03-26T08:23:46.281 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:46 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:23:46.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:46 smithi183 ceph-mon[35818]: purged_snaps scrub starts 2024-03-26T08:23:46.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:46 smithi183 ceph-mon[35818]: purged_snaps scrub ok 2024-03-26T08:23:46.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:46 smithi183 ceph-mon[35818]: pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:46.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:46 smithi183 ceph-mon[35818]: osd.0 [v2:172.21.15.43:6802/936601146,v1:172.21.15.43:6803/936601146] boot 2024-03-26T08:23:46.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:46 smithi183 ceph-mon[35818]: osdmap e8: 1 total, 1 up, 1 in 2024-03-26T08:23:46.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:46 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:23:46.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:46 smithi052 ceph-mon[35071]: purged_snaps scrub starts 2024-03-26T08:23:46.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:46 smithi052 ceph-mon[35071]: purged_snaps scrub ok 2024-03-26T08:23:46.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:46 smithi052 ceph-mon[35071]: pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:23:46.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:46 smithi052 ceph-mon[35071]: osd.0 [v2:172.21.15.43:6802/936601146,v1:172.21.15.43:6803/936601146] boot 2024-03-26T08:23:46.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:46 smithi052 ceph-mon[35071]: osdmap e8: 1 total, 1 up, 1 in 2024-03-26T08:23:46.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:46 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:23:47.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:47 smithi052 ceph-mon[35071]: osdmap e9: 1 total, 1 up, 1 in 2024-03-26T08:23:47.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:47 smithi043 ceph-mon[31929]: osdmap e9: 1 total, 1 up, 1 in 2024-03-26T08:23:47.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:47 smithi183 ceph-mon[35818]: osdmap e9: 1 total, 1 up, 1 in 2024-03-26T08:23:47.821 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-26T08:23:47.822 INFO:teuthology.orchestra.run.smithi052.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-26T08:23:47.822 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10+0 records in 2024-03-26T08:23:47.822 INFO:teuthology.orchestra.run.smithi052.stderr:10+0 records out 2024-03-26T08:23:47.822 INFO:teuthology.orchestra.run.smithi052.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0383838 s, 273 MB/s 2024-03-26T08:23:47.822 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping successful for: 2024-03-26T08:23:48.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:48 smithi043 ceph-mon[31929]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:23:48.531 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:48 smithi052 ceph-mon[35071]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:23:48.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:48 smithi183 ceph-mon[35818]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:23:48.612 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi052:vg_nvme/lv_4 2024-03-26T08:23:49.098 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T08:23:49.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:49 smithi043 ceph-mon[31929]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:23:49.435 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:49 smithi052 ceph-mon[35071]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:23:49.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:49 smithi183 ceph-mon[35818]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:23:50.313 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:50 smithi052 ceph-mon[35071]: Detected new or changed devices on smithi043 2024-03-26T08:23:50.313 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:50 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:50.313 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:50 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:50.313 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:50 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:23:50.314 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:50 smithi052 ceph-mon[35071]: Adjusting osd_memory_target on smithi043 to 17095M 2024-03-26T08:23:50.314 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:50 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:50.314 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:50 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:50.314 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:50 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:23:50.314 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:50 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:50.475 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:50 smithi183 ceph-mon[35818]: Detected new or changed devices on smithi043 2024-03-26T08:23:50.476 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:50 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:50.476 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:50 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:50.476 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:50 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:23:50.476 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:50 smithi183 ceph-mon[35818]: Adjusting osd_memory_target on smithi043 to 17095M 2024-03-26T08:23:50.476 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:50 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:50.476 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:50 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:50.476 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:50 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:23:50.476 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:50 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:50.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:50 smithi043 ceph-mon[31929]: Detected new or changed devices on smithi043 2024-03-26T08:23:50.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:50 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:50.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:50 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:50.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:50 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:23:50.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:50 smithi043 ceph-mon[31929]: Adjusting osd_memory_target on smithi043 to 17095M 2024-03-26T08:23:50.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:50 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:50.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:50 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:50.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:50 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:23:50.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:50 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:23:51.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.080+0000 7f7bfe200700 1 -- 172.21.15.52:0/3193362475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf8105d80 msgr2=0x7f7bf8106200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:23:51.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.080+0000 7f7bfe200700 1 --2- 172.21.15.52:0/3193362475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf8105d80 0x7f7bf8106200 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000b0a0 tx=0x7f7bf000b3b0 comp rx=0 tx=0).stop 2024-03-26T08:23:51.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.080+0000 7f7bfe200700 1 -- 172.21.15.52:0/3193362475 shutdown_connections 2024-03-26T08:23:51.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.080+0000 7f7bfe200700 1 --2- 172.21.15.52:0/3193362475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf8105d80 0x7f7bf8106200 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:23:51.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.080+0000 7f7bfe200700 1 --2- 172.21.15.52:0/3193362475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7bf8100a40 0x7f7bf8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:23:51.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.080+0000 7f7bfe200700 1 --2- 172.21.15.52:0/3193362475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bf81033b0 0x7f7bf8105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:23:51.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.080+0000 7f7bfe200700 1 -- 172.21.15.52:0/3193362475 >> 172.21.15.52:0/3193362475 conn(0x7f7bf80fc660 msgr2=0x7f7bf80feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:23:51.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.080+0000 7f7bfe200700 1 -- 172.21.15.52:0/3193362475 shutdown_connections 2024-03-26T08:23:51.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.081+0000 7f7bfe200700 1 -- 172.21.15.52:0/3193362475 wait complete. 2024-03-26T08:23:51.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.081+0000 7f7bfe200700 1 Processor -- start 2024-03-26T08:23:51.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.082+0000 7f7bfe200700 1 -- start start 2024-03-26T08:23:51.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.082+0000 7f7bfe200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bf8100a40 0x7f7bf819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:23:51.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.082+0000 7f7bfe200700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf81033b0 0x7f7bf819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:23:51.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.082+0000 7f7bfe200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7bf8105d80 0x7f7bf81a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:23:51.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.082+0000 7f7bfe200700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf8114200 con 0x7f7bf8100a40 2024-03-26T08:23:51.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.082+0000 7f7bfe200700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf8114080 con 0x7f7bf81033b0 2024-03-26T08:23:51.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.082+0000 7f7bfe200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf8114380 con 0x7f7bf8105d80 2024-03-26T08:23:51.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.082+0000 7f7bf6a00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf81033b0 0x7f7bf819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:23:51.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.082+0000 7f7bf6a00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf81033b0 0x7f7bf819ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46914/0 (socket says 172.21.15.52:46914) 2024-03-26T08:23:51.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.082+0000 7f7bf6a00700 1 -- 172.21.15.52:0/540844876 learned_addr learned my addr 172.21.15.52:0/540844876 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-26T08:23:51.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.083+0000 7f7bf7e00700 1 --2- 172.21.15.52:0/540844876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7bf8105d80 0x7f7bf81a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:23:51.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.083+0000 7f7bf7400700 1 --2- 172.21.15.52:0/540844876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bf8100a40 0x7f7bf819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:23:51.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.083+0000 7f7bf6a00700 1 -- 172.21.15.52:0/540844876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7bf8105d80 msgr2=0x7f7bf81a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:23:51.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.083+0000 7f7bf6a00700 1 --2- 172.21.15.52:0/540844876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7bf8105d80 0x7f7bf81a5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:23:51.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.083+0000 7f7bf6a00700 1 -- 172.21.15.52:0/540844876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bf8100a40 msgr2=0x7f7bf819e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:23:51.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.083+0000 7f7bf6a00700 1 --2- 172.21.15.52:0/540844876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bf8100a40 0x7f7bf819e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:23:51.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.083+0000 7f7bf6a00700 1 -- 172.21.15.52:0/540844876 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf81a6450 con 0x7f7bf81033b0 2024-03-26T08:23:51.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.083+0000 7f7bf6a00700 1 --2- 172.21.15.52:0/540844876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf81033b0 0x7f7bf819ed30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7be400c9b0 tx=0x7f7be400ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:23:51.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.083+0000 7f7befe00700 1 -- 172.21.15.52:0/540844876 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be4007b00 con 0x7f7bf81033b0 2024-03-26T08:23:51.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.084+0000 7f7befe00700 1 -- 172.21.15.52:0/540844876 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7be4007ca0 con 0x7f7bf81033b0 2024-03-26T08:23:51.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.084+0000 7f7bfe200700 1 -- 172.21.15.52:0/540844876 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf81a6740 con 0x7f7bf81033b0 2024-03-26T08:23:51.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.084+0000 7f7befe00700 1 -- 172.21.15.52:0/540844876 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be4018660 con 0x7f7bf81033b0 2024-03-26T08:23:51.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.084+0000 7f7bfe200700 1 -- 172.21.15.52:0/540844876 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf81a6c80 con 0x7f7bf81033b0 2024-03-26T08:23:51.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.085+0000 7f7bfe200700 1 -- 172.21.15.52:0/540844876 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf8068e50 con 0x7f7bf81033b0 2024-03-26T08:23:51.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.087+0000 7f7befe00700 1 -- 172.21.15.52:0/540844876 <== mon.2 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 15) v1 ==== 137234+0+0 (secure 0 0 0) 0x7f7be400f450 con 0x7f7bf81033b0 2024-03-26T08:23:51.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.088+0000 7f7befe00700 1 --2- 172.21.15.52:0/540844876 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f7bdc0a2880 0x7f7bdc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:23:51.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.088+0000 7f7befe00700 1 -- 172.21.15.52:0/540844876 <== mon.2 v2:172.21.15.52:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f7be40cd2b0 con 0x7f7bf81033b0 2024-03-26T08:23:51.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.089+0000 7f7bf7400700 1 --2- 172.21.15.52:0/540844876 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f7bdc0a2880 0x7f7bdc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:23:51.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.090+0000 7f7bf7400700 1 --2- 172.21.15.52:0/540844876 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f7bdc0a2880 0x7f7bdc0a4d40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7be8005950 tx=0x7f7be80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:23:51.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.092+0000 7f7befe00700 1 -- 172.21.15.52:0/540844876 <== mon.2 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7be4014030 con 0x7f7bf81033b0 2024-03-26T08:23:51.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:23:51.329+0000 7f7bfe200700 1 -- 172.21.15.52:0/540844876 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f7bf80625d0 con 0x7f7bdc0a2880 2024-03-26T08:23:51.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:51 smithi183 ceph-mon[35818]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:23:51.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:51 smithi052 ceph-mon[35071]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:23:51.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:51 smithi043 ceph-mon[31929]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:23:52.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:52 smithi183 ceph-mon[35818]: from='client.34139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:23:52.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:52 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:23:52.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:52 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:23:52.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:52 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:52.570 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:52 smithi052 ceph-mon[35071]: from='client.34139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:23:52.570 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:23:52.570 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:23:52.570 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:52 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:52.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:52 smithi043 ceph-mon[31929]: from='client.34139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:23:52.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:52 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:23:52.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:52 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:23:52.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:52 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:23:53.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:53 smithi183 ceph-mon[35818]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:23:53.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:53 smithi052 ceph-mon[35071]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:23:53.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:53 smithi043 ceph-mon[31929]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:23:55.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:55 smithi183 ceph-mon[35818]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:23:55.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:55 smithi183 ceph-mon[35818]: from='client.? 172.21.15.52:0/2611854064' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa8ad6b6-277a-42b8-b0f8-b36fc219de2f"}]: dispatch 2024-03-26T08:23:55.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:55 smithi183 ceph-mon[35818]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa8ad6b6-277a-42b8-b0f8-b36fc219de2f"}]: dispatch 2024-03-26T08:23:55.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:55 smithi183 ceph-mon[35818]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa8ad6b6-277a-42b8-b0f8-b36fc219de2f"}]': finished 2024-03-26T08:23:55.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:55 smithi183 ceph-mon[35818]: osdmap e10: 2 total, 1 up, 2 in 2024-03-26T08:23:55.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:55 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:23:55.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:55 smithi052 ceph-mon[35071]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:23:55.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:55 smithi052 ceph-mon[35071]: from='client.? 172.21.15.52:0/2611854064' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa8ad6b6-277a-42b8-b0f8-b36fc219de2f"}]: dispatch 2024-03-26T08:23:55.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:55 smithi052 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa8ad6b6-277a-42b8-b0f8-b36fc219de2f"}]: dispatch 2024-03-26T08:23:55.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:55 smithi052 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa8ad6b6-277a-42b8-b0f8-b36fc219de2f"}]': finished 2024-03-26T08:23:55.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:55 smithi052 ceph-mon[35071]: osdmap e10: 2 total, 1 up, 2 in 2024-03-26T08:23:55.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:55 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:23:55.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:55 smithi043 ceph-mon[31929]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:23:55.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:55 smithi043 ceph-mon[31929]: from='client.? 172.21.15.52:0/2611854064' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa8ad6b6-277a-42b8-b0f8-b36fc219de2f"}]: dispatch 2024-03-26T08:23:55.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:55 smithi043 ceph-mon[31929]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa8ad6b6-277a-42b8-b0f8-b36fc219de2f"}]: dispatch 2024-03-26T08:23:55.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:55 smithi043 ceph-mon[31929]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa8ad6b6-277a-42b8-b0f8-b36fc219de2f"}]': finished 2024-03-26T08:23:55.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:55 smithi043 ceph-mon[31929]: osdmap e10: 2 total, 1 up, 2 in 2024-03-26T08:23:55.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:55 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:23:56.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:56 smithi183 ceph-mon[35818]: from='client.? 172.21.15.52:0/3942539264' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:23:56.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:56 smithi052 ceph-mon[35071]: from='client.? 172.21.15.52:0/3942539264' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:23:56.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:56 smithi043 ceph-mon[31929]: from='client.? 172.21.15.52:0/3942539264' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:23:57.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:57 smithi052 ceph-mon[35071]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:23:57.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:57 smithi043 ceph-mon[31929]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:23:57.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:57 smithi183 ceph-mon[35818]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:23:59.542 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:23:59 smithi052 ceph-mon[35071]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:23:59.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:23:59 smithi043 ceph-mon[31929]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:23:59.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:23:59 smithi183 ceph-mon[35818]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:01.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:01 smithi052 ceph-mon[35071]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:01.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:01 smithi043 ceph-mon[31929]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:01.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:01 smithi183 ceph-mon[35818]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:03.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:03 smithi052 ceph-mon[35071]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:03.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:03 smithi043 ceph-mon[31929]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:03.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:03 smithi183 ceph-mon[35818]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:04.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:04 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-26T08:24:04.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:04 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:24:04.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:04 smithi043 ceph-mon[31929]: Deploying daemon osd.1 on smithi052 2024-03-26T08:24:04.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:04 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-26T08:24:04.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:04 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:24:04.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:04 smithi183 ceph-mon[35818]: Deploying daemon osd.1 on smithi052 2024-03-26T08:24:04.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:04 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-26T08:24:04.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:04 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:24:04.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:04 smithi052 ceph-mon[35071]: Deploying daemon osd.1 on smithi052 2024-03-26T08:24:05.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:05 smithi043 ceph-mon[31929]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:05.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:05 smithi052 ceph-mon[35071]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:05.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:05 smithi183 ceph-mon[35818]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:07.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:07 smithi052 ceph-mon[35071]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:07.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:07 smithi043 ceph-mon[31929]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:07.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:07 smithi183 ceph-mon[35818]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:09.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:09 smithi052 ceph-mon[35071]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:09.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:09 smithi043 ceph-mon[31929]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:09.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:09 smithi183 ceph-mon[35818]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:11.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:11 smithi043 ceph-mon[31929]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:11.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:11 smithi183 ceph-mon[35818]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:11.812 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:11 smithi052 ceph-mon[35071]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:14.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:13 smithi183 ceph-mon[35818]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:14.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:13 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:14.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:13 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:24:14.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:13 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:14.062 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:13 smithi052 ceph-mon[35071]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:14.063 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:13 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:14.063 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:13 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:24:14.063 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:13 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:14.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:13 smithi043 ceph-mon[31929]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:14.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:13 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:14.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:13 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:24:14.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:13 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:15.971 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:15 smithi052 ceph-mon[35071]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:16.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:15 smithi183 ceph-mon[35818]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:16.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:15 smithi043 ceph-mon[31929]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:17.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:17 smithi183 ceph-mon[35818]: pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:17.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:17 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:17.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:17 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:17.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:17 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:24:17.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:17 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:24:17.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:17 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:17.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:17 smithi183 ceph-mon[35818]: from='osd.1 [v2:172.21.15.52:6800/3268412328,v1:172.21.15.52:6801/3268412328]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T08:24:17.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:17 smithi183 ceph-mon[35818]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T08:24:17.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:17 smithi052 ceph-mon[35071]: pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:17.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:17 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:17.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:17 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:17.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:17 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:24:17.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:17 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:24:17.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:17 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:17.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:17 smithi052 ceph-mon[35071]: from='osd.1 [v2:172.21.15.52:6800/3268412328,v1:172.21.15.52:6801/3268412328]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T08:24:17.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:17 smithi052 ceph-mon[35071]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T08:24:17.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:17 smithi043 ceph-mon[31929]: pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:17.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:17 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:17.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:17 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:17.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:17 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:24:17.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:17 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:24:17.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:17 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:17.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:17 smithi043 ceph-mon[31929]: from='osd.1 [v2:172.21.15.52:6800/3268412328,v1:172.21.15.52:6801/3268412328]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T08:24:17.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:17 smithi043 ceph-mon[31929]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T08:24:17.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:24:17.899+0000 7f7befe00700 1 -- 172.21.15.52:0/540844876 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7bf80625d0 con 0x7f7bdc0a2880 2024-03-26T08:24:17.894 INFO:teuthology.orchestra.run.smithi052.stdout:Created osd(s) 1 on host 'smithi052' 2024-03-26T08:24:17.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:24:17.903+0000 7f7bfe200700 1 -- 172.21.15.52:0/540844876 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f7bdc0a2880 msgr2=0x7f7bdc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:24:17.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:24:17.903+0000 7f7bfe200700 1 --2- 172.21.15.52:0/540844876 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f7bdc0a2880 0x7f7bdc0a4d40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7be8005950 tx=0x7f7be80058e0 comp rx=0 tx=0).stop 2024-03-26T08:24:17.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:24:17.903+0000 7f7bfe200700 1 -- 172.21.15.52:0/540844876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf81033b0 msgr2=0x7f7bf819ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:24:17.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:24:17.903+0000 7f7bfe200700 1 --2- 172.21.15.52:0/540844876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf81033b0 0x7f7bf819ed30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7be400c9b0 tx=0x7f7be400ccc0 comp rx=0 tx=0).stop 2024-03-26T08:24:17.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:24:17.903+0000 7f7bfe200700 1 -- 172.21.15.52:0/540844876 shutdown_connections 2024-03-26T08:24:17.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:24:17.903+0000 7f7bfe200700 1 --2- 172.21.15.52:0/540844876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7bf8105d80 0x7f7bf81a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:24:17.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:24:17.903+0000 7f7bfe200700 1 --2- 172.21.15.52:0/540844876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bf8100a40 0x7f7bf819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:24:17.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:24:17.903+0000 7f7bfe200700 1 --2- 172.21.15.52:0/540844876 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f7bdc0a2880 0x7f7bdc0a4d40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:24:17.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:24:17.903+0000 7f7bfe200700 1 --2- 172.21.15.52:0/540844876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf81033b0 0x7f7bf819ed30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:24:17.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:24:17.903+0000 7f7bfe200700 1 -- 172.21.15.52:0/540844876 >> 172.21.15.52:0/540844876 conn(0x7f7bf80fc660 msgr2=0x7f7bf81016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:24:17.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:24:17.903+0000 7f7bfe200700 1 -- 172.21.15.52:0/540844876 shutdown_connections 2024-03-26T08:24:17.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:24:17.903+0000 7f7bfe200700 1 -- 172.21.15.52:0/540844876 wait complete. 2024-03-26T08:24:18.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:18 smithi183 ceph-mon[35818]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-26T08:24:18.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:18 smithi183 ceph-mon[35818]: osdmap e11: 2 total, 1 up, 2 in 2024-03-26T08:24:18.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:18 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:24:18.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:18 smithi183 ceph-mon[35818]: from='osd.1 [v2:172.21.15.52:6800/3268412328,v1:172.21.15.52:6801/3268412328]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-26T08:24:18.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:18 smithi183 ceph-mon[35818]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-26T08:24:18.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:18 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:18.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:18 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:18.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:18 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:24:18.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:18 smithi052 ceph-mon[35071]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-26T08:24:18.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:18 smithi052 ceph-mon[35071]: osdmap e11: 2 total, 1 up, 2 in 2024-03-26T08:24:18.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:18 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:24:18.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:18 smithi052 ceph-mon[35071]: from='osd.1 [v2:172.21.15.52:6800/3268412328,v1:172.21.15.52:6801/3268412328]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-26T08:24:18.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:18 smithi052 ceph-mon[35071]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-26T08:24:18.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:18 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:18.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:18 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:18.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:18 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:24:18.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:18 smithi043 ceph-mon[31929]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-26T08:24:18.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:18 smithi043 ceph-mon[31929]: osdmap e11: 2 total, 1 up, 2 in 2024-03-26T08:24:18.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:18 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:24:18.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:18 smithi043 ceph-mon[31929]: from='osd.1 [v2:172.21.15.52:6800/3268412328,v1:172.21.15.52:6801/3268412328]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-26T08:24:18.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:18 smithi043 ceph-mon[31929]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-26T08:24:18.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:18 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:18.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:18 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:18.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:18 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:24:18.832 DEBUG:teuthology.orchestra.run.smithi052:osd.1> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.1.service 2024-03-26T08:24:18.839 INFO:tasks.cephadm:Deploying osd.2 on smithi183 with /dev/vg_nvme/lv_4... 2024-03-26T08:24:18.839 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-26T08:24:19.295 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:24:19.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:19 smithi183 ceph-mon[35818]: pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:19.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:19 smithi183 ceph-mon[35818]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-03-26T08:24:19.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:19 smithi183 ceph-mon[35818]: osdmap e12: 2 total, 1 up, 2 in 2024-03-26T08:24:19.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:19 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:24:19.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:19 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:24:19.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:19 smithi052 ceph-mon[35071]: pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:19.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:19 smithi052 ceph-mon[35071]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-03-26T08:24:19.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:19 smithi052 ceph-mon[35071]: osdmap e12: 2 total, 1 up, 2 in 2024-03-26T08:24:19.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:19 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:24:19.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:19 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:24:19.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:19 smithi043 ceph-mon[31929]: pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:24:19.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:19 smithi043 ceph-mon[31929]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-03-26T08:24:19.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:19 smithi043 ceph-mon[31929]: osdmap e12: 2 total, 1 up, 2 in 2024-03-26T08:24:19.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:19 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:24:19.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:19 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:24:20.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:20 smithi052 ceph-mon[35071]: purged_snaps scrub starts 2024-03-26T08:24:20.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:20 smithi052 ceph-mon[35071]: purged_snaps scrub ok 2024-03-26T08:24:20.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:20 smithi052 ceph-mon[35071]: osd.1 [v2:172.21.15.52:6800/3268412328,v1:172.21.15.52:6801/3268412328] boot 2024-03-26T08:24:20.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:20 smithi052 ceph-mon[35071]: osdmap e13: 2 total, 2 up, 2 in 2024-03-26T08:24:20.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:20 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:24:20.462 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:20 smithi183 ceph-mon[35818]: purged_snaps scrub starts 2024-03-26T08:24:20.463 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:20 smithi183 ceph-mon[35818]: purged_snaps scrub ok 2024-03-26T08:24:20.463 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:20 smithi183 ceph-mon[35818]: osd.1 [v2:172.21.15.52:6800/3268412328,v1:172.21.15.52:6801/3268412328] boot 2024-03-26T08:24:20.463 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:20 smithi183 ceph-mon[35818]: osdmap e13: 2 total, 2 up, 2 in 2024-03-26T08:24:20.463 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:20 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:24:20.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:20 smithi043 ceph-mon[31929]: purged_snaps scrub starts 2024-03-26T08:24:20.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:20 smithi043 ceph-mon[31929]: purged_snaps scrub ok 2024-03-26T08:24:20.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:20 smithi043 ceph-mon[31929]: osd.1 [v2:172.21.15.52:6800/3268412328,v1:172.21.15.52:6801/3268412328] boot 2024-03-26T08:24:20.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:20 smithi043 ceph-mon[31929]: osdmap e13: 2 total, 2 up, 2 in 2024-03-26T08:24:20.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:20 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:24:21.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:21 smithi183 ceph-mon[35818]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:21.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:21 smithi183 ceph-mon[35818]: osdmap e14: 2 total, 2 up, 2 in 2024-03-26T08:24:21.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:21 smithi052 ceph-mon[35071]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:21.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:21 smithi052 ceph-mon[35071]: osdmap e14: 2 total, 2 up, 2 in 2024-03-26T08:24:21.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:21 smithi043 ceph-mon[31929]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:21.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:21 smithi043 ceph-mon[31929]: osdmap e14: 2 total, 2 up, 2 in 2024-03-26T08:24:21.895 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-26T08:24:21.895 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-26T08:24:21.895 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-03-26T08:24:21.895 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-03-26T08:24:21.895 INFO:teuthology.orchestra.run.smithi183.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0329767 s, 318 MB/s 2024-03-26T08:24:21.895 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 2024-03-26T08:24:21.895 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-03-26T08:24:22.700 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi183:vg_nvme/lv_4 2024-03-26T08:24:23.149 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:24:23.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:23 smithi183 ceph-mon[35818]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:23.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:23 smithi052 ceph-mon[35071]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:23.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:23 smithi043 ceph-mon[31929]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:24.913 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:24 smithi183 ceph-mon[35818]: Detected new or changed devices on smithi052 2024-03-26T08:24:24.913 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:24 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:24.913 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:24 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:24.913 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:24 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:24:24.914 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:24 smithi183 ceph-mon[35818]: Adjusting osd_memory_target on smithi052 to 17095M 2024-03-26T08:24:24.914 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:24 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:24.914 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:24 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:24:24.914 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:24 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:24:24.914 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:24 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:25.079 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.085+0000 7f6a89c00700 1 -- 172.21.15.183:0/679227047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a84104410 msgr2=0x7f6a84104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:24:25.079 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.085+0000 7f6a89c00700 1 --2- 172.21.15.183:0/679227047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a84104410 0x7f6a84104830 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6a74009990 tx=0x7f6a74009ca0 comp rx=0 tx=0).stop 2024-03-26T08:24:25.080 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.085+0000 7f6a89c00700 1 -- 172.21.15.183:0/679227047 shutdown_connections 2024-03-26T08:24:25.080 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.085+0000 7f6a89c00700 1 --2- 172.21.15.183:0/679227047 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a84105fb0 0x7f6a8410c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:24:25.080 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.085+0000 7f6a89c00700 1 --2- 172.21.15.183:0/679227047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a84104410 0x7f6a84104830 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:24:25.080 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.085+0000 7f6a89c00700 1 --2- 172.21.15.183:0/679227047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a84105610 0x7f6a84105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:24:25.080 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.085+0000 7f6a89c00700 1 -- 172.21.15.183:0/679227047 >> 172.21.15.183:0/679227047 conn(0x7f6a840ff9b0 msgr2=0x7f6a84101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:24:25.080 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.086+0000 7f6a89c00700 1 -- 172.21.15.183:0/679227047 shutdown_connections 2024-03-26T08:24:25.080 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.086+0000 7f6a89c00700 1 -- 172.21.15.183:0/679227047 wait complete. 2024-03-26T08:24:25.080 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.087+0000 7f6a89c00700 1 Processor -- start 2024-03-26T08:24:25.081 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.087+0000 7f6a89c00700 1 -- start start 2024-03-26T08:24:25.081 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.087+0000 7f6a89c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a84104410 0x7f6a8419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:24:25.081 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.087+0000 7f6a89c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a84105610 0x7f6a8419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:24:25.082 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.087+0000 7f6a89c00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a84105fb0 0x7f6a841a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:24:25.082 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.087+0000 7f6a89c00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8410ff00 con 0x7f6a84105610 2024-03-26T08:24:25.082 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.087+0000 7f6a89c00700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8410fd80 con 0x7f6a84105fb0 2024-03-26T08:24:25.082 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.087+0000 7f6a89c00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a84110080 con 0x7f6a84104410 2024-03-26T08:24:25.082 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.087+0000 7f6a82a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a84104410 0x7f6a8419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:24:25.082 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.088+0000 7f6a82a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a84104410 0x7f6a8419a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:56648/0 (socket says 172.21.15.183:56648) 2024-03-26T08:24:25.082 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.088+0000 7f6a82a00700 1 -- 172.21.15.183:0/3459403937 learned_addr learned my addr 172.21.15.183:0/3459403937 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-03-26T08:24:25.082 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.088+0000 7f6a82000700 1 --2- 172.21.15.183:0/3459403937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a84105610 0x7f6a8419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:24:25.082 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.088+0000 7f6a83400700 1 --2- 172.21.15.183:0/3459403937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a84105fb0 0x7f6a841a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:24:25.082 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.088+0000 7f6a82a00700 1 -- 172.21.15.183:0/3459403937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a84105fb0 msgr2=0x7f6a841a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:24:25.083 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.088+0000 7f6a82a00700 1 --2- 172.21.15.183:0/3459403937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a84105fb0 0x7f6a841a1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:24:25.083 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.088+0000 7f6a82a00700 1 -- 172.21.15.183:0/3459403937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a84105610 msgr2=0x7f6a8419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:24:25.083 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.088+0000 7f6a82a00700 1 --2- 172.21.15.183:0/3459403937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a84105610 0x7f6a8419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:24:25.083 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.088+0000 7f6a82a00700 1 -- 172.21.15.183:0/3459403937 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a841a21d0 con 0x7f6a84104410 2024-03-26T08:24:25.083 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.088+0000 7f6a82000700 1 --2- 172.21.15.183:0/3459403937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a84105610 0x7f6a8419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:24:25.083 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.088+0000 7f6a82a00700 1 --2- 172.21.15.183:0/3459403937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a84104410 0x7f6a8419a5c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6a740047b0 tx=0x7f6a74005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:24:25.083 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.089+0000 7f6a7b400700 1 -- 172.21.15.183:0/3459403937 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7400fc70 con 0x7f6a84104410 2024-03-26T08:24:25.084 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.089+0000 7f6a89c00700 1 -- 172.21.15.183:0/3459403937 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a841a2460 con 0x7f6a84104410 2024-03-26T08:24:25.084 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.089+0000 7f6a7b400700 1 -- 172.21.15.183:0/3459403937 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6a7400fe10 con 0x7f6a84104410 2024-03-26T08:24:25.084 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.089+0000 7f6a7b400700 1 -- 172.21.15.183:0/3459403937 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a740173f0 con 0x7f6a84104410 2024-03-26T08:24:25.084 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.089+0000 7f6a89c00700 1 -- 172.21.15.183:0/3459403937 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a841a28b0 con 0x7f6a84104410 2024-03-26T08:24:25.084 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.090+0000 7f6a89c00700 1 -- 172.21.15.183:0/3459403937 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a84068e50 con 0x7f6a84104410 2024-03-26T08:24:25.086 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.092+0000 7f6a7b400700 1 -- 172.21.15.183:0/3459403937 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 15) v1 ==== 137234+0+0 (secure 0 0 0) 0x7f6a74028060 con 0x7f6a84104410 2024-03-26T08:24:25.087 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.093+0000 7f6a7b400700 1 --2- 172.21.15.183:0/3459403937 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6a680a2800 0x7f6a680a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:24:25.087 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.094+0000 7f6a7b400700 1 -- 172.21.15.183:0/3459403937 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f6a740ce220 con 0x7f6a84104410 2024-03-26T08:24:25.087 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.094+0000 7f6a82000700 1 --2- 172.21.15.183:0/3459403937 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6a680a2800 0x7f6a680a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:24:25.089 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.095+0000 7f6a82000700 1 --2- 172.21.15.183:0/3459403937 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6a680a2800 0x7f6a680a4cc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009cb0 tx=0x7f6a70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:24:25.090 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.097+0000 7f6a7b400700 1 -- 172.21.15.183:0/3459403937 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6a7405c740 con 0x7f6a84104410 2024-03-26T08:24:25.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:24 smithi043 ceph-mon[31929]: Detected new or changed devices on smithi052 2024-03-26T08:24:25.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:24 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:25.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:24 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:25.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:24 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:24:25.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:24 smithi043 ceph-mon[31929]: Adjusting osd_memory_target on smithi052 to 17095M 2024-03-26T08:24:25.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:24 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:25.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:24 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:24:25.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:24 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:24:25.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:24 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:25.322 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:25.328+0000 7f6a89c00700 1 -- 172.21.15.183:0/3459403937 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a840625d0 con 0x7f6a680a2800 2024-03-26T08:24:25.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:24 smithi052 ceph-mon[35071]: Detected new or changed devices on smithi052 2024-03-26T08:24:25.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:24 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:25.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:24 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:25.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:24 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:24:25.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:24 smithi052 ceph-mon[35071]: Adjusting osd_memory_target on smithi052 to 17095M 2024-03-26T08:24:25.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:24 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:25.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:24 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:24:25.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:24 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:24:25.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:24 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:26.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:25 smithi043 ceph-mon[31929]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:26.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:25 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:24:26.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:25 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:24:26.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:25 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:24:26.235 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:25 smithi183 ceph-mon[35818]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:26.235 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:25 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:24:26.235 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:25 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:24:26.235 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:25 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:24:26.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:25 smithi052 ceph-mon[35071]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:26.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:25 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:24:26.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:25 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:24:26.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:25 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:24:27.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:26 smithi043 ceph-mon[31929]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:24:27.207 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:26 smithi183 ceph-mon[35818]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:24:27.235 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:26 smithi052 ceph-mon[35071]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:24:28.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:27 smithi183 ceph-mon[35818]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:28.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:27 smithi043 ceph-mon[31929]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:28.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:27 smithi052 ceph-mon[35071]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:30.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:29 smithi183 ceph-mon[35818]: pgmap v90: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:30.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:29 smithi183 ceph-mon[35818]: from='client.? 172.21.15.183:0/1328095261' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec0039c3-f9bd-419a-81de-fa4418359808"}]: dispatch 2024-03-26T08:24:30.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:29 smithi183 ceph-mon[35818]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec0039c3-f9bd-419a-81de-fa4418359808"}]: dispatch 2024-03-26T08:24:30.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:29 smithi183 ceph-mon[35818]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec0039c3-f9bd-419a-81de-fa4418359808"}]': finished 2024-03-26T08:24:30.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:29 smithi183 ceph-mon[35818]: osdmap e15: 3 total, 2 up, 3 in 2024-03-26T08:24:30.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:29 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:24:30.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:29 smithi183 ceph-mon[35818]: from='client.? 172.21.15.183:0/3267588158' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:24:30.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:29 smithi052 ceph-mon[35071]: pgmap v90: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:30.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:29 smithi052 ceph-mon[35071]: from='client.? 172.21.15.183:0/1328095261' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec0039c3-f9bd-419a-81de-fa4418359808"}]: dispatch 2024-03-26T08:24:30.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:29 smithi052 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec0039c3-f9bd-419a-81de-fa4418359808"}]: dispatch 2024-03-26T08:24:30.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:29 smithi052 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec0039c3-f9bd-419a-81de-fa4418359808"}]': finished 2024-03-26T08:24:30.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:29 smithi052 ceph-mon[35071]: osdmap e15: 3 total, 2 up, 3 in 2024-03-26T08:24:30.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:29 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:24:30.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:29 smithi052 ceph-mon[35071]: from='client.? 172.21.15.183:0/3267588158' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:24:30.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:29 smithi043 ceph-mon[31929]: pgmap v90: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:30.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:29 smithi043 ceph-mon[31929]: from='client.? 172.21.15.183:0/1328095261' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec0039c3-f9bd-419a-81de-fa4418359808"}]: dispatch 2024-03-26T08:24:30.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:29 smithi043 ceph-mon[31929]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec0039c3-f9bd-419a-81de-fa4418359808"}]: dispatch 2024-03-26T08:24:30.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:29 smithi043 ceph-mon[31929]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec0039c3-f9bd-419a-81de-fa4418359808"}]': finished 2024-03-26T08:24:30.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:29 smithi043 ceph-mon[31929]: osdmap e15: 3 total, 2 up, 3 in 2024-03-26T08:24:30.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:29 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:24:30.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:29 smithi043 ceph-mon[31929]: from='client.? 172.21.15.183:0/3267588158' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:24:32.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:31 smithi183 ceph-mon[35818]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:32.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:31 smithi052 ceph-mon[35071]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:32.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:31 smithi043 ceph-mon[31929]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:34.206 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:33 smithi043 ceph-mon[31929]: pgmap v93: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:34.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:33 smithi183 ceph-mon[35818]: pgmap v93: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:34.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:33 smithi052 ceph-mon[35071]: pgmap v93: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:36.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:35 smithi183 ceph-mon[35818]: pgmap v94: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:36.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:35 smithi052 ceph-mon[35071]: pgmap v94: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:36.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:35 smithi043 ceph-mon[31929]: pgmap v94: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:38.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:37 smithi183 ceph-mon[35818]: pgmap v95: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:38.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:37 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-26T08:24:38.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:37 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:24:38.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:37 smithi052 ceph-mon[35071]: pgmap v95: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:38.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:37 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-26T08:24:38.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:37 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:24:38.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:37 smithi043 ceph-mon[31929]: pgmap v95: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:38.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:37 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-26T08:24:38.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:37 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:24:39.202 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:38 smithi183 ceph-mon[35818]: Deploying daemon osd.2 on smithi183 2024-03-26T08:24:39.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:38 smithi052 ceph-mon[35071]: Deploying daemon osd.2 on smithi183 2024-03-26T08:24:39.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:38 smithi043 ceph-mon[31929]: Deploying daemon osd.2 on smithi183 2024-03-26T08:24:40.219 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:39 smithi183 ceph-mon[35818]: pgmap v96: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:40.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:39 smithi052 ceph-mon[35071]: pgmap v96: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:40.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:39 smithi043 ceph-mon[31929]: pgmap v96: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:42.226 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:41 smithi052 ceph-mon[35071]: pgmap v97: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:42.270 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:41 smithi183 ceph-mon[35818]: pgmap v97: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:42.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:41 smithi043 ceph-mon[31929]: pgmap v97: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:44.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:43 smithi183 ceph-mon[35818]: pgmap v98: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:44.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:43 smithi052 ceph-mon[35071]: pgmap v98: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:44.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:43 smithi043 ceph-mon[31929]: pgmap v98: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:46.254 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:45 smithi183 ceph-mon[35818]: pgmap v99: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:46.254 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:45 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:46.254 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:45 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:24:46.254 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:45 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:46.305 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:45 smithi043 ceph-mon[31929]: pgmap v99: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:46.305 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:45 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:46.305 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:45 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:24:46.305 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:45 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:46.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:45 smithi052 ceph-mon[35071]: pgmap v99: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:46.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:45 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:46.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:45 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:24:46.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:45 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:48.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:47 smithi183 ceph-mon[35818]: pgmap v100: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:48.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:47 smithi052 ceph-mon[35071]: pgmap v100: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:48.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:47 smithi043 ceph-mon[31929]: pgmap v100: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:49.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:49 smithi183 ceph-mon[35818]: pgmap v101: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:49.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:49 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:49.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:49 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:49.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:49 smithi183 ceph-mon[35818]: from='osd.2 [v2:172.21.15.183:6800/2103180908,v1:172.21.15.183:6801/2103180908]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T08:24:49.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:49 smithi183 ceph-mon[35818]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T08:24:49.936 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:49.940+0000 7f6a7b400700 1 -- 172.21.15.183:0/3459403937 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6a840625d0 con 0x7f6a680a2800 2024-03-26T08:24:49.936 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 2 on host 'smithi183' 2024-03-26T08:24:49.939 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:49.944+0000 7f6a89c00700 1 -- 172.21.15.183:0/3459403937 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6a680a2800 msgr2=0x7f6a680a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:24:49.939 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:49.944+0000 7f6a89c00700 1 --2- 172.21.15.183:0/3459403937 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6a680a2800 0x7f6a680a4cc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009cb0 tx=0x7f6a70009260 comp rx=0 tx=0).stop 2024-03-26T08:24:49.939 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:49.944+0000 7f6a89c00700 1 -- 172.21.15.183:0/3459403937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a84104410 msgr2=0x7f6a8419a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:24:49.939 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:49.944+0000 7f6a89c00700 1 --2- 172.21.15.183:0/3459403937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a84104410 0x7f6a8419a5c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6a740047b0 tx=0x7f6a74005e80 comp rx=0 tx=0).stop 2024-03-26T08:24:49.939 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:49.944+0000 7f6a89c00700 1 -- 172.21.15.183:0/3459403937 shutdown_connections 2024-03-26T08:24:49.939 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:49.944+0000 7f6a89c00700 1 --2- 172.21.15.183:0/3459403937 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6a680a2800 0x7f6a680a4cc0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:24:49.939 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:49.944+0000 7f6a89c00700 1 --2- 172.21.15.183:0/3459403937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a84105fb0 0x7f6a841a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:24:49.939 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:49.944+0000 7f6a89c00700 1 --2- 172.21.15.183:0/3459403937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a84104410 0x7f6a8419a5c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:24:49.939 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:49.944+0000 7f6a89c00700 1 --2- 172.21.15.183:0/3459403937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a84105610 0x7f6a8419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:24:49.939 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:49.944+0000 7f6a89c00700 1 -- 172.21.15.183:0/3459403937 >> 172.21.15.183:0/3459403937 conn(0x7f6a840ff9b0 msgr2=0x7f6a84108910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:24:49.940 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:49.944+0000 7f6a89c00700 1 -- 172.21.15.183:0/3459403937 shutdown_connections 2024-03-26T08:24:49.940 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:24:49.944+0000 7f6a89c00700 1 -- 172.21.15.183:0/3459403937 wait complete. 2024-03-26T08:24:50.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:49 smithi052 ceph-mon[35071]: pgmap v101: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:50.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:49 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:50.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:49 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:50.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:49 smithi052 ceph-mon[35071]: from='osd.2 [v2:172.21.15.183:6800/2103180908,v1:172.21.15.183:6801/2103180908]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T08:24:50.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:49 smithi052 ceph-mon[35071]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T08:24:50.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:49 smithi043 ceph-mon[31929]: pgmap v101: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:50.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:49 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:50.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:49 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:50.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:49 smithi043 ceph-mon[31929]: from='osd.2 [v2:172.21.15.183:6800/2103180908,v1:172.21.15.183:6801/2103180908]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T08:24:50.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:49 smithi043 ceph-mon[31929]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T08:24:50.725 DEBUG:teuthology.orchestra.run.smithi183:osd.2> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.2.service 2024-03-26T08:24:50.731 INFO:tasks.cephadm:Deploying osd.3 on smithi043 with /dev/vg_nvme/lv_3... 2024-03-26T08:24:50.732 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-26T08:24:51.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:50 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:24:51.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:50 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:24:51.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:50 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:51.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:50 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:51.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:50 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:51.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:50 smithi183 ceph-mon[35818]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-26T08:24:51.051 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:50 smithi183 ceph-mon[35818]: from='osd.2 [v2:172.21.15.183:6800/2103180908,v1:172.21.15.183:6801/2103180908]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-26T08:24:51.051 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:50 smithi183 ceph-mon[35818]: osdmap e16: 3 total, 2 up, 3 in 2024-03-26T08:24:51.051 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:50 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:24:51.051 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:50 smithi183 ceph-mon[35818]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-26T08:24:51.051 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:50 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:24:51.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:50 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:24:51.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:50 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:24:51.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:50 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:51.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:50 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:51.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:50 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:51.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:50 smithi052 ceph-mon[35071]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-26T08:24:51.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:50 smithi052 ceph-mon[35071]: from='osd.2 [v2:172.21.15.183:6800/2103180908,v1:172.21.15.183:6801/2103180908]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-26T08:24:51.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:50 smithi052 ceph-mon[35071]: osdmap e16: 3 total, 2 up, 3 in 2024-03-26T08:24:51.075 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:50 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:24:51.075 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:50 smithi052 ceph-mon[35071]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-26T08:24:51.075 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:50 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:24:51.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:50 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:24:51.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:50 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:24:51.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:50 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:51.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:50 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:51.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:50 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:51.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:50 smithi043 ceph-mon[31929]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-26T08:24:51.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:50 smithi043 ceph-mon[31929]: from='osd.2 [v2:172.21.15.183:6800/2103180908,v1:172.21.15.183:6801/2103180908]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-26T08:24:51.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:50 smithi043 ceph-mon[31929]: osdmap e16: 3 total, 2 up, 3 in 2024-03-26T08:24:51.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:50 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:24:51.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:50 smithi043 ceph-mon[31929]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-26T08:24:51.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:50 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:24:51.214 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:24:51.896 INFO:journalctl@ceph.osd.2.smithi183.stdout:Mar 26 08:24:51 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-2[40992]: 2024-03-26T08:24:51.624+0000 7f4715c00700 -1 osd.2 0 waiting for initial osdmap 2024-03-26T08:24:51.897 INFO:journalctl@ceph.osd.2.smithi183.stdout:Mar 26 08:24:51 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-2[40992]: 2024-03-26T08:24:51.644+0000 7f470ba00700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T08:24:52.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:51 smithi183 ceph-mon[35818]: pgmap v103: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:52.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:51 smithi183 ceph-mon[35818]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-03-26T08:24:52.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:51 smithi183 ceph-mon[35818]: osdmap e17: 3 total, 2 up, 3 in 2024-03-26T08:24:52.193 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:51 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:24:52.193 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:51 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:24:52.193 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:51 smithi183 ceph-mon[35818]: from='osd.2 ' entity='osd.2' 2024-03-26T08:24:52.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:51 smithi052 ceph-mon[35071]: pgmap v103: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:52.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:51 smithi052 ceph-mon[35071]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-03-26T08:24:52.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:51 smithi052 ceph-mon[35071]: osdmap e17: 3 total, 2 up, 3 in 2024-03-26T08:24:52.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:51 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:24:52.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:51 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:24:52.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:51 smithi052 ceph-mon[35071]: from='osd.2 ' entity='osd.2' 2024-03-26T08:24:52.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:51 smithi043 ceph-mon[31929]: pgmap v103: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:52.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:51 smithi043 ceph-mon[31929]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-03-26T08:24:52.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:51 smithi043 ceph-mon[31929]: osdmap e17: 3 total, 2 up, 3 in 2024-03-26T08:24:52.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:51 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:24:52.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:51 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:24:52.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:51 smithi043 ceph-mon[31929]: from='osd.2 ' entity='osd.2' 2024-03-26T08:24:53.148 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:52 smithi043 ceph-mon[31929]: purged_snaps scrub starts 2024-03-26T08:24:53.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:52 smithi043 ceph-mon[31929]: purged_snaps scrub ok 2024-03-26T08:24:53.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:52 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:24:53.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:52 smithi043 ceph-mon[31929]: osd.2 [v2:172.21.15.183:6800/2103180908,v1:172.21.15.183:6801/2103180908] boot 2024-03-26T08:24:53.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:53 smithi043 ceph-mon[31929]: osdmap e18: 3 total, 3 up, 3 in 2024-03-26T08:24:53.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:53 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:24:53.227 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:53 smithi183 ceph-mon[35818]: purged_snaps scrub starts 2024-03-26T08:24:53.227 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:53 smithi183 ceph-mon[35818]: purged_snaps scrub ok 2024-03-26T08:24:53.227 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:53 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:24:53.227 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:53 smithi183 ceph-mon[35818]: osd.2 [v2:172.21.15.183:6800/2103180908,v1:172.21.15.183:6801/2103180908] boot 2024-03-26T08:24:53.227 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:53 smithi183 ceph-mon[35818]: osdmap e18: 3 total, 3 up, 3 in 2024-03-26T08:24:53.228 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:53 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:24:53.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:53 smithi052 ceph-mon[35071]: purged_snaps scrub starts 2024-03-26T08:24:53.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:53 smithi052 ceph-mon[35071]: purged_snaps scrub ok 2024-03-26T08:24:53.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:53 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:24:53.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:53 smithi052 ceph-mon[35071]: osd.2 [v2:172.21.15.183:6800/2103180908,v1:172.21.15.183:6801/2103180908] boot 2024-03-26T08:24:53.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:53 smithi052 ceph-mon[35071]: osdmap e18: 3 total, 3 up, 3 in 2024-03-26T08:24:53.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:53 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:24:54.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:54 smithi043 ceph-mon[31929]: pgmap v105: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:54.183 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-26T08:24:54.183 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-26T08:24:54.183 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2024-03-26T08:24:54.183 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2024-03-26T08:24:54.183 INFO:teuthology.orchestra.run.smithi043.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0383501 s, 273 MB/s 2024-03-26T08:24:54.184 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2024-03-26T08:24:54.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:54 smithi183 ceph-mon[35818]: pgmap v105: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:54.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:54 smithi052 ceph-mon[35071]: pgmap v105: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:24:55.115 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi043:vg_nvme/lv_3 2024-03-26T08:24:55.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:55 smithi183 ceph-mon[35818]: osdmap e19: 3 total, 3 up, 3 in 2024-03-26T08:24:55.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:55 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-26T08:24:55.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:55 smithi052 ceph-mon[35071]: osdmap e19: 3 total, 3 up, 3 in 2024-03-26T08:24:55.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:55 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-26T08:24:55.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:55 smithi043 ceph-mon[31929]: osdmap e19: 3 total, 3 up, 3 in 2024-03-26T08:24:55.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:55 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-26T08:24:55.560 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:24:56.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:56 smithi183 ceph-mon[35818]: pgmap v108: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:24:56.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:56 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-26T08:24:56.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:56 smithi183 ceph-mon[35818]: osdmap e20: 3 total, 3 up, 3 in 2024-03-26T08:24:56.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:56 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T08:24:56.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:56 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:56.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:56 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:56.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:56 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:24:56.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:56 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:56.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:56 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:24:56.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:56 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:24:56.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:56 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:56.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:56 smithi052 ceph-mon[35071]: pgmap v108: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:24:56.325 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:56 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-26T08:24:56.325 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:56 smithi052 ceph-mon[35071]: osdmap e20: 3 total, 3 up, 3 in 2024-03-26T08:24:56.325 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:56 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T08:24:56.325 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:56 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:56.325 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:56 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:56.325 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:56 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:24:56.325 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:56 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:56.325 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:56 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:24:56.325 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:56 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:24:56.325 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:56 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:56.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:56 smithi043 ceph-mon[31929]: pgmap v108: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:24:56.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:56 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-26T08:24:56.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:56 smithi043 ceph-mon[31929]: osdmap e20: 3 total, 3 up, 3 in 2024-03-26T08:24:56.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:56 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T08:24:56.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:56 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:56.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:56 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:56.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:56 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:24:56.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:56 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:56.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:56 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:24:56.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:56 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:24:56.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:56 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:57.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:57 smithi183 ceph-mon[35818]: Detected new or changed devices on smithi183 2024-03-26T08:24:57.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:57 smithi183 ceph-mon[35818]: Adjusting osd_memory_target on smithi183 to 17091M 2024-03-26T08:24:57.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:57 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-26T08:24:57.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:57 smithi183 ceph-mon[35818]: osdmap e21: 3 total, 3 up, 3 in 2024-03-26T08:24:57.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:57 smithi052 ceph-mon[35071]: Detected new or changed devices on smithi183 2024-03-26T08:24:57.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:57 smithi052 ceph-mon[35071]: Adjusting osd_memory_target on smithi183 to 17091M 2024-03-26T08:24:57.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:57 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-26T08:24:57.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:57 smithi052 ceph-mon[35071]: osdmap e21: 3 total, 3 up, 3 in 2024-03-26T08:24:57.364 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:57 smithi043 ceph-mon[31929]: Detected new or changed devices on smithi183 2024-03-26T08:24:57.364 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:57 smithi043 ceph-mon[31929]: Adjusting osd_memory_target on smithi183 to 17091M 2024-03-26T08:24:57.364 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:57 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-26T08:24:57.364 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:57 smithi043 ceph-mon[31929]: osdmap e21: 3 total, 3 up, 3 in 2024-03-26T08:24:57.365 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 26 08:24:57 smithi043 sudo[48136]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T08:24:57.365 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 26 08:24:57 smithi043 sudo[48136]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T08:24:57.365 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 26 08:24:57 smithi043 sudo[48136]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T08:24:57.365 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 26 08:24:57 smithi043 sudo[48136]: pam_unix(sudo:session): session closed for user root 2024-03-26T08:24:57.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.659+0000 7fadd5000700 1 -- 172.21.15.43:0/2020492015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd01032e0 msgr2=0x7fadd0103740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:24:57.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.659+0000 7fadd5000700 1 --2- 172.21.15.43:0/2020492015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd01032e0 0x7fadd0103740 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0099c0 tx=0x7fadbc009cd0 comp rx=0 tx=0).stop 2024-03-26T08:24:57.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.659+0000 7fadd5000700 1 -- 172.21.15.43:0/2020492015 shutdown_connections 2024-03-26T08:24:57.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.659+0000 7fadd5000700 1 --2- 172.21.15.43:0/2020492015 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadd0103c80 0x7fadd010a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:24:57.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.659+0000 7fadd5000700 1 --2- 172.21.15.43:0/2020492015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadd01020e0 0x7fadd0102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:24:57.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.659+0000 7fadd5000700 1 --2- 172.21.15.43:0/2020492015 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd01032e0 0x7fadd0103740 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:24:57.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.659+0000 7fadd5000700 1 -- 172.21.15.43:0/2020492015 >> 172.21.15.43:0/2020492015 conn(0x7fadd00fd6a0 msgr2=0x7fadd00ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:24:57.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.659+0000 7fadd5000700 1 -- 172.21.15.43:0/2020492015 shutdown_connections 2024-03-26T08:24:57.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.659+0000 7fadd5000700 1 -- 172.21.15.43:0/2020492015 wait complete. 2024-03-26T08:24:57.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.660+0000 7fadd5000700 1 Processor -- start 2024-03-26T08:24:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.660+0000 7fadd5000700 1 -- start start 2024-03-26T08:24:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.660+0000 7fadd5000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd01020e0 0x7fadd019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:24:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.660+0000 7fadd5000700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadd01032e0 0x7fadd019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:24:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.660+0000 7fadd5000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadd0103c80 0x7fadd01a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:24:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.661+0000 7fadd5000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd010dbd0 con 0x7fadd01020e0 2024-03-26T08:24:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.661+0000 7fadd5000700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd010da50 con 0x7fadd01032e0 2024-03-26T08:24:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.661+0000 7fadd5000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd010dd50 con 0x7fadd0103c80 2024-03-26T08:24:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.661+0000 7fadcf400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd01020e0 0x7fadd019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:24:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.661+0000 7fadcf400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd01020e0 0x7fadd019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52386/0 (socket says 172.21.15.43:52386) 2024-03-26T08:24:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.661+0000 7fadcf400700 1 -- 172.21.15.43:0/3023855412 learned_addr learned my addr 172.21.15.43:0/3023855412 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:24:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.661+0000 7fadcf400700 1 -- 172.21.15.43:0/3023855412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadd0103c80 msgr2=0x7fadd01a1a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T08:24:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.661+0000 7fadcf400700 1 --2- 172.21.15.43:0/3023855412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadd0103c80 0x7fadd01a1a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:24:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.661+0000 7fadcf400700 1 -- 172.21.15.43:0/3023855412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadd01032e0 msgr2=0x7fadd019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:24:57.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.661+0000 7fadcf400700 1 --2- 172.21.15.43:0/3023855412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadd01032e0 0x7fadd019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:24:57.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.661+0000 7fadcf400700 1 -- 172.21.15.43:0/3023855412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd01a20e0 con 0x7fadd01020e0 2024-03-26T08:24:57.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.662+0000 7fadcf400700 1 --2- 172.21.15.43:0/3023855412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd01020e0 0x7fadd019a480 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fadc000d9b0 tx=0x7fadc000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:24:57.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.662+0000 7fadc7e00700 1 -- 172.21.15.43:0/3023855412 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0017070 con 0x7fadd01020e0 2024-03-26T08:24:57.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.662+0000 7fadc7e00700 1 -- 172.21.15.43:0/3023855412 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fadc0010460 con 0x7fadd01020e0 2024-03-26T08:24:57.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.662+0000 7fadd5000700 1 -- 172.21.15.43:0/3023855412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd01a23d0 con 0x7fadd01020e0 2024-03-26T08:24:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.662+0000 7fadc7e00700 1 -- 172.21.15.43:0/3023855412 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc000f620 con 0x7fadd01020e0 2024-03-26T08:24:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.662+0000 7fadd5000700 1 -- 172.21.15.43:0/3023855412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd01a2910 con 0x7fadd01020e0 2024-03-26T08:24:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.664+0000 7fadd5000700 1 -- 172.21.15.43:0/3023855412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd0068e50 con 0x7fadd01020e0 2024-03-26T08:24:57.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.666+0000 7fadc7e00700 1 -- 172.21.15.43:0/3023855412 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 15) v1 ==== 137234+0+0 (secure 0 0 0) 0x7fadc0010610 con 0x7fadd01020e0 2024-03-26T08:24:57.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.667+0000 7fadc7e00700 1 --2- 172.21.15.43:0/3023855412 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fadb40a2880 0x7fadb40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:24:57.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.667+0000 7fadc7e00700 1 -- 172.21.15.43:0/3023855412 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 3237+0+0 (secure 0 0 0) 0x7fadc00cd200 con 0x7fadd01020e0 2024-03-26T08:24:57.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.667+0000 7fadcea00700 1 --2- 172.21.15.43:0/3023855412 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fadb40a2880 0x7fadb40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:24:57.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.668+0000 7fadcea00700 1 --2- 172.21.15.43:0/3023855412 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fadb40a2880 0x7fadb40a4d40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0098e0 tx=0x7fadbc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:24:57.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.673+0000 7fadc7e00700 1 -- 172.21.15.43:0/3023855412 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fadc009b6b0 con 0x7fadd01020e0 2024-03-26T08:24:57.903 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 26 08:24:57 smithi043 sudo[48157]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T08:24:57.903 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 26 08:24:57 smithi043 sudo[48157]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T08:24:57.903 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 26 08:24:57 smithi043 sudo[48157]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T08:24:57.903 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 26 08:24:57 smithi043 sudo[48157]: pam_unix(sudo:session): session closed for user root 2024-03-26T08:24:57.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:57.912+0000 7fadd5000700 1 -- 172.21.15.43:0/3023855412 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fadd010a1d0 con 0x7fadb40a2880 2024-03-26T08:24:58.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:58 smithi183 ceph-mon[35818]: pgmap v111: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:24:58.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:58 smithi183 ceph-mon[35818]: osdmap e22: 3 total, 3 up, 3 in 2024-03-26T08:24:58.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:58 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:58.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:58 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:24:58.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:58 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:24:58.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:58 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:24:58.323 INFO:journalctl@ceph.osd.1.smithi052.stdout:Mar 26 08:24:58 smithi052 sudo[43913]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T08:24:58.323 INFO:journalctl@ceph.osd.1.smithi052.stdout:Mar 26 08:24:58 smithi052 sudo[43913]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T08:24:58.323 INFO:journalctl@ceph.osd.1.smithi052.stdout:Mar 26 08:24:58 smithi052 sudo[43913]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T08:24:58.323 INFO:journalctl@ceph.osd.1.smithi052.stdout:Mar 26 08:24:58 smithi052 sudo[43913]: pam_unix(sudo:session): session closed for user root 2024-03-26T08:24:58.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:58 smithi052 ceph-mon[35071]: pgmap v111: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:24:58.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:58 smithi052 ceph-mon[35071]: osdmap e22: 3 total, 3 up, 3 in 2024-03-26T08:24:58.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:58 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:58.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:58 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:24:58.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:58 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:24:58.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:58 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:24:58.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:58 smithi043 ceph-mon[31929]: pgmap v111: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:24:58.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:58 smithi043 ceph-mon[31929]: osdmap e22: 3 total, 3 up, 3 in 2024-03-26T08:24:58.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:58 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:58.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:58 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:24:58.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:58 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:24:58.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:58 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:24:59.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:24:59.081+0000 7fadc7e00700 1 -- 172.21.15.43:0/3023855412 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fadc0097620 con 0x7fadd01020e0 2024-03-26T08:24:59.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:59 smithi043 ceph-mon[31929]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:24:59.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:24:59 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:59.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:59 smithi052 ceph-mon[35071]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:24:59.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:24:59 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:59.324 INFO:journalctl@ceph.osd.1.smithi052.stdout:Mar 26 08:24:58 smithi052 sudo[43916]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T08:24:59.324 INFO:journalctl@ceph.osd.1.smithi052.stdout:Mar 26 08:24:58 smithi052 sudo[43916]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T08:24:59.324 INFO:journalctl@ceph.osd.1.smithi052.stdout:Mar 26 08:24:58 smithi052 sudo[43916]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T08:24:59.324 INFO:journalctl@ceph.osd.1.smithi052.stdout:Mar 26 08:24:58 smithi052 sudo[43916]: pam_unix(sudo:session): session closed for user root 2024-03-26T08:24:59.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:59 smithi183 ceph-mon[35818]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:24:59.473 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:24:59 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:24:59.799 INFO:journalctl@ceph.osd.2.smithi183.stdout:Mar 26 08:24:59 smithi183 sudo[44999]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T08:24:59.799 INFO:journalctl@ceph.osd.2.smithi183.stdout:Mar 26 08:24:59 smithi183 sudo[44999]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T08:24:59.800 INFO:journalctl@ceph.osd.2.smithi183.stdout:Mar 26 08:24:59 smithi183 sudo[44999]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T08:24:59.800 INFO:journalctl@ceph.osd.2.smithi183.stdout:Mar 26 08:24:59 smithi183 sudo[44999]: pam_unix(sudo:session): session closed for user root 2024-03-26T08:25:00.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:00 smithi052 ceph-mon[35071]: pgmap v113: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:00.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:00 smithi052 ceph-mon[35071]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-03-26T08:25:00.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:00 smithi043 ceph-mon[31929]: pgmap v113: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:00.395 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:00 smithi043 ceph-mon[31929]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-03-26T08:25:00.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:00 smithi183 ceph-mon[35818]: pgmap v113: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:00.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:00 smithi183 ceph-mon[35818]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-03-26T08:25:00.550 INFO:journalctl@ceph.osd.2.smithi183.stdout:Mar 26 08:25:00 smithi183 sudo[45002]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T08:25:00.550 INFO:journalctl@ceph.osd.2.smithi183.stdout:Mar 26 08:25:00 smithi183 sudo[45002]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T08:25:00.550 INFO:journalctl@ceph.osd.2.smithi183.stdout:Mar 26 08:25:00 smithi183 sudo[45002]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T08:25:00.550 INFO:journalctl@ceph.osd.2.smithi183.stdout:Mar 26 08:25:00 smithi183 sudo[45002]: pam_unix(sudo:session): session closed for user root 2024-03-26T08:25:00.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:00 smithi043 sudo[48347]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T08:25:00.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:00 smithi043 sudo[48347]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T08:25:00.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:00 smithi043 sudo[48347]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T08:25:00.957 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:00 smithi043 sudo[48347]: pam_unix(sudo:session): session closed for user root 2024-03-26T08:25:01.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:01 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:01.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:01 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:01.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:01 smithi043 sudo[48367]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T08:25:01.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:01 smithi043 sudo[48367]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T08:25:01.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:01 smithi043 sudo[48367]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T08:25:01.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:01 smithi043 sudo[48367]: pam_unix(sudo:session): session closed for user root 2024-03-26T08:25:01.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:01 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:02.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:01 smithi183 sudo[45005]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T08:25:02.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:01 smithi183 sudo[45005]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T08:25:02.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:01 smithi183 sudo[45005]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T08:25:02.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:01 smithi183 sudo[45005]: pam_unix(sudo:session): session closed for user root 2024-03-26T08:25:02.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:02 smithi183 sudo[45008]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T08:25:02.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:02 smithi183 sudo[45008]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T08:25:02.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:02 smithi183 sudo[45008]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T08:25:02.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:02 smithi183 sudo[45008]: pam_unix(sudo:session): session closed for user root 2024-03-26T08:25:03.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:02 smithi052 sudo[43931]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T08:25:03.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:02 smithi052 sudo[43931]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T08:25:03.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:02 smithi052 sudo[43931]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T08:25:03.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:02 smithi052 sudo[43931]: pam_unix(sudo:session): session closed for user root 2024-03-26T08:25:03.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:03 smithi183 ceph-mon[35818]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T08:25:03.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:03 smithi183 ceph-mon[35818]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:03.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:03 smithi183 ceph-mon[35818]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T08:25:03.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:03 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:25:03.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:03 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:25:03.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:03 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:25:03.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:03 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:25:03.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:03 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:25:03.801 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:03 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:25:03.801 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:03 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1897366415' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7dcbf00-6586-4664-a69e-3cb0df8dfd30"}]: dispatch 2024-03-26T08:25:03.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:03 smithi052 sudo[43934]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T08:25:03.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:03 smithi052 sudo[43934]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T08:25:03.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:03 smithi052 sudo[43934]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T08:25:03.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:03 smithi052 sudo[43934]: pam_unix(sudo:session): session closed for user root 2024-03-26T08:25:03.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:03 smithi052 ceph-mon[35071]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T08:25:03.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:03 smithi052 ceph-mon[35071]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:03.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:03 smithi052 ceph-mon[35071]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T08:25:03.825 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:03 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:25:03.825 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:03 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:25:03.825 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:03 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:25:03.825 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:03 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:25:03.825 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:03 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:25:03.825 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:03 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:25:03.825 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:03 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1897366415' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7dcbf00-6586-4664-a69e-3cb0df8dfd30"}]: dispatch 2024-03-26T08:25:03.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:03 smithi043 ceph-mon[31929]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T08:25:03.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:03 smithi043 ceph-mon[31929]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:03.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:03 smithi043 ceph-mon[31929]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T08:25:03.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:03 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:25:03.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:03 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:25:03.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:03 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:25:03.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:03 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:25:03.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:03 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:25:03.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:03 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:25:03.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:03 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1897366415' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7dcbf00-6586-4664-a69e-3cb0df8dfd30"}]: dispatch 2024-03-26T08:25:04.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:04 smithi183 ceph-mon[35818]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T08:25:04.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:04 smithi183 ceph-mon[35818]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:04.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:04 smithi183 ceph-mon[35818]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T08:25:04.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:04 smithi183 ceph-mon[35818]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T08:25:04.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:04 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:25:04.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:04 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:25:04.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:04 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:25:04.801 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:04 smithi183 ceph-mon[35818]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T08:25:04.801 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:04 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1897366415' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7dcbf00-6586-4664-a69e-3cb0df8dfd30"}]': finished 2024-03-26T08:25:04.801 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:04 smithi183 ceph-mon[35818]: osdmap e23: 4 total, 3 up, 4 in 2024-03-26T08:25:04.801 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:04 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:25:04.801 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:04 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2651313348' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:25:04.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:04 smithi052 ceph-mon[35071]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T08:25:04.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:04 smithi052 ceph-mon[35071]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:04.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:04 smithi052 ceph-mon[35071]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T08:25:04.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:04 smithi052 ceph-mon[35071]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T08:25:04.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:04 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:25:04.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:04 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:25:04.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:04 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:25:04.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:04 smithi052 ceph-mon[35071]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T08:25:04.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:04 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1897366415' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7dcbf00-6586-4664-a69e-3cb0df8dfd30"}]': finished 2024-03-26T08:25:04.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:04 smithi052 ceph-mon[35071]: osdmap e23: 4 total, 3 up, 4 in 2024-03-26T08:25:04.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:04 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:25:04.825 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:04 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2651313348' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:25:04.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:04 smithi043 ceph-mon[31929]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T08:25:04.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:04 smithi043 ceph-mon[31929]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:04.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:04 smithi043 ceph-mon[31929]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T08:25:04.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:04 smithi043 ceph-mon[31929]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T08:25:04.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:04 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:25:04.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:04 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:25:04.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:04 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:25:04.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:04 smithi043 ceph-mon[31929]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T08:25:04.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:04 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1897366415' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7dcbf00-6586-4664-a69e-3cb0df8dfd30"}]': finished 2024-03-26T08:25:04.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:04 smithi043 ceph-mon[31929]: osdmap e23: 4 total, 3 up, 4 in 2024-03-26T08:25:04.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:04 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:25:04.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:04 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2651313348' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:25:05.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:05 smithi043 ceph-mon[31929]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:05.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:05 smithi052 ceph-mon[35071]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:06.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:05 smithi183 ceph-mon[35818]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:07.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:07 smithi052 ceph-mon[35071]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-26T08:25:07.824 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:07 smithi043 ceph-mon[31929]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-26T08:25:08.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:07 smithi183 ceph-mon[35818]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-26T08:25:09.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:09 smithi052 ceph-mon[35071]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-03-26T08:25:09.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:09 smithi043 ceph-mon[31929]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-03-26T08:25:10.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:09 smithi183 ceph-mon[35818]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-03-26T08:25:11.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:11 smithi043 ceph-mon[31929]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-26T08:25:11.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:11 smithi052 ceph-mon[35071]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-26T08:25:12.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:11 smithi183 ceph-mon[35818]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-26T08:25:13.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:13 smithi043 ceph-mon[31929]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-26T08:25:13.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:13 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-26T08:25:13.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:13 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:25:14.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:13 smithi183 ceph-mon[35818]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-26T08:25:14.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:13 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-26T08:25:14.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:13 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:25:14.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:13 smithi052 ceph-mon[35071]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-26T08:25:14.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:13 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-26T08:25:14.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:13 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:25:14.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:14 smithi043 ceph-mon[31929]: Deploying daemon osd.3 on smithi043 2024-03-26T08:25:15.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:14 smithi183 ceph-mon[35818]: Deploying daemon osd.3 on smithi043 2024-03-26T08:25:15.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:14 smithi052 ceph-mon[35071]: Deploying daemon osd.3 on smithi043 2024-03-26T08:25:15.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:15 smithi043 ceph-mon[31929]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 73 KiB/s wr, 5 op/s 2024-03-26T08:25:16.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:15 smithi183 ceph-mon[35818]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 73 KiB/s wr, 5 op/s 2024-03-26T08:25:16.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:15 smithi052 ceph-mon[35071]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 73 KiB/s wr, 5 op/s 2024-03-26T08:25:17.874 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:17 smithi043 ceph-mon[31929]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-26T08:25:18.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:17 smithi183 ceph-mon[35818]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-26T08:25:18.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:17 smithi052 ceph-mon[35071]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-26T08:25:19.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:19 smithi043 ceph-mon[31929]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:20.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:19 smithi183 ceph-mon[35818]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:20.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:19 smithi052 ceph-mon[35071]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:21.881 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:21 smithi183 ceph-mon[35818]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:21.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:21 smithi043 ceph-mon[31929]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:22.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:21 smithi052 ceph-mon[35071]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:23.699 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:23 smithi043 ceph-mon[31929]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:23.699 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:23 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:23.700 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:23 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:23.700 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:23 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:25:23.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:23 smithi183 ceph-mon[35818]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:23.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:23 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:23.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:23 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:23.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:23 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:25:23.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:23 smithi052 ceph-mon[35071]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:23.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:23 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:23.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:23 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:23.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:23 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:25:25.773 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:25 smithi043 ceph-mon[31929]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:25.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:25 smithi183 ceph-mon[35818]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:25.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:25 smithi052 ceph-mon[35071]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:27.138 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:27 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:27.138 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:27 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:27.138 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:27 smithi043 ceph-mon[31929]: from='osd.3 [v2:172.21.15.43:6810/3722846033,v1:172.21.15.43:6811/3722846033]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-26T08:25:27.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:27 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:27.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:27 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:27.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:27 smithi183 ceph-mon[35818]: from='osd.3 [v2:172.21.15.43:6810/3722846033,v1:172.21.15.43:6811/3722846033]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-26T08:25:27.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:27 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:27.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:27 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:27.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:27 smithi052 ceph-mon[35071]: from='osd.3 [v2:172.21.15.43:6810/3722846033,v1:172.21.15.43:6811/3722846033]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-26T08:25:27.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:25:27.711+0000 7fadc7e00700 1 -- 172.21.15.43:0/3023855412 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fadd010a1d0 con 0x7fadb40a2880 2024-03-26T08:25:27.708 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 3 on host 'smithi043' 2024-03-26T08:25:27.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:25:27.715+0000 7fadd5000700 1 -- 172.21.15.43:0/3023855412 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fadb40a2880 msgr2=0x7fadb40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:25:27.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:25:27.715+0000 7fadd5000700 1 --2- 172.21.15.43:0/3023855412 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fadb40a2880 0x7fadb40a4d40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0098e0 tx=0x7fadbc000bc0 comp rx=0 tx=0).stop 2024-03-26T08:25:27.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:25:27.715+0000 7fadd5000700 1 -- 172.21.15.43:0/3023855412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd01020e0 msgr2=0x7fadd019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:25:27.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:25:27.715+0000 7fadd5000700 1 --2- 172.21.15.43:0/3023855412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd01020e0 0x7fadd019a480 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fadc000d9b0 tx=0x7fadc000dd70 comp rx=0 tx=0).stop 2024-03-26T08:25:27.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:25:27.715+0000 7fadd5000700 1 -- 172.21.15.43:0/3023855412 shutdown_connections 2024-03-26T08:25:27.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:25:27.715+0000 7fadd5000700 1 --2- 172.21.15.43:0/3023855412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadd0103c80 0x7fadd01a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:25:27.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:25:27.715+0000 7fadd5000700 1 --2- 172.21.15.43:0/3023855412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd01020e0 0x7fadd019a480 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:25:27.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:25:27.715+0000 7fadd5000700 1 --2- 172.21.15.43:0/3023855412 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fadb40a2880 0x7fadb40a4d40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:25:27.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:25:27.716+0000 7fadd5000700 1 --2- 172.21.15.43:0/3023855412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadd01032e0 0x7fadd019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:25:27.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:25:27.716+0000 7fadd5000700 1 -- 172.21.15.43:0/3023855412 >> 172.21.15.43:0/3023855412 conn(0x7fadd00fd6a0 msgr2=0x7fadd0108ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:25:27.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:25:27.716+0000 7fadd5000700 1 -- 172.21.15.43:0/3023855412 shutdown_connections 2024-03-26T08:25:27.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:25:27.716+0000 7fadd5000700 1 -- 172.21.15.43:0/3023855412 wait complete. 2024-03-26T08:25:28.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:28 smithi043 ceph-mon[31929]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:28.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:28 smithi043 ceph-mon[31929]: from='osd.3 [v2:172.21.15.43:6810/3722846033,v1:172.21.15.43:6811/3722846033]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-26T08:25:28.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:28 smithi043 ceph-mon[31929]: osdmap e24: 4 total, 3 up, 4 in 2024-03-26T08:25:28.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:28 smithi043 ceph-mon[31929]: from='osd.3 [v2:172.21.15.43:6810/3722846033,v1:172.21.15.43:6811/3722846033]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-26T08:25:28.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:28 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:25:28.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:28 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:25:28.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:28 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:25:28.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:28 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:28.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:28 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:28.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:28 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:28.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:28 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:25:28.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:28 smithi183 ceph-mon[35818]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:28.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:28 smithi183 ceph-mon[35818]: from='osd.3 [v2:172.21.15.43:6810/3722846033,v1:172.21.15.43:6811/3722846033]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-26T08:25:28.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:28 smithi183 ceph-mon[35818]: osdmap e24: 4 total, 3 up, 4 in 2024-03-26T08:25:28.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:28 smithi183 ceph-mon[35818]: from='osd.3 [v2:172.21.15.43:6810/3722846033,v1:172.21.15.43:6811/3722846033]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-26T08:25:28.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:28 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:25:28.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:28 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:25:28.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:28 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:25:28.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:28 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:28.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:28 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:28.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:28 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:28.551 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:28 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:25:28.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:28 smithi052 ceph-mon[35071]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:28.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:28 smithi052 ceph-mon[35071]: from='osd.3 [v2:172.21.15.43:6810/3722846033,v1:172.21.15.43:6811/3722846033]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-26T08:25:28.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:28 smithi052 ceph-mon[35071]: osdmap e24: 4 total, 3 up, 4 in 2024-03-26T08:25:28.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:28 smithi052 ceph-mon[35071]: from='osd.3 [v2:172.21.15.43:6810/3722846033,v1:172.21.15.43:6811/3722846033]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-26T08:25:28.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:28 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:25:28.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:28 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:25:28.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:28 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:25:28.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:28 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:28.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:28 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:28.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:28 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:28.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:28 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:25:28.707 DEBUG:teuthology.orchestra.run.smithi043:osd.3> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.3.service 2024-03-26T08:25:28.711 INFO:tasks.cephadm:Deploying osd.4 on smithi052 with /dev/vg_nvme/lv_3... 2024-03-26T08:25:28.711 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-26T08:25:29.184 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T08:25:29.232 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:29 smithi043 ceph-mon[31929]: from='osd.3 [v2:172.21.15.43:6810/3722846033,v1:172.21.15.43:6811/3722846033]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-03-26T08:25:29.232 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:29 smithi043 ceph-mon[31929]: osdmap e25: 4 total, 3 up, 4 in 2024-03-26T08:25:29.232 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:29 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:25:29.232 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:29 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:25:29.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:29 smithi043 ceph-mon[31929]: from='osd.3 [v2:172.21.15.43:6810/3722846033,v1:172.21.15.43:6811/3722846033]' entity='osd.3' 2024-03-26T08:25:29.466 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:29 smithi052 ceph-mon[35071]: from='osd.3 [v2:172.21.15.43:6810/3722846033,v1:172.21.15.43:6811/3722846033]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-03-26T08:25:29.466 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:29 smithi052 ceph-mon[35071]: osdmap e25: 4 total, 3 up, 4 in 2024-03-26T08:25:29.466 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:29 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:25:29.466 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:29 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:25:29.467 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:29 smithi052 ceph-mon[35071]: from='osd.3 [v2:172.21.15.43:6810/3722846033,v1:172.21.15.43:6811/3722846033]' entity='osd.3' 2024-03-26T08:25:29.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:29 smithi183 ceph-mon[35818]: from='osd.3 [v2:172.21.15.43:6810/3722846033,v1:172.21.15.43:6811/3722846033]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-03-26T08:25:29.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:29 smithi183 ceph-mon[35818]: osdmap e25: 4 total, 3 up, 4 in 2024-03-26T08:25:29.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:29 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:25:29.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:29 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:25:29.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:29 smithi183 ceph-mon[35818]: from='osd.3 [v2:172.21.15.43:6810/3722846033,v1:172.21.15.43:6811/3722846033]' entity='osd.3' 2024-03-26T08:25:30.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:30 smithi043 ceph-mon[31929]: purged_snaps scrub starts 2024-03-26T08:25:30.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:30 smithi043 ceph-mon[31929]: purged_snaps scrub ok 2024-03-26T08:25:30.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:30 smithi043 ceph-mon[31929]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:30.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:30 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:25:30.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:30 smithi043 ceph-mon[31929]: osd.3 [v2:172.21.15.43:6810/3722846033,v1:172.21.15.43:6811/3722846033] boot 2024-03-26T08:25:30.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:30 smithi043 ceph-mon[31929]: osdmap e26: 4 total, 4 up, 4 in 2024-03-26T08:25:30.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:30 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:25:30.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:30 smithi043 ceph-mon[31929]: osdmap e27: 4 total, 4 up, 4 in 2024-03-26T08:25:30.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:30 smithi183 ceph-mon[35818]: purged_snaps scrub starts 2024-03-26T08:25:30.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:30 smithi183 ceph-mon[35818]: purged_snaps scrub ok 2024-03-26T08:25:30.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:30 smithi183 ceph-mon[35818]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:30.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:30 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:25:30.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:30 smithi183 ceph-mon[35818]: osd.3 [v2:172.21.15.43:6810/3722846033,v1:172.21.15.43:6811/3722846033] boot 2024-03-26T08:25:30.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:30 smithi183 ceph-mon[35818]: osdmap e26: 4 total, 4 up, 4 in 2024-03-26T08:25:30.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:30 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:25:30.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:30 smithi183 ceph-mon[35818]: osdmap e27: 4 total, 4 up, 4 in 2024-03-26T08:25:30.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:30 smithi052 ceph-mon[35071]: purged_snaps scrub starts 2024-03-26T08:25:30.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:30 smithi052 ceph-mon[35071]: purged_snaps scrub ok 2024-03-26T08:25:30.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:30 smithi052 ceph-mon[35071]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:25:30.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:30 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:25:30.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:30 smithi052 ceph-mon[35071]: osd.3 [v2:172.21.15.43:6810/3722846033,v1:172.21.15.43:6811/3722846033] boot 2024-03-26T08:25:30.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:30 smithi052 ceph-mon[35071]: osdmap e26: 4 total, 4 up, 4 in 2024-03-26T08:25:30.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:30 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:25:30.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:30 smithi052 ceph-mon[35071]: osdmap e27: 4 total, 4 up, 4 in 2024-03-26T08:25:32.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:31 smithi043 ceph-mon[31929]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:25:32.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:31 smithi043 ceph-mon[31929]: osdmap e28: 4 total, 4 up, 4 in 2024-03-26T08:25:32.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:31 smithi183 ceph-mon[35818]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:25:32.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:31 smithi183 ceph-mon[35818]: osdmap e28: 4 total, 4 up, 4 in 2024-03-26T08:25:32.305 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:31 smithi052 ceph-mon[35071]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:25:32.305 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:31 smithi052 ceph-mon[35071]: osdmap e28: 4 total, 4 up, 4 in 2024-03-26T08:25:32.306 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-26T08:25:32.306 INFO:teuthology.orchestra.run.smithi052.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-26T08:25:32.306 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10+0 records in 2024-03-26T08:25:32.306 INFO:teuthology.orchestra.run.smithi052.stderr:10+0 records out 2024-03-26T08:25:32.306 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0510437 s, 205 MB/s 2024-03-26T08:25:32.306 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping successful for: 2024-03-26T08:25:33.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:32 smithi052 ceph-mon[35071]: osdmap e29: 4 total, 4 up, 4 in 2024-03-26T08:25:33.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:32 smithi043 ceph-mon[31929]: osdmap e29: 4 total, 4 up, 4 in 2024-03-26T08:25:33.172 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi052:vg_nvme/lv_3 2024-03-26T08:25:33.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:32 smithi183 ceph-mon[35818]: osdmap e29: 4 total, 4 up, 4 in 2024-03-26T08:25:33.617 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T08:25:34.039 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:33 smithi052 ceph-mon[35071]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:25:34.039 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:33 smithi052 ceph-mon[35071]: osdmap e30: 4 total, 4 up, 4 in 2024-03-26T08:25:34.039 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:33 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:34.039 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:33 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:34.040 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:33 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:25:34.040 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:33 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:25:34.040 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:33 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:34.040 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:33 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:25:34.040 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:33 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:25:34.040 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:33 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:34.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:33 smithi043 ceph-mon[31929]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:25:34.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:33 smithi043 ceph-mon[31929]: osdmap e30: 4 total, 4 up, 4 in 2024-03-26T08:25:34.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:33 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:34.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:33 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:34.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:33 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:25:34.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:33 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:25:34.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:33 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:34.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:33 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:25:34.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:33 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:25:34.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:33 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:34.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:33 smithi183 ceph-mon[35818]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:25:34.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:33 smithi183 ceph-mon[35818]: osdmap e30: 4 total, 4 up, 4 in 2024-03-26T08:25:34.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:33 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:34.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:33 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:34.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:33 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:25:34.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:33 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:25:34.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:33 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:34.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:33 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:25:34.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:33 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:25:34.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:33 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:35.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:34 smithi052 ceph-mon[35071]: Detected new or changed devices on smithi043 2024-03-26T08:25:35.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:34 smithi052 ceph-mon[35071]: Adjusting osd_memory_target on smithi043 to 8547M 2024-03-26T08:25:35.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:34 smithi183 ceph-mon[35818]: Detected new or changed devices on smithi043 2024-03-26T08:25:35.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:34 smithi183 ceph-mon[35818]: Adjusting osd_memory_target on smithi043 to 8547M 2024-03-26T08:25:35.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:34 smithi043 ceph-mon[31929]: Detected new or changed devices on smithi043 2024-03-26T08:25:35.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:34 smithi043 ceph-mon[31929]: Adjusting osd_memory_target on smithi043 to 8547M 2024-03-26T08:25:36.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.047+0000 7f2b57600700 1 -- 172.21.15.52:0/277785285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b50102c50 msgr2=0x7f2b50105040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:25:36.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.047+0000 7f2b57600700 1 --2- 172.21.15.52:0/277785285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b50102c50 0x7f2b50105040 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009990 tx=0x7f2b44009ca0 comp rx=0 tx=0).stop 2024-03-26T08:25:36.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.047+0000 7f2b57600700 1 -- 172.21.15.52:0/277785285 shutdown_connections 2024-03-26T08:25:36.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.047+0000 7f2b57600700 1 --2- 172.21.15.52:0/277785285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b50107eb0 0x7f2b5010a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:25:36.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.047+0000 7f2b57600700 1 --2- 172.21.15.52:0/277785285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b50102c50 0x7f2b50105040 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:25:36.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.047+0000 7f2b57600700 1 --2- 172.21.15.52:0/277785285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b50105580 0x7f2b50107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:25:36.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.047+0000 7f2b57600700 1 -- 172.21.15.52:0/277785285 >> 172.21.15.52:0/277785285 conn(0x7f2b500fc660 msgr2=0x7f2b500feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:25:36.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.047+0000 7f2b57600700 1 -- 172.21.15.52:0/277785285 shutdown_connections 2024-03-26T08:25:36.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.047+0000 7f2b57600700 1 -- 172.21.15.52:0/277785285 wait complete. 2024-03-26T08:25:36.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.048+0000 7f2b57600700 1 Processor -- start 2024-03-26T08:25:36.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.048+0000 7f2b57600700 1 -- start start 2024-03-26T08:25:36.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.049+0000 7f2b57600700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b50102c50 0x7f2b50102560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:25:36.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.049+0000 7f2b57600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b50105580 0x7f2b50100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:25:36.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.049+0000 7f2b57600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b50107eb0 0x7f2b50101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:25:36.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.049+0000 7f2b57600700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5010dc50 con 0x7f2b50105580 2024-03-26T08:25:36.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.049+0000 7f2b57600700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5010dad0 con 0x7f2b50102c50 2024-03-26T08:25:36.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.049+0000 7f2b57600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5010ddd0 con 0x7f2b50107eb0 2024-03-26T08:25:36.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.049+0000 7f2b54c00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b50102c50 0x7f2b50102560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:25:36.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.049+0000 7f2b54c00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b50102c50 0x7f2b50102560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34912/0 (socket says 172.21.15.52:34912) 2024-03-26T08:25:36.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.049+0000 7f2b54c00700 1 -- 172.21.15.52:0/1474459416 learned_addr learned my addr 172.21.15.52:0/1474459416 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-26T08:25:36.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.049+0000 7f2b4fe00700 1 --2- 172.21.15.52:0/1474459416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b50105580 0x7f2b50100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:25:36.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.049+0000 7f2b55600700 1 --2- 172.21.15.52:0/1474459416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b50107eb0 0x7f2b50101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:25:36.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.049+0000 7f2b54c00700 1 -- 172.21.15.52:0/1474459416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b50107eb0 msgr2=0x7f2b50101140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:25:36.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.049+0000 7f2b54c00700 1 --2- 172.21.15.52:0/1474459416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b50107eb0 0x7f2b50101140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:25:36.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.050+0000 7f2b54c00700 1 -- 172.21.15.52:0/1474459416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b50105580 msgr2=0x7f2b50100c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:25:36.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.050+0000 7f2b54c00700 1 --2- 172.21.15.52:0/1474459416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b50105580 0x7f2b50100c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:25:36.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.050+0000 7f2b54c00700 1 -- 172.21.15.52:0/1474459416 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b50101a10 con 0x7f2b50102c50 2024-03-26T08:25:36.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.050+0000 7f2b54c00700 1 --2- 172.21.15.52:0/1474459416 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b50102c50 0x7f2b50102560 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2b440038b0 tx=0x7f2b440057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:25:36.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.050+0000 7f2b4d600700 1 -- 172.21.15.52:0/1474459416 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44027070 con 0x7f2b50102c50 2024-03-26T08:25:36.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.050+0000 7f2b4d600700 1 -- 172.21.15.52:0/1474459416 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2b44004110 con 0x7f2b50102c50 2024-03-26T08:25:36.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.050+0000 7f2b4d600700 1 -- 172.21.15.52:0/1474459416 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44022e40 con 0x7f2b50102c50 2024-03-26T08:25:36.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.050+0000 7f2b57600700 1 -- 172.21.15.52:0/1474459416 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b501a2490 con 0x7f2b50102c50 2024-03-26T08:25:36.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.050+0000 7f2b57600700 1 -- 172.21.15.52:0/1474459416 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b501a2940 con 0x7f2b50102c50 2024-03-26T08:25:36.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.051+0000 7f2b57600700 1 -- 172.21.15.52:0/1474459416 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b50068e50 con 0x7f2b50102c50 2024-03-26T08:25:36.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.055+0000 7f2b4d600700 1 -- 172.21.15.52:0/1474459416 <== mon.2 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f2b4402a070 con 0x7f2b50102c50 2024-03-26T08:25:36.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.056+0000 7f2b4d600700 1 --2- 172.21.15.52:0/1474459416 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2b380a2890 0x7f2b380a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:25:36.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.057+0000 7f2b4d600700 1 -- 172.21.15.52:0/1474459416 <== mon.2 v2:172.21.15.52:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3669+0+0 (secure 0 0 0) 0x7f2b440cece0 con 0x7f2b50102c50 2024-03-26T08:25:36.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.057+0000 7f2b4fe00700 1 --2- 172.21.15.52:0/1474459416 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2b380a2890 0x7f2b380a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:25:36.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.058+0000 7f2b4fe00700 1 --2- 172.21.15.52:0/1474459416 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2b380a2890 0x7f2b380a4d50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c005fd0 tx=0x7f2b3c005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:25:36.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.060+0000 7f2b4d600700 1 -- 172.21.15.52:0/1474459416 <== mon.2 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2b44013070 con 0x7f2b50102c50 2024-03-26T08:25:36.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:25:36.295+0000 7f2b57600700 1 -- 172.21.15.52:0/1474459416 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b500625d0 con 0x7f2b380a2890 2024-03-26T08:25:36.290 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:35 smithi052 ceph-mon[35071]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:25:36.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:35 smithi183 ceph-mon[35818]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:25:36.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:35 smithi043 ceph-mon[31929]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:25:37.266 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:36 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:25:37.266 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:36 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:25:37.266 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:36 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:25:37.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:36 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:25:37.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:36 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:25:37.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:36 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:25:37.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:36 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:25:37.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:36 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:25:37.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:36 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:25:38.273 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:37 smithi052 ceph-mon[35071]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T08:25:38.274 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:37 smithi052 ceph-mon[35071]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:25:38.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:37 smithi183 ceph-mon[35818]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T08:25:38.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:37 smithi183 ceph-mon[35818]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:25:38.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:37 smithi043 ceph-mon[31929]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T08:25:38.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:37 smithi043 ceph-mon[31929]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:25:40.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:39 smithi183 ceph-mon[35818]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-26T08:25:40.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:39 smithi052 ceph-mon[35071]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-26T08:25:40.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:39 smithi043 ceph-mon[31929]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-26T08:25:41.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:40 smithi183 ceph-mon[35818]: from='client.? 172.21.15.52:0/2725552739' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db089a49-5e98-432d-9687-cdc044114c63"}]: dispatch 2024-03-26T08:25:41.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:40 smithi183 ceph-mon[35818]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db089a49-5e98-432d-9687-cdc044114c63"}]: dispatch 2024-03-26T08:25:41.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:40 smithi183 ceph-mon[35818]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db089a49-5e98-432d-9687-cdc044114c63"}]': finished 2024-03-26T08:25:41.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:40 smithi183 ceph-mon[35818]: osdmap e31: 5 total, 4 up, 5 in 2024-03-26T08:25:41.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:40 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:25:41.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:40 smithi052 ceph-mon[35071]: from='client.? 172.21.15.52:0/2725552739' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db089a49-5e98-432d-9687-cdc044114c63"}]: dispatch 2024-03-26T08:25:41.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:40 smithi052 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db089a49-5e98-432d-9687-cdc044114c63"}]: dispatch 2024-03-26T08:25:41.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:40 smithi052 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db089a49-5e98-432d-9687-cdc044114c63"}]': finished 2024-03-26T08:25:41.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:40 smithi052 ceph-mon[35071]: osdmap e31: 5 total, 4 up, 5 in 2024-03-26T08:25:41.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:40 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:25:41.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:40 smithi043 ceph-mon[31929]: from='client.? 172.21.15.52:0/2725552739' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db089a49-5e98-432d-9687-cdc044114c63"}]: dispatch 2024-03-26T08:25:41.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:40 smithi043 ceph-mon[31929]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db089a49-5e98-432d-9687-cdc044114c63"}]: dispatch 2024-03-26T08:25:41.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:40 smithi043 ceph-mon[31929]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db089a49-5e98-432d-9687-cdc044114c63"}]': finished 2024-03-26T08:25:41.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:40 smithi043 ceph-mon[31929]: osdmap e31: 5 total, 4 up, 5 in 2024-03-26T08:25:41.404 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:40 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:25:42.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:41 smithi183 ceph-mon[35818]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-26T08:25:42.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:41 smithi183 ceph-mon[35818]: from='client.? 172.21.15.52:0/1155951868' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:25:42.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:41 smithi052 ceph-mon[35071]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-26T08:25:42.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:41 smithi052 ceph-mon[35071]: from='client.? 172.21.15.52:0/1155951868' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:25:42.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:41 smithi043 ceph-mon[31929]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-26T08:25:42.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:41 smithi043 ceph-mon[31929]: from='client.? 172.21.15.52:0/1155951868' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:25:44.232 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:43 smithi043 ceph-mon[31929]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-26T08:25:44.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:43 smithi183 ceph-mon[35818]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-26T08:25:44.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:43 smithi052 ceph-mon[35071]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-26T08:25:46.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:45 smithi183 ceph-mon[35818]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T08:25:46.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:45 smithi052 ceph-mon[35071]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T08:25:46.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:45 smithi043 ceph-mon[31929]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T08:25:48.228 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:47 smithi052 ceph-mon[35071]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:25:48.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:47 smithi183 ceph-mon[35818]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:25:48.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:47 smithi043 ceph-mon[31929]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:25:50.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:49 smithi183 ceph-mon[35818]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:25:50.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:49 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-26T08:25:50.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:49 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:25:50.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:49 smithi052 ceph-mon[35071]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:25:50.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:49 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-26T08:25:50.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:49 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:25:50.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:49 smithi043 ceph-mon[31929]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:25:50.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:49 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-26T08:25:50.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:49 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:25:51.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:50 smithi052 ceph-mon[35071]: Deploying daemon osd.4 on smithi052 2024-03-26T08:25:51.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:50 smithi183 ceph-mon[35818]: Deploying daemon osd.4 on smithi052 2024-03-26T08:25:51.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:50 smithi043 ceph-mon[31929]: Deploying daemon osd.4 on smithi052 2024-03-26T08:25:52.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:51 smithi183 ceph-mon[35818]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:25:52.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:51 smithi052 ceph-mon[35071]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:25:52.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:51 smithi043 ceph-mon[31929]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:25:54.296 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:53 smithi052 ceph-mon[35071]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:25:54.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:53 smithi183 ceph-mon[35818]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:25:54.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:53 smithi043 ceph-mon[31929]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:25:56.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:56 smithi183 ceph-mon[35818]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:25:56.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:56 smithi052 ceph-mon[35071]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:25:56.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:55 smithi043 ceph-mon[31929]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:25:58.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:58 smithi183 ceph-mon[35818]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:25:58.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:58 smithi052 ceph-mon[35071]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:25:58.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:58 smithi043 ceph-mon[31929]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:25:59.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:59 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:59.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:59 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:59.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:25:59 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:25:59.422 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:59 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:59.422 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:59 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:59.422 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:25:59 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:25:59.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:59 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:59.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:59 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:25:59.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:25:59 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:26:00.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:00 smithi043 ceph-mon[31929]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:26:00.473 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:00 smithi052 ceph-mon[35071]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:26:00.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:00 smithi183 ceph-mon[35818]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:26:01.432 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:01 smithi052 ceph-mon[35071]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:26:01.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:01 smithi183 ceph-mon[35818]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:26:01.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:01 smithi043 ceph-mon[31929]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:26:02.799 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:02 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:02.800 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:02 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:02.800 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:02 smithi052 ceph-mon[35071]: from='osd.4 [v2:172.21.15.52:6808/4028436523,v1:172.21.15.52:6809/4028436523]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T08:26:02.800 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:02 smithi052 ceph-mon[35071]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T08:26:03.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:26:03.018+0000 7f2b4d600700 1 -- 172.21.15.52:0/1474459416 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2b500625d0 con 0x7f2b380a2890 2024-03-26T08:26:03.013 INFO:teuthology.orchestra.run.smithi052.stdout:Created osd(s) 4 on host 'smithi052' 2024-03-26T08:26:03.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:26:03.021+0000 7f2b57600700 1 -- 172.21.15.52:0/1474459416 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2b380a2890 msgr2=0x7f2b380a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:26:03.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:26:03.021+0000 7f2b57600700 1 --2- 172.21.15.52:0/1474459416 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2b380a2890 0x7f2b380a4d50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c005fd0 tx=0x7f2b3c005f60 comp rx=0 tx=0).stop 2024-03-26T08:26:03.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:26:03.021+0000 7f2b57600700 1 -- 172.21.15.52:0/1474459416 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b50102c50 msgr2=0x7f2b50102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:26:03.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:26:03.021+0000 7f2b57600700 1 --2- 172.21.15.52:0/1474459416 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b50102c50 0x7f2b50102560 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2b440038b0 tx=0x7f2b440057c0 comp rx=0 tx=0).stop 2024-03-26T08:26:03.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:26:03.022+0000 7f2b57600700 1 -- 172.21.15.52:0/1474459416 shutdown_connections 2024-03-26T08:26:03.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:26:03.022+0000 7f2b57600700 1 --2- 172.21.15.52:0/1474459416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b50107eb0 0x7f2b50101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:26:03.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:26:03.022+0000 7f2b57600700 1 --2- 172.21.15.52:0/1474459416 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2b380a2890 0x7f2b380a4d50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:26:03.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:26:03.022+0000 7f2b57600700 1 --2- 172.21.15.52:0/1474459416 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b50102c50 0x7f2b50102560 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:26:03.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:26:03.022+0000 7f2b57600700 1 --2- 172.21.15.52:0/1474459416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b50105580 0x7f2b50100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:26:03.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:26:03.022+0000 7f2b57600700 1 -- 172.21.15.52:0/1474459416 >> 172.21.15.52:0/1474459416 conn(0x7f2b500fc660 msgr2=0x7f2b501036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:26:03.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:26:03.022+0000 7f2b57600700 1 -- 172.21.15.52:0/1474459416 shutdown_connections 2024-03-26T08:26:03.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:26:03.022+0000 7f2b57600700 1 -- 172.21.15.52:0/1474459416 wait complete. 2024-03-26T08:26:03.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:02 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:03.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:02 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:03.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:02 smithi183 ceph-mon[35818]: from='osd.4 [v2:172.21.15.52:6808/4028436523,v1:172.21.15.52:6809/4028436523]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T08:26:03.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:02 smithi183 ceph-mon[35818]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T08:26:03.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:02 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:03.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:02 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:03.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:02 smithi043 ceph-mon[31929]: from='osd.4 [v2:172.21.15.52:6808/4028436523,v1:172.21.15.52:6809/4028436523]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T08:26:03.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:02 smithi043 ceph-mon[31929]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T08:26:03.696 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:03 smithi052 ceph-mon[35071]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:26:03.696 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:03 smithi052 ceph-mon[35071]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-26T08:26:03.696 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:03 smithi052 ceph-mon[35071]: from='osd.4 [v2:172.21.15.52:6808/4028436523,v1:172.21.15.52:6809/4028436523]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-26T08:26:03.697 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:03 smithi052 ceph-mon[35071]: osdmap e32: 5 total, 4 up, 5 in 2024-03-26T08:26:03.697 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:03 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:26:03.697 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:03 smithi052 ceph-mon[35071]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-26T08:26:03.697 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:03 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:26:03.697 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:03 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:26:03.697 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:03 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:03.698 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:03 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:03.698 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:03 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:03.698 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:03 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:26:03.864 DEBUG:teuthology.orchestra.run.smithi052:osd.4> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.4.service 2024-03-26T08:26:03.871 INFO:tasks.cephadm:Deploying osd.5 on smithi183 with /dev/vg_nvme/lv_3... 2024-03-26T08:26:03.871 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-26T08:26:03.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:03 smithi183 ceph-mon[35818]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:26:03.954 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:03 smithi183 ceph-mon[35818]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-26T08:26:03.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:03 smithi183 ceph-mon[35818]: from='osd.4 [v2:172.21.15.52:6808/4028436523,v1:172.21.15.52:6809/4028436523]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-26T08:26:03.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:03 smithi183 ceph-mon[35818]: osdmap e32: 5 total, 4 up, 5 in 2024-03-26T08:26:03.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:03 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:26:03.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:03 smithi183 ceph-mon[35818]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-26T08:26:03.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:03 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:26:03.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:03 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:26:03.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:03 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:03.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:03 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:03.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:03 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:03.955 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:03 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:26:04.055 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:03 smithi043 ceph-mon[31929]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:26:04.055 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:03 smithi043 ceph-mon[31929]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-26T08:26:04.055 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:03 smithi043 ceph-mon[31929]: from='osd.4 [v2:172.21.15.52:6808/4028436523,v1:172.21.15.52:6809/4028436523]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-26T08:26:04.055 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:03 smithi043 ceph-mon[31929]: osdmap e32: 5 total, 4 up, 5 in 2024-03-26T08:26:04.055 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:03 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:26:04.055 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:03 smithi043 ceph-mon[31929]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-26T08:26:04.055 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:03 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:26:04.055 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:03 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:26:04.056 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:03 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:04.056 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:03 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:04.056 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:03 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:04.056 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:03 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:26:04.304 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:26:04.656 INFO:journalctl@ceph.osd.4.smithi052.stdout:Mar 26 08:26:04 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-4[46638]: 2024-03-26T08:26:04.396+0000 7f3e15400700 -1 osd.4 0 waiting for initial osdmap 2024-03-26T08:26:04.657 INFO:journalctl@ceph.osd.4.smithi052.stdout:Mar 26 08:26:04 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-4[46638]: 2024-03-26T08:26:04.411+0000 7f3e0b200700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T08:26:04.996 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:04 smithi052 ceph-mon[35071]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-03-26T08:26:04.997 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:04 smithi052 ceph-mon[35071]: osdmap e33: 5 total, 4 up, 5 in 2024-03-26T08:26:04.997 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:04 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:26:04.997 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:04 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:26:04.997 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:04 smithi052 ceph-mon[35071]: from='osd.4 ' entity='osd.4' 2024-03-26T08:26:05.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:04 smithi183 ceph-mon[35818]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-03-26T08:26:05.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:04 smithi183 ceph-mon[35818]: osdmap e33: 5 total, 4 up, 5 in 2024-03-26T08:26:05.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:04 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:26:05.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:04 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:26:05.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:04 smithi183 ceph-mon[35818]: from='osd.4 ' entity='osd.4' 2024-03-26T08:26:05.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:04 smithi043 ceph-mon[31929]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-03-26T08:26:05.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:04 smithi043 ceph-mon[31929]: osdmap e33: 5 total, 4 up, 5 in 2024-03-26T08:26:05.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:04 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:26:05.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:04 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:26:05.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:04 smithi043 ceph-mon[31929]: from='osd.4 ' entity='osd.4' 2024-03-26T08:26:05.931 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:05 smithi183 ceph-mon[35818]: purged_snaps scrub starts 2024-03-26T08:26:05.932 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:05 smithi183 ceph-mon[35818]: purged_snaps scrub ok 2024-03-26T08:26:05.932 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:05 smithi183 ceph-mon[35818]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:26:05.932 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:05 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:26:05.932 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:05 smithi183 ceph-mon[35818]: osd.4 [v2:172.21.15.52:6808/4028436523,v1:172.21.15.52:6809/4028436523] boot 2024-03-26T08:26:05.932 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:05 smithi183 ceph-mon[35818]: osdmap e34: 5 total, 5 up, 5 in 2024-03-26T08:26:05.932 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:05 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:26:06.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:05 smithi052 ceph-mon[35071]: purged_snaps scrub starts 2024-03-26T08:26:06.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:05 smithi052 ceph-mon[35071]: purged_snaps scrub ok 2024-03-26T08:26:06.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:05 smithi052 ceph-mon[35071]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:26:06.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:05 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:26:06.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:05 smithi052 ceph-mon[35071]: osd.4 [v2:172.21.15.52:6808/4028436523,v1:172.21.15.52:6809/4028436523] boot 2024-03-26T08:26:06.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:05 smithi052 ceph-mon[35071]: osdmap e34: 5 total, 5 up, 5 in 2024-03-26T08:26:06.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:05 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:26:06.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:05 smithi043 ceph-mon[31929]: purged_snaps scrub starts 2024-03-26T08:26:06.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:05 smithi043 ceph-mon[31929]: purged_snaps scrub ok 2024-03-26T08:26:06.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:05 smithi043 ceph-mon[31929]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-26T08:26:06.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:05 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:26:06.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:05 smithi043 ceph-mon[31929]: osd.4 [v2:172.21.15.52:6808/4028436523,v1:172.21.15.52:6809/4028436523] boot 2024-03-26T08:26:06.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:05 smithi043 ceph-mon[31929]: osdmap e34: 5 total, 5 up, 5 in 2024-03-26T08:26:06.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:05 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:26:06.949 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-26T08:26:06.949 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-26T08:26:06.949 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-03-26T08:26:06.949 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-03-26T08:26:06.949 INFO:teuthology.orchestra.run.smithi183.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0378652 s, 277 MB/s 2024-03-26T08:26:06.950 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-03-26T08:26:07.748 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi183:vg_nvme/lv_3 2024-03-26T08:26:08.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:07 smithi183 ceph-mon[35818]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:08.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:07 smithi183 ceph-mon[35818]: osdmap e35: 5 total, 5 up, 5 in 2024-03-26T08:26:08.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:07 smithi052 ceph-mon[35071]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:08.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:07 smithi052 ceph-mon[35071]: osdmap e35: 5 total, 5 up, 5 in 2024-03-26T08:26:08.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:07 smithi043 ceph-mon[31929]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:08.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:07 smithi043 ceph-mon[31929]: osdmap e35: 5 total, 5 up, 5 in 2024-03-26T08:26:08.184 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:26:09.904 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:09 smithi183 ceph-mon[35818]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:09.904 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:09 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:09.904 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:09 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:09.905 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:09 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:26:09.905 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:09 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:26:09.905 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:09 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:09.905 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:09 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:26:09.905 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:09 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:26:09.905 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:09 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:10.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:09 smithi052 ceph-mon[35071]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:10.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:09 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:10.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:09 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:10.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:09 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:26:10.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:09 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:26:10.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:09 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:10.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:09 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:26:10.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:09 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:26:10.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:09 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:10.075 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.081+0000 7f0e88800700 1 -- 172.21.15.183:0/102925525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e80109a10 msgr2=0x7f0e80109e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:26:10.075 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.081+0000 7f0e88800700 1 --2- 172.21.15.183:0/102925525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e80109a10 0x7f0e80109e70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009990 tx=0x7f0e70009ca0 comp rx=0 tx=0).stop 2024-03-26T08:26:10.075 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.081+0000 7f0e88800700 1 -- 172.21.15.183:0/102925525 shutdown_connections 2024-03-26T08:26:10.075 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.081+0000 7f0e88800700 1 --2- 172.21.15.183:0/102925525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e8010a3b0 0x7f0e80110c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:26:10.075 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.081+0000 7f0e88800700 1 --2- 172.21.15.183:0/102925525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e80108810 0x7f0e80108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:26:10.075 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.081+0000 7f0e88800700 1 --2- 172.21.15.183:0/102925525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e80109a10 0x7f0e80109e70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:26:10.076 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.081+0000 7f0e88800700 1 -- 172.21.15.183:0/102925525 >> 172.21.15.183:0/102925525 conn(0x7f0e800777e0 msgr2=0x7f0e80079c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:26:10.076 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.081+0000 7f0e88800700 1 -- 172.21.15.183:0/102925525 shutdown_connections 2024-03-26T08:26:10.076 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.081+0000 7f0e88800700 1 -- 172.21.15.183:0/102925525 wait complete. 2024-03-26T08:26:10.076 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.082+0000 7f0e88800700 1 Processor -- start 2024-03-26T08:26:10.076 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.082+0000 7f0e88800700 1 -- start start 2024-03-26T08:26:10.077 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.082+0000 7f0e88800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e80108810 0x7f0e8019e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:26:10.077 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.082+0000 7f0e88800700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e80109a10 0x7f0e8019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:26:10.077 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.083+0000 7f0e88800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e8010a3b0 0x7f0e801a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:26:10.077 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.083+0000 7f0e88800700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e801142d0 con 0x7f0e80108810 2024-03-26T08:26:10.078 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.083+0000 7f0e88800700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e80114150 con 0x7f0e80109a10 2024-03-26T08:26:10.078 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.083+0000 7f0e88800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e80114450 con 0x7f0e8010a3b0 2024-03-26T08:26:10.078 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.083+0000 7f0e86800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e8010a3b0 0x7f0e801a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:26:10.078 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.083+0000 7f0e86800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e8010a3b0 0x7f0e801a5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:55748/0 (socket says 172.21.15.183:55748) 2024-03-26T08:26:10.078 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.083+0000 7f0e86800700 1 -- 172.21.15.183:0/2079853606 learned_addr learned my addr 172.21.15.183:0/2079853606 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-03-26T08:26:10.078 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.083+0000 7f0e85e00700 1 --2- 172.21.15.183:0/2079853606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e80108810 0x7f0e8019e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:26:10.078 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.083+0000 7f0e85400700 1 --2- 172.21.15.183:0/2079853606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e80109a10 0x7f0e8019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:26:10.078 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.083+0000 7f0e86800700 1 -- 172.21.15.183:0/2079853606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e80109a10 msgr2=0x7f0e8019ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:26:10.078 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.083+0000 7f0e86800700 1 --2- 172.21.15.183:0/2079853606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e80109a10 0x7f0e8019ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:26:10.078 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.083+0000 7f0e86800700 1 -- 172.21.15.183:0/2079853606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e80108810 msgr2=0x7f0e8019e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:26:10.078 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.083+0000 7f0e86800700 1 --2- 172.21.15.183:0/2079853606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e80108810 0x7f0e8019e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:26:10.078 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.083+0000 7f0e86800700 1 -- 172.21.15.183:0/2079853606 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e801a6570 con 0x7f0e8010a3b0 2024-03-26T08:26:10.078 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.084+0000 7f0e86800700 1 --2- 172.21.15.183:0/2079853606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e8010a3b0 0x7f0e801a5ed0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00add0 tx=0x7f0e7c00c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:26:10.078 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.084+0000 7f0e7aa00700 1 -- 172.21.15.183:0/2079853606 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c017020 con 0x7f0e8010a3b0 2024-03-26T08:26:10.079 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.084+0000 7f0e7aa00700 1 -- 172.21.15.183:0/2079853606 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0e7c00cdf0 con 0x7f0e8010a3b0 2024-03-26T08:26:10.079 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.084+0000 7f0e88800700 1 -- 172.21.15.183:0/2079853606 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e801a6860 con 0x7f0e8010a3b0 2024-03-26T08:26:10.079 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.084+0000 7f0e7aa00700 1 -- 172.21.15.183:0/2079853606 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c005460 con 0x7f0e8010a3b0 2024-03-26T08:26:10.079 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.084+0000 7f0e88800700 1 -- 172.21.15.183:0/2079853606 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e801a6df0 con 0x7f0e8010a3b0 2024-03-26T08:26:10.079 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.085+0000 7f0e88800700 1 -- 172.21.15.183:0/2079853606 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e80068e50 con 0x7f0e8010a3b0 2024-03-26T08:26:10.080 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.087+0000 7f0e7aa00700 1 -- 172.21.15.183:0/2079853606 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0e7c004ad0 con 0x7f0e8010a3b0 2024-03-26T08:26:10.083 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.090+0000 7f0e7aa00700 1 --2- 172.21.15.183:0/2079853606 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0e680a2910 0x7f0e680a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:26:10.083 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.090+0000 7f0e7aa00700 1 -- 172.21.15.183:0/2079853606 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f0e7c0cd1c0 con 0x7f0e8010a3b0 2024-03-26T08:26:10.083 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.090+0000 7f0e85e00700 1 --2- 172.21.15.183:0/2079853606 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0e680a2910 0x7f0e680a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:26:10.085 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.091+0000 7f0e85e00700 1 --2- 172.21.15.183:0/2079853606 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0e680a2910 0x7f0e680a4dd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0e74005fd0 tx=0x7f0e74005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:26:10.089 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.095+0000 7f0e7aa00700 1 -- 172.21.15.183:0/2079853606 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0e7c021020 con 0x7f0e8010a3b0 2024-03-26T08:26:10.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:09 smithi043 ceph-mon[31929]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:10.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:09 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:10.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:09 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:10.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:09 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:26:10.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:09 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:26:10.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:09 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:10.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:09 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:26:10.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:09 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:26:10.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:09 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:10.316 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:10.323+0000 7f0e88800700 1 -- 172.21.15.183:0/2079853606 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e800625d0 con 0x7f0e680a2910 2024-03-26T08:26:11.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:10 smithi183 ceph-mon[35818]: Detected new or changed devices on smithi052 2024-03-26T08:26:11.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:10 smithi183 ceph-mon[35818]: Adjusting osd_memory_target on smithi052 to 8547M 2024-03-26T08:26:11.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:10 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:26:11.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:10 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:26:11.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:10 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:26:11.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:10 smithi052 ceph-mon[35071]: Detected new or changed devices on smithi052 2024-03-26T08:26:11.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:10 smithi052 ceph-mon[35071]: Adjusting osd_memory_target on smithi052 to 8547M 2024-03-26T08:26:11.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:10 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:26:11.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:10 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:26:11.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:10 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:26:11.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:10 smithi043 ceph-mon[31929]: Detected new or changed devices on smithi052 2024-03-26T08:26:11.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:10 smithi043 ceph-mon[31929]: Adjusting osd_memory_target on smithi052 to 8547M 2024-03-26T08:26:11.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:10 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:26:11.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:10 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:26:11.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:10 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:26:12.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:11 smithi183 ceph-mon[35818]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:12.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:11 smithi183 ceph-mon[35818]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:26:12.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:11 smithi052 ceph-mon[35071]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:12.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:11 smithi052 ceph-mon[35071]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:26:12.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:11 smithi043 ceph-mon[31929]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:12.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:11 smithi043 ceph-mon[31929]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:26:14.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:13 smithi183 ceph-mon[35818]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:14.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:13 smithi052 ceph-mon[35071]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:14.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:13 smithi043 ceph-mon[31929]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:15.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:14 smithi183 ceph-mon[35818]: from='client.? 172.21.15.183:0/1180167841' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "556ac804-02ea-423b-a599-55baf5be4ca1"}]: dispatch 2024-03-26T08:26:15.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:14 smithi183 ceph-mon[35818]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "556ac804-02ea-423b-a599-55baf5be4ca1"}]: dispatch 2024-03-26T08:26:15.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:14 smithi183 ceph-mon[35818]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "556ac804-02ea-423b-a599-55baf5be4ca1"}]': finished 2024-03-26T08:26:15.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:14 smithi183 ceph-mon[35818]: osdmap e36: 6 total, 5 up, 6 in 2024-03-26T08:26:15.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:14 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:26:15.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:14 smithi052 ceph-mon[35071]: from='client.? 172.21.15.183:0/1180167841' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "556ac804-02ea-423b-a599-55baf5be4ca1"}]: dispatch 2024-03-26T08:26:15.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:14 smithi052 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "556ac804-02ea-423b-a599-55baf5be4ca1"}]: dispatch 2024-03-26T08:26:15.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:14 smithi052 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "556ac804-02ea-423b-a599-55baf5be4ca1"}]': finished 2024-03-26T08:26:15.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:14 smithi052 ceph-mon[35071]: osdmap e36: 6 total, 5 up, 6 in 2024-03-26T08:26:15.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:14 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:26:15.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:14 smithi043 ceph-mon[31929]: from='client.? 172.21.15.183:0/1180167841' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "556ac804-02ea-423b-a599-55baf5be4ca1"}]: dispatch 2024-03-26T08:26:15.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:14 smithi043 ceph-mon[31929]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "556ac804-02ea-423b-a599-55baf5be4ca1"}]: dispatch 2024-03-26T08:26:15.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:14 smithi043 ceph-mon[31929]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "556ac804-02ea-423b-a599-55baf5be4ca1"}]': finished 2024-03-26T08:26:15.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:14 smithi043 ceph-mon[31929]: osdmap e36: 6 total, 5 up, 6 in 2024-03-26T08:26:15.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:14 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:26:16.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:15 smithi183 ceph-mon[35818]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:16.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:15 smithi183 ceph-mon[35818]: from='client.? 172.21.15.183:0/2073723623' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:26:16.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:15 smithi052 ceph-mon[35071]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:16.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:15 smithi052 ceph-mon[35071]: from='client.? 172.21.15.183:0/2073723623' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:26:16.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:15 smithi043 ceph-mon[31929]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:16.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:15 smithi043 ceph-mon[31929]: from='client.? 172.21.15.183:0/2073723623' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:26:18.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:17 smithi183 ceph-mon[35818]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:18.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:17 smithi052 ceph-mon[35071]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:18.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:17 smithi043 ceph-mon[31929]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:20.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:19 smithi183 ceph-mon[35818]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:20.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:19 smithi052 ceph-mon[35071]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:20.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:19 smithi043 ceph-mon[31929]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:22.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:21 smithi183 ceph-mon[35818]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:22.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:21 smithi052 ceph-mon[35071]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:22.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:21 smithi043 ceph-mon[31929]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:24.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:23 smithi183 ceph-mon[35818]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:24.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:23 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-26T08:26:24.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:23 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:26:24.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:23 smithi052 ceph-mon[35071]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:24.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:23 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-26T08:26:24.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:23 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:26:24.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:23 smithi043 ceph-mon[31929]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:24.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:23 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-26T08:26:24.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:23 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:26:24.900 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:24 smithi183 ceph-mon[35818]: Deploying daemon osd.5 on smithi183 2024-03-26T08:26:25.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:24 smithi052 ceph-mon[35071]: Deploying daemon osd.5 on smithi183 2024-03-26T08:26:25.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:24 smithi043 ceph-mon[31929]: Deploying daemon osd.5 on smithi183 2024-03-26T08:26:25.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:25 smithi183 ceph-mon[35818]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:26.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:25 smithi052 ceph-mon[35071]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:26.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:25 smithi043 ceph-mon[31929]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:28.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:27 smithi183 ceph-mon[35818]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:28.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:27 smithi052 ceph-mon[35071]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:28.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:27 smithi043 ceph-mon[31929]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:30.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:29 smithi052 ceph-mon[35071]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:30.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:29 smithi043 ceph-mon[31929]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:30.292 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:29 smithi183 ceph-mon[35818]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:32.318 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:32 smithi183 ceph-mon[35818]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:32.319 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:32 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:32.319 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:32 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:32.319 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:32 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:26:32.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:32 smithi043 ceph-mon[31929]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:32.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:32 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:32.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:32 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:32.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:32 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:26:32.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:32 smithi052 ceph-mon[35071]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:32.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:32 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:32.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:32 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:32.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:32 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:26:34.278 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:34 smithi052 ceph-mon[35071]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:34.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:34 smithi043 ceph-mon[31929]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:34.408 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:34 smithi183 ceph-mon[35818]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:35.896 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:35 smithi183 ceph-mon[35818]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:35.896 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:35 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:35.896 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:35 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:35.896 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:35 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:26:35.896 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:35 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:26:35.896 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:35 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:35.896 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:35 smithi183 ceph-mon[35818]: from='osd.5 [v2:172.21.15.183:6808/787132252,v1:172.21.15.183:6809/787132252]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T08:26:35.896 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:35 smithi183 ceph-mon[35818]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T08:26:36.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:36.036+0000 7f0e7aa00700 1 -- 172.21.15.183:0/2079853606 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0e800625d0 con 0x7f0e680a2910 2024-03-26T08:26:36.031 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 5 on host 'smithi183' 2024-03-26T08:26:36.034 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:36.040+0000 7f0e88800700 1 -- 172.21.15.183:0/2079853606 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0e680a2910 msgr2=0x7f0e680a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:26:36.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:36.040+0000 7f0e88800700 1 --2- 172.21.15.183:0/2079853606 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0e680a2910 0x7f0e680a4dd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0e74005fd0 tx=0x7f0e74005dd0 comp rx=0 tx=0).stop 2024-03-26T08:26:36.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:36.040+0000 7f0e88800700 1 -- 172.21.15.183:0/2079853606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e8010a3b0 msgr2=0x7f0e801a5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:26:36.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:36.040+0000 7f0e88800700 1 --2- 172.21.15.183:0/2079853606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e8010a3b0 0x7f0e801a5ed0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00add0 tx=0x7f0e7c00c330 comp rx=0 tx=0).stop 2024-03-26T08:26:36.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:36.040+0000 7f0e88800700 1 -- 172.21.15.183:0/2079853606 shutdown_connections 2024-03-26T08:26:36.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:36.040+0000 7f0e88800700 1 --2- 172.21.15.183:0/2079853606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e8010a3b0 0x7f0e801a5ed0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:26:36.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:36.040+0000 7f0e88800700 1 --2- 172.21.15.183:0/2079853606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e80108810 0x7f0e8019e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:26:36.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:36.040+0000 7f0e88800700 1 --2- 172.21.15.183:0/2079853606 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0e680a2910 0x7f0e680a4dd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:26:36.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:36.040+0000 7f0e88800700 1 --2- 172.21.15.183:0/2079853606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e80109a10 0x7f0e8019ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:26:36.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:36.040+0000 7f0e88800700 1 -- 172.21.15.183:0/2079853606 >> 172.21.15.183:0/2079853606 conn(0x7f0e800777e0 msgr2=0x7f0e8010cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:26:36.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:36.040+0000 7f0e88800700 1 -- 172.21.15.183:0/2079853606 shutdown_connections 2024-03-26T08:26:36.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:26:36.040+0000 7f0e88800700 1 -- 172.21.15.183:0/2079853606 wait complete. 2024-03-26T08:26:36.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:35 smithi052 ceph-mon[35071]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:36.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:35 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:36.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:35 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:36.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:35 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:26:36.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:35 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:26:36.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:35 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:36.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:35 smithi052 ceph-mon[35071]: from='osd.5 [v2:172.21.15.183:6808/787132252,v1:172.21.15.183:6809/787132252]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T08:26:36.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:35 smithi052 ceph-mon[35071]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T08:26:36.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:35 smithi043 ceph-mon[31929]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:36.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:35 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:36.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:35 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:36.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:35 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:26:36.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:35 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:26:36.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:35 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:36.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:35 smithi043 ceph-mon[31929]: from='osd.5 [v2:172.21.15.183:6808/787132252,v1:172.21.15.183:6809/787132252]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T08:26:36.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:35 smithi043 ceph-mon[31929]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T08:26:36.786 DEBUG:teuthology.orchestra.run.smithi183:osd.5> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.5.service 2024-03-26T08:26:36.791 INFO:tasks.cephadm:Deploying osd.6 on smithi043 with /dev/vg_nvme/lv_2... 2024-03-26T08:26:36.791 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-26T08:26:37.029 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:36 smithi183 ceph-mon[35818]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-26T08:26:37.029 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:36 smithi183 ceph-mon[35818]: osdmap e37: 6 total, 5 up, 6 in 2024-03-26T08:26:37.029 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:36 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:26:37.029 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:36 smithi183 ceph-mon[35818]: from='osd.5 [v2:172.21.15.183:6808/787132252,v1:172.21.15.183:6809/787132252]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-26T08:26:37.029 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:36 smithi183 ceph-mon[35818]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-26T08:26:37.029 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:36 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:37.029 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:36 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:37.029 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:36 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:26:37.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:36 smithi052 ceph-mon[35071]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-26T08:26:37.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:36 smithi052 ceph-mon[35071]: osdmap e37: 6 total, 5 up, 6 in 2024-03-26T08:26:37.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:36 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:26:37.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:36 smithi052 ceph-mon[35071]: from='osd.5 [v2:172.21.15.183:6808/787132252,v1:172.21.15.183:6809/787132252]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-26T08:26:37.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:36 smithi052 ceph-mon[35071]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-26T08:26:37.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:36 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:37.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:36 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:37.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:36 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:26:37.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:36 smithi043 ceph-mon[31929]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-26T08:26:37.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:36 smithi043 ceph-mon[31929]: osdmap e37: 6 total, 5 up, 6 in 2024-03-26T08:26:37.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:36 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:26:37.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:36 smithi043 ceph-mon[31929]: from='osd.5 [v2:172.21.15.183:6808/787132252,v1:172.21.15.183:6809/787132252]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-26T08:26:37.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:36 smithi043 ceph-mon[31929]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-26T08:26:37.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:36 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:37.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:36 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:37.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:36 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:26:37.246 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:26:37.574 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:26:37 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:26:37.401+0000 7f32fb200700 -1 osd.5 0 waiting for initial osdmap 2024-03-26T08:26:37.574 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:26:37 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:26:37.423+0000 7f32f0600700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T08:26:37.830 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:37 smithi183 ceph-mon[35818]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:37.830 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:37 smithi183 ceph-mon[35818]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-03-26T08:26:37.830 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:37 smithi183 ceph-mon[35818]: osdmap e38: 6 total, 5 up, 6 in 2024-03-26T08:26:37.830 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:37 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:26:37.830 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:37 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:26:37.830 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:37 smithi183 ceph-mon[35818]: from='osd.5 ' entity='osd.5' 2024-03-26T08:26:38.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:37 smithi052 ceph-mon[35071]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:38.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:37 smithi052 ceph-mon[35071]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-03-26T08:26:38.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:37 smithi052 ceph-mon[35071]: osdmap e38: 6 total, 5 up, 6 in 2024-03-26T08:26:38.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:37 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:26:38.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:37 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:26:38.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:37 smithi052 ceph-mon[35071]: from='osd.5 ' entity='osd.5' 2024-03-26T08:26:38.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:37 smithi043 ceph-mon[31929]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:38.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:37 smithi043 ceph-mon[31929]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-03-26T08:26:38.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:37 smithi043 ceph-mon[31929]: osdmap e38: 6 total, 5 up, 6 in 2024-03-26T08:26:38.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:37 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:26:38.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:37 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:26:38.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:37 smithi043 ceph-mon[31929]: from='osd.5 ' entity='osd.5' 2024-03-26T08:26:39.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:38 smithi183 ceph-mon[35818]: purged_snaps scrub starts 2024-03-26T08:26:39.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:38 smithi183 ceph-mon[35818]: purged_snaps scrub ok 2024-03-26T08:26:39.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:38 smithi183 ceph-mon[35818]: osd.5 [v2:172.21.15.183:6808/787132252,v1:172.21.15.183:6809/787132252] boot 2024-03-26T08:26:39.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:38 smithi183 ceph-mon[35818]: osdmap e39: 6 total, 6 up, 6 in 2024-03-26T08:26:39.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:38 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:26:39.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:38 smithi183 ceph-mon[35818]: osdmap e40: 6 total, 6 up, 6 in 2024-03-26T08:26:39.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:38 smithi052 ceph-mon[35071]: purged_snaps scrub starts 2024-03-26T08:26:39.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:38 smithi052 ceph-mon[35071]: purged_snaps scrub ok 2024-03-26T08:26:39.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:38 smithi052 ceph-mon[35071]: osd.5 [v2:172.21.15.183:6808/787132252,v1:172.21.15.183:6809/787132252] boot 2024-03-26T08:26:39.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:38 smithi052 ceph-mon[35071]: osdmap e39: 6 total, 6 up, 6 in 2024-03-26T08:26:39.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:38 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:26:39.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:38 smithi052 ceph-mon[35071]: osdmap e40: 6 total, 6 up, 6 in 2024-03-26T08:26:39.113 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:38 smithi043 ceph-mon[31929]: purged_snaps scrub starts 2024-03-26T08:26:39.113 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:38 smithi043 ceph-mon[31929]: purged_snaps scrub ok 2024-03-26T08:26:39.113 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:38 smithi043 ceph-mon[31929]: osd.5 [v2:172.21.15.183:6808/787132252,v1:172.21.15.183:6809/787132252] boot 2024-03-26T08:26:39.113 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:38 smithi043 ceph-mon[31929]: osdmap e39: 6 total, 6 up, 6 in 2024-03-26T08:26:39.113 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:38 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:26:39.113 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:38 smithi043 ceph-mon[31929]: osdmap e40: 6 total, 6 up, 6 in 2024-03-26T08:26:40.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:39 smithi043 ceph-mon[31929]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:40.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:39 smithi043 ceph-mon[31929]: osdmap e41: 6 total, 6 up, 6 in 2024-03-26T08:26:40.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:39 smithi183 ceph-mon[35818]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:40.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:39 smithi183 ceph-mon[35818]: osdmap e41: 6 total, 6 up, 6 in 2024-03-26T08:26:40.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:39 smithi052 ceph-mon[35071]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:26:40.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:39 smithi052 ceph-mon[35071]: osdmap e41: 6 total, 6 up, 6 in 2024-03-26T08:26:40.362 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-26T08:26:40.362 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-26T08:26:40.362 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2024-03-26T08:26:40.362 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2024-03-26T08:26:40.362 INFO:teuthology.orchestra.run.smithi043.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0359352 s, 292 MB/s 2024-03-26T08:26:40.362 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 2024-03-26T08:26:40.362 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2024-03-26T08:26:41.260 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi043:vg_nvme/lv_2 2024-03-26T08:26:41.721 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:26:42.088 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:41 smithi043 ceph-mon[31929]: pgmap v183: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:26:42.089 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:41 smithi043 ceph-mon[31929]: osdmap e42: 6 total, 6 up, 6 in 2024-03-26T08:26:42.089 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:42.089 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:42.089 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:26:42.089 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:26:42.090 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:42.090 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:26:42.090 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:26:42.090 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:42.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:41 smithi183 ceph-mon[35818]: pgmap v183: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:26:42.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:41 smithi183 ceph-mon[35818]: osdmap e42: 6 total, 6 up, 6 in 2024-03-26T08:26:42.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:42.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:42.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:26:42.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:26:42.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:42.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:26:42.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:26:42.301 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:42.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:41 smithi052 ceph-mon[35071]: pgmap v183: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:26:42.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:41 smithi052 ceph-mon[35071]: osdmap e42: 6 total, 6 up, 6 in 2024-03-26T08:26:42.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:42.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:42.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:26:42.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:26:42.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:42.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:26:42.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:26:42.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:26:43.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:42 smithi043 ceph-mon[31929]: Detected new or changed devices on smithi183 2024-03-26T08:26:43.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:42 smithi043 ceph-mon[31929]: Adjusting osd_memory_target on smithi183 to 8545M 2024-03-26T08:26:43.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:42 smithi043 ceph-mon[31929]: osdmap e43: 6 total, 6 up, 6 in 2024-03-26T08:26:43.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:42 smithi183 ceph-mon[35818]: Detected new or changed devices on smithi183 2024-03-26T08:26:43.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:42 smithi183 ceph-mon[35818]: Adjusting osd_memory_target on smithi183 to 8545M 2024-03-26T08:26:43.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:42 smithi183 ceph-mon[35818]: osdmap e43: 6 total, 6 up, 6 in 2024-03-26T08:26:43.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:42 smithi052 ceph-mon[35071]: Detected new or changed devices on smithi183 2024-03-26T08:26:43.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:42 smithi052 ceph-mon[35071]: Adjusting osd_memory_target on smithi183 to 8545M 2024-03-26T08:26:43.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:42 smithi052 ceph-mon[35071]: osdmap e43: 6 total, 6 up, 6 in 2024-03-26T08:26:44.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.102+0000 7fc665400700 1 -- 172.21.15.43:0/2389231452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc660104010 msgr2=0x7fc660104470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:26:44.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.102+0000 7fc665400700 1 --2- 172.21.15.43:0/2389231452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc660104010 0x7fc660104470 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc648009990 tx=0x7fc648009ca0 comp rx=0 tx=0).stop 2024-03-26T08:26:44.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.102+0000 7fc665400700 1 -- 172.21.15.43:0/2389231452 shutdown_connections 2024-03-26T08:26:44.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.102+0000 7fc665400700 1 --2- 172.21.15.43:0/2389231452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6601049b0 0x7fc66010c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:26:44.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.102+0000 7fc665400700 1 --2- 172.21.15.43:0/2389231452 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc660105ad0 0x7fc660105ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:26:44.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.102+0000 7fc665400700 1 --2- 172.21.15.43:0/2389231452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc660104010 0x7fc660104470 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:26:44.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.102+0000 7fc665400700 1 -- 172.21.15.43:0/2389231452 >> 172.21.15.43:0/2389231452 conn(0x7fc6600ff9b0 msgr2=0x7fc660101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:26:44.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.102+0000 7fc665400700 1 -- 172.21.15.43:0/2389231452 shutdown_connections 2024-03-26T08:26:44.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.102+0000 7fc665400700 1 -- 172.21.15.43:0/2389231452 wait complete. 2024-03-26T08:26:44.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.103+0000 7fc665400700 1 Processor -- start 2024-03-26T08:26:44.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.104+0000 7fc665400700 1 -- start start 2024-03-26T08:26:44.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.104+0000 7fc665400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc660104010 0x7fc66019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:26:44.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.104+0000 7fc665400700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6601049b0 0x7fc66019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:26:44.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.104+0000 7fc665400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc660105ad0 0x7fc6601a1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:26:44.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.104+0000 7fc665400700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66010ff90 con 0x7fc660104010 2024-03-26T08:26:44.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.104+0000 7fc665400700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66010fe10 con 0x7fc6601049b0 2024-03-26T08:26:44.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.104+0000 7fc665400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc660110110 con 0x7fc660105ad0 2024-03-26T08:26:44.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.104+0000 7fc65e800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc660104010 0x7fc66019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:26:44.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.104+0000 7fc65e800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc660104010 0x7fc66019a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60580/0 (socket says 172.21.15.43:60580) 2024-03-26T08:26:44.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.104+0000 7fc65e800700 1 -- 172.21.15.43:0/3908744721 learned_addr learned my addr 172.21.15.43:0/3908744721 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:26:44.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.105+0000 7fc657e00700 1 --2- 172.21.15.43:0/3908744721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6601049b0 0x7fc66019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:26:44.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.105+0000 7fc65f200700 1 --2- 172.21.15.43:0/3908744721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc660105ad0 0x7fc6601a1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:26:44.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.105+0000 7fc65e800700 1 -- 172.21.15.43:0/3908744721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc660105ad0 msgr2=0x7fc6601a1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:26:44.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.105+0000 7fc65e800700 1 --2- 172.21.15.43:0/3908744721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc660105ad0 0x7fc6601a1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:26:44.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.105+0000 7fc65e800700 1 -- 172.21.15.43:0/3908744721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6601049b0 msgr2=0x7fc66019ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:26:44.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.105+0000 7fc65e800700 1 --2- 172.21.15.43:0/3908744721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6601049b0 0x7fc66019ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:26:44.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.105+0000 7fc65e800700 1 -- 172.21.15.43:0/3908744721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6601a2230 con 0x7fc660104010 2024-03-26T08:26:44.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.105+0000 7fc65e800700 1 --2- 172.21.15.43:0/3908744721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc660104010 0x7fc66019a5d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc65000e9a0 tx=0x7fc65000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:26:44.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.105+0000 7fc657400700 1 -- 172.21.15.43:0/3908744721 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65000cbd0 con 0x7fc660104010 2024-03-26T08:26:44.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.106+0000 7fc665400700 1 -- 172.21.15.43:0/3908744721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6601a2520 con 0x7fc660104010 2024-03-26T08:26:44.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.106+0000 7fc657400700 1 -- 172.21.15.43:0/3908744721 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc6500044d0 con 0x7fc660104010 2024-03-26T08:26:44.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.106+0000 7fc657400700 1 -- 172.21.15.43:0/3908744721 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc650003e30 con 0x7fc660104010 2024-03-26T08:26:44.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.106+0000 7fc665400700 1 -- 172.21.15.43:0/3908744721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6601a2ab0 con 0x7fc660104010 2024-03-26T08:26:44.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.107+0000 7fc665400700 1 -- 172.21.15.43:0/3908744721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc660068e50 con 0x7fc660104010 2024-03-26T08:26:44.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.109+0000 7fc657400700 1 -- 172.21.15.43:0/3908744721 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc650004000 con 0x7fc660104010 2024-03-26T08:26:44.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.110+0000 7fc657400700 1 --2- 172.21.15.43:0/3908744721 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fc6440a2910 0x7fc6440a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:26:44.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.111+0000 7fc657400700 1 -- 172.21.15.43:0/3908744721 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4533+0+0 (secure 0 0 0) 0x7fc6500cdee0 con 0x7fc660104010 2024-03-26T08:26:44.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.111+0000 7fc657e00700 1 --2- 172.21.15.43:0/3908744721 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fc6440a2910 0x7fc6440a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:26:44.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.111+0000 7fc657e00700 1 --2- 172.21.15.43:0/3908744721 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fc6440a2910 0x7fc6440a4dd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc64800b2e0 tx=0x7fc64801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:26:44.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.115+0000 7fc657400700 1 -- 172.21.15.43:0/3908744721 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc6500042f0 con 0x7fc660104010 2024-03-26T08:26:44.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:43 smithi043 ceph-mon[31929]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:26:44.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:43 smithi183 ceph-mon[35818]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:26:44.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:43 smithi052 ceph-mon[35071]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:26:44.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:26:44.359+0000 7fc665400700 1 -- 172.21.15.43:0/3908744721 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6600625d0 con 0x7fc6440a2910 2024-03-26T08:26:45.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:44 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:26:45.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:44 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:26:45.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:44 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:26:45.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:44 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:26:45.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:44 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:26:45.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:44 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:26:45.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:44 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:26:45.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:44 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:26:45.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:44 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:26:46.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:45 smithi043 ceph-mon[31929]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:26:46.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:45 smithi043 ceph-mon[31929]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:26:46.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:45 smithi183 ceph-mon[35818]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:26:46.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:45 smithi183 ceph-mon[35818]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:26:46.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:45 smithi052 ceph-mon[35071]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:26:46.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:45 smithi052 ceph-mon[35071]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:26:48.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:47 smithi043 ceph-mon[31929]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-03-26T08:26:48.212 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:47 smithi183 ceph-mon[35818]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-03-26T08:26:48.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:47 smithi052 ceph-mon[35071]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-03-26T08:26:49.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:48 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3960304662' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3de2a49d-6cef-4580-9d8a-f90cdfcecd9a"}]: dispatch 2024-03-26T08:26:49.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:48 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3960304662' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3de2a49d-6cef-4580-9d8a-f90cdfcecd9a"}]': finished 2024-03-26T08:26:49.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:48 smithi043 ceph-mon[31929]: osdmap e44: 7 total, 6 up, 7 in 2024-03-26T08:26:49.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:48 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:26:49.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:48 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3960304662' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3de2a49d-6cef-4580-9d8a-f90cdfcecd9a"}]: dispatch 2024-03-26T08:26:49.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:48 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3960304662' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3de2a49d-6cef-4580-9d8a-f90cdfcecd9a"}]': finished 2024-03-26T08:26:49.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:48 smithi183 ceph-mon[35818]: osdmap e44: 7 total, 6 up, 7 in 2024-03-26T08:26:49.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:48 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:26:49.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:48 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3960304662' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3de2a49d-6cef-4580-9d8a-f90cdfcecd9a"}]: dispatch 2024-03-26T08:26:49.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:48 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3960304662' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3de2a49d-6cef-4580-9d8a-f90cdfcecd9a"}]': finished 2024-03-26T08:26:49.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:48 smithi052 ceph-mon[35071]: osdmap e44: 7 total, 6 up, 7 in 2024-03-26T08:26:49.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:48 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:26:50.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:49 smithi043 ceph-mon[31929]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T08:26:50.154 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:49 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1707669604' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:26:50.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:49 smithi183 ceph-mon[35818]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T08:26:50.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:49 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1707669604' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:26:50.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:49 smithi052 ceph-mon[35071]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T08:26:50.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:49 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1707669604' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:26:52.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:51 smithi183 ceph-mon[35818]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-26T08:26:52.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:51 smithi052 ceph-mon[35071]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-26T08:26:52.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:51 smithi043 ceph-mon[31929]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-26T08:26:54.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:53 smithi183 ceph-mon[35818]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T08:26:54.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:53 smithi052 ceph-mon[35071]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T08:26:54.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:53 smithi043 ceph-mon[31929]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T08:26:56.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:55 smithi043 ceph-mon[31929]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T08:26:56.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:55 smithi183 ceph-mon[35818]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T08:26:56.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:55 smithi052 ceph-mon[35071]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T08:26:58.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:57 smithi183 ceph-mon[35818]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:26:58.308 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:57 smithi043 ceph-mon[31929]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:26:58.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:57 smithi052 ceph-mon[35071]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:26:59.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:58 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-26T08:26:59.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:58 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:26:59.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:58 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-26T08:26:59.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:58 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:26:59.373 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:58 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-26T08:26:59.373 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:58 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:27:00.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:59 smithi043 ceph-mon[31929]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:27:00.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:26:59 smithi043 ceph-mon[31929]: Deploying daemon osd.6 on smithi043 2024-03-26T08:27:00.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:59 smithi183 ceph-mon[35818]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:27:00.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:26:59 smithi183 ceph-mon[35818]: Deploying daemon osd.6 on smithi043 2024-03-26T08:27:00.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:59 smithi052 ceph-mon[35071]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:27:00.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:26:59 smithi052 ceph-mon[35071]: Deploying daemon osd.6 on smithi043 2024-03-26T08:27:02.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:01 smithi043 ceph-mon[31929]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:27:02.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:01 smithi183 ceph-mon[35818]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:27:02.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:01 smithi052 ceph-mon[35071]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:27:04.100 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:03 smithi043 ceph-mon[31929]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:27:04.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:03 smithi183 ceph-mon[35818]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:27:04.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:03 smithi052 ceph-mon[35071]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:27:06.091 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:05 smithi043 ceph-mon[31929]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:27:06.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:05 smithi183 ceph-mon[35818]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:27:06.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:05 smithi052 ceph-mon[35071]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:27:08.349 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:08 smithi043 ceph-mon[31929]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:27:08.350 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:08 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:08.350 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:08 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:08.350 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:08 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:27:08.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:08 smithi183 ceph-mon[35818]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:27:08.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:08 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:08.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:08 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:08.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:08 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:27:08.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:08 smithi052 ceph-mon[35071]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:27:08.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:08 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:08.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:08 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:08.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:08 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:27:10.265 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:10 smithi043 ceph-mon[31929]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:27:10.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:10 smithi183 ceph-mon[35818]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:27:10.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:10 smithi052 ceph-mon[35071]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:27:12.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:27:12.269+0000 7fc657400700 1 -- 172.21.15.43:0/3908744721 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc6600625d0 con 0x7fc6440a2910 2024-03-26T08:27:12.264 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 6 on host 'smithi043' 2024-03-26T08:27:12.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:27:12.274+0000 7fc665400700 1 -- 172.21.15.43:0/3908744721 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fc6440a2910 msgr2=0x7fc6440a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:27:12.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:27:12.274+0000 7fc665400700 1 --2- 172.21.15.43:0/3908744721 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fc6440a2910 0x7fc6440a4dd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc64800b2e0 tx=0x7fc64801a040 comp rx=0 tx=0).stop 2024-03-26T08:27:12.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:27:12.274+0000 7fc665400700 1 -- 172.21.15.43:0/3908744721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc660104010 msgr2=0x7fc66019a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:27:12.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:27:12.274+0000 7fc665400700 1 --2- 172.21.15.43:0/3908744721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc660104010 0x7fc66019a5d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc65000e9a0 tx=0x7fc65000ed60 comp rx=0 tx=0).stop 2024-03-26T08:27:12.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:27:12.275+0000 7fc665400700 1 -- 172.21.15.43:0/3908744721 shutdown_connections 2024-03-26T08:27:12.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:27:12.275+0000 7fc665400700 1 --2- 172.21.15.43:0/3908744721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc660105ad0 0x7fc6601a1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:27:12.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:27:12.275+0000 7fc665400700 1 --2- 172.21.15.43:0/3908744721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc660104010 0x7fc66019a5d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:27:12.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:27:12.275+0000 7fc665400700 1 --2- 172.21.15.43:0/3908744721 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fc6440a2910 0x7fc6440a4dd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:27:12.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:27:12.275+0000 7fc665400700 1 --2- 172.21.15.43:0/3908744721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6601049b0 0x7fc66019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:27:12.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:27:12.275+0000 7fc665400700 1 -- 172.21.15.43:0/3908744721 >> 172.21.15.43:0/3908744721 conn(0x7fc6600ff9b0 msgr2=0x7fc6601088f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:27:12.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:27:12.275+0000 7fc665400700 1 -- 172.21.15.43:0/3908744721 shutdown_connections 2024-03-26T08:27:12.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:27:12.275+0000 7fc665400700 1 -- 172.21.15.43:0/3908744721 wait complete. 2024-03-26T08:27:12.270 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:12 smithi043 ceph-mon[31929]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:27:12.270 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:12 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:12.270 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:12 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:12.271 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:12 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:27:12.271 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:12 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:27:12.271 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:12 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:12.271 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:12 smithi043 ceph-mon[31929]: from='osd.6 [v2:172.21.15.43:6818/4206861158,v1:172.21.15.43:6819/4206861158]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T08:27:12.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:12 smithi183 ceph-mon[35818]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:27:12.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:12 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:12.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:12 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:12.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:12 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:27:12.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:12 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:27:12.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:12 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:12.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:12 smithi183 ceph-mon[35818]: from='osd.6 [v2:172.21.15.43:6818/4206861158,v1:172.21.15.43:6819/4206861158]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T08:27:12.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:12 smithi052 ceph-mon[35071]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:27:12.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:12 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:12.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:12 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:12.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:12 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:27:12.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:12 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:27:12.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:12 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:12.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:12 smithi052 ceph-mon[35071]: from='osd.6 [v2:172.21.15.43:6818/4206861158,v1:172.21.15.43:6819/4206861158]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T08:27:13.095 DEBUG:teuthology.orchestra.run.smithi043:osd.6> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.6.service 2024-03-26T08:27:13.100 INFO:tasks.cephadm:Deploying osd.7 on smithi052 with /dev/vg_nvme/lv_2... 2024-03-26T08:27:13.101 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-26T08:27:13.291 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:13 smithi043 ceph-mon[31929]: from='osd.6 [v2:172.21.15.43:6818/4206861158,v1:172.21.15.43:6819/4206861158]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-26T08:27:13.291 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:13 smithi043 ceph-mon[31929]: osdmap e45: 7 total, 6 up, 7 in 2024-03-26T08:27:13.291 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:13 smithi043 ceph-mon[31929]: from='osd.6 [v2:172.21.15.43:6818/4206861158,v1:172.21.15.43:6819/4206861158]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-26T08:27:13.292 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:13 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:27:13.292 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:13 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:13.292 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:13 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:13.292 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:13 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:27:13.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:13 smithi052 ceph-mon[35071]: from='osd.6 [v2:172.21.15.43:6818/4206861158,v1:172.21.15.43:6819/4206861158]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-26T08:27:13.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:13 smithi052 ceph-mon[35071]: osdmap e45: 7 total, 6 up, 7 in 2024-03-26T08:27:13.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:13 smithi052 ceph-mon[35071]: from='osd.6 [v2:172.21.15.43:6818/4206861158,v1:172.21.15.43:6819/4206861158]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-26T08:27:13.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:13 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:27:13.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:13 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:13.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:13 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:13.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:13 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:27:13.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:13 smithi183 ceph-mon[35818]: from='osd.6 [v2:172.21.15.43:6818/4206861158,v1:172.21.15.43:6819/4206861158]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-26T08:27:13.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:13 smithi183 ceph-mon[35818]: osdmap e45: 7 total, 6 up, 7 in 2024-03-26T08:27:13.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:13 smithi183 ceph-mon[35818]: from='osd.6 [v2:172.21.15.43:6818/4206861158,v1:172.21.15.43:6819/4206861158]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-26T08:27:13.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:13 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:27:13.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:13 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:13.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:13 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:13.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:13 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:27:13.565 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T08:27:13.903 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:27:13 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:27:13.713+0000 7fefc0e00700 -1 osd.6 0 waiting for initial osdmap 2024-03-26T08:27:13.903 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:27:13 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:27:13.733+0000 7fefb6200700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T08:27:14.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:14 smithi052 ceph-mon[35071]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:27:14.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:14 smithi052 ceph-mon[35071]: from='osd.6 [v2:172.21.15.43:6818/4206861158,v1:172.21.15.43:6819/4206861158]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-03-26T08:27:14.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:14 smithi052 ceph-mon[35071]: osdmap e46: 7 total, 6 up, 7 in 2024-03-26T08:27:14.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:14 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:27:14.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:14 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:27:14.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:14 smithi052 ceph-mon[35071]: from='osd.6 [v2:172.21.15.43:6818/4206861158,v1:172.21.15.43:6819/4206861158]' entity='osd.6' 2024-03-26T08:27:14.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:14 smithi043 ceph-mon[31929]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:27:14.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:14 smithi043 ceph-mon[31929]: from='osd.6 [v2:172.21.15.43:6818/4206861158,v1:172.21.15.43:6819/4206861158]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-03-26T08:27:14.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:14 smithi043 ceph-mon[31929]: osdmap e46: 7 total, 6 up, 7 in 2024-03-26T08:27:14.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:14 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:27:14.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:14 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:27:14.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:14 smithi043 ceph-mon[31929]: from='osd.6 [v2:172.21.15.43:6818/4206861158,v1:172.21.15.43:6819/4206861158]' entity='osd.6' 2024-03-26T08:27:14.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:14 smithi183 ceph-mon[35818]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:27:14.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:14 smithi183 ceph-mon[35818]: from='osd.6 [v2:172.21.15.43:6818/4206861158,v1:172.21.15.43:6819/4206861158]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-03-26T08:27:14.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:14 smithi183 ceph-mon[35818]: osdmap e46: 7 total, 6 up, 7 in 2024-03-26T08:27:14.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:14 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:27:14.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:14 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:27:14.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:14 smithi183 ceph-mon[35818]: from='osd.6 [v2:172.21.15.43:6818/4206861158,v1:172.21.15.43:6819/4206861158]' entity='osd.6' 2024-03-26T08:27:15.269 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:15 smithi043 ceph-mon[31929]: purged_snaps scrub starts 2024-03-26T08:27:15.270 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:15 smithi043 ceph-mon[31929]: purged_snaps scrub ok 2024-03-26T08:27:15.270 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:15 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:27:15.270 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:15 smithi043 ceph-mon[31929]: osd.6 [v2:172.21.15.43:6818/4206861158,v1:172.21.15.43:6819/4206861158] boot 2024-03-26T08:27:15.270 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:15 smithi043 ceph-mon[31929]: osdmap e47: 7 total, 7 up, 7 in 2024-03-26T08:27:15.270 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:15 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:27:15.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:15 smithi052 ceph-mon[35071]: purged_snaps scrub starts 2024-03-26T08:27:15.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:15 smithi052 ceph-mon[35071]: purged_snaps scrub ok 2024-03-26T08:27:15.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:15 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:27:15.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:15 smithi052 ceph-mon[35071]: osd.6 [v2:172.21.15.43:6818/4206861158,v1:172.21.15.43:6819/4206861158] boot 2024-03-26T08:27:15.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:15 smithi052 ceph-mon[35071]: osdmap e47: 7 total, 7 up, 7 in 2024-03-26T08:27:15.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:15 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:27:15.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:15 smithi183 ceph-mon[35818]: purged_snaps scrub starts 2024-03-26T08:27:15.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:15 smithi183 ceph-mon[35818]: purged_snaps scrub ok 2024-03-26T08:27:15.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:15 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:27:15.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:15 smithi183 ceph-mon[35818]: osd.6 [v2:172.21.15.43:6818/4206861158,v1:172.21.15.43:6819/4206861158] boot 2024-03-26T08:27:15.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:15 smithi183 ceph-mon[35818]: osdmap e47: 7 total, 7 up, 7 in 2024-03-26T08:27:15.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:15 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:27:16.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:16 smithi043 ceph-mon[31929]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:27:16.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:16 smithi183 ceph-mon[35818]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:27:16.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:16 smithi052 ceph-mon[35071]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:27:16.866 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-26T08:27:16.866 INFO:teuthology.orchestra.run.smithi052.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-26T08:27:16.866 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10+0 records in 2024-03-26T08:27:16.866 INFO:teuthology.orchestra.run.smithi052.stderr:10+0 records out 2024-03-26T08:27:16.867 INFO:teuthology.orchestra.run.smithi052.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0375581 s, 279 MB/s 2024-03-26T08:27:16.867 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping successful for: 2024-03-26T08:27:17.287 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:17 smithi052 ceph-mon[35071]: osdmap e48: 7 total, 7 up, 7 in 2024-03-26T08:27:17.339 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:17 smithi043 ceph-mon[31929]: osdmap e48: 7 total, 7 up, 7 in 2024-03-26T08:27:17.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:17 smithi183 ceph-mon[35818]: osdmap e48: 7 total, 7 up, 7 in 2024-03-26T08:27:17.791 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi052:vg_nvme/lv_2 2024-03-26T08:27:18.260 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T08:27:18.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:18 smithi043 ceph-mon[31929]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:18.413 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:18 smithi052 ceph-mon[35071]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:18.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:18 smithi183 ceph-mon[35818]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:19.450 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:19 smithi052 ceph-mon[35071]: Detected new or changed devices on smithi043 2024-03-26T08:27:19.450 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:19 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:19.450 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:19 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:19.450 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:19 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:27:19.450 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:19 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:27:19.450 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:19 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:27:19.450 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:19 smithi052 ceph-mon[35071]: Adjusting osd_memory_target on smithi043 to 5698M 2024-03-26T08:27:19.451 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:19 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:19.451 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:19 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:27:19.451 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:19 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:27:19.451 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:19 smithi052 ceph-mon[35071]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:19.451 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:19 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:19.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:19 smithi183 ceph-mon[35818]: Detected new or changed devices on smithi043 2024-03-26T08:27:19.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:19 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:19.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:19 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:19.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:19 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:27:19.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:19 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:27:19.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:19 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:27:19.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:19 smithi183 ceph-mon[35818]: Adjusting osd_memory_target on smithi043 to 5698M 2024-03-26T08:27:19.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:19 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:19.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:19 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:27:19.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:19 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:27:19.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:19 smithi183 ceph-mon[35818]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:19.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:19 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:19.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:19 smithi043 ceph-mon[31929]: Detected new or changed devices on smithi043 2024-03-26T08:27:19.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:19 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:19.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:19 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:19.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:19 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:27:19.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:19 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:27:19.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:19 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:27:19.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:19 smithi043 ceph-mon[31929]: Adjusting osd_memory_target on smithi043 to 5698M 2024-03-26T08:27:19.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:19 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:19.655 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:19 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:27:19.655 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:19 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:27:19.655 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:19 smithi043 ceph-mon[31929]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:19.655 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:19 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:20.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.478+0000 7efd3e600700 1 -- 172.21.15.52:0/3866414380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd38105d80 msgr2=0x7efd38106200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:27:20.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.478+0000 7efd3e600700 1 --2- 172.21.15.52:0/3866414380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd38105d80 0x7efd38106200 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efd3000b0a0 tx=0x7efd3000b3b0 comp rx=0 tx=0).stop 2024-03-26T08:27:20.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.478+0000 7efd3e600700 1 -- 172.21.15.52:0/3866414380 shutdown_connections 2024-03-26T08:27:20.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.478+0000 7efd3e600700 1 --2- 172.21.15.52:0/3866414380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd38105d80 0x7efd38106200 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:27:20.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.478+0000 7efd3e600700 1 --2- 172.21.15.52:0/3866414380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd38100a40 0x7efd38102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:27:20.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.478+0000 7efd3e600700 1 --2- 172.21.15.52:0/3866414380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd381033b0 0x7efd38105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:27:20.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.478+0000 7efd3e600700 1 -- 172.21.15.52:0/3866414380 >> 172.21.15.52:0/3866414380 conn(0x7efd380fc660 msgr2=0x7efd380feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:27:20.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.478+0000 7efd3e600700 1 -- 172.21.15.52:0/3866414380 shutdown_connections 2024-03-26T08:27:20.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.478+0000 7efd3e600700 1 -- 172.21.15.52:0/3866414380 wait complete. 2024-03-26T08:27:20.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.479+0000 7efd3e600700 1 Processor -- start 2024-03-26T08:27:20.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.479+0000 7efd3e600700 1 -- start start 2024-03-26T08:27:20.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.479+0000 7efd3e600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd38100a40 0x7efd3819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:27:20.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.479+0000 7efd3e600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd381033b0 0x7efd3819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:27:20.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.479+0000 7efd3e600700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd38105d80 0x7efd381a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:27:20.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.479+0000 7efd3e600700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd38114230 con 0x7efd38100a40 2024-03-26T08:27:20.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.479+0000 7efd3e600700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd381140b0 con 0x7efd38105d80 2024-03-26T08:27:20.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.479+0000 7efd3e600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd381143b0 con 0x7efd381033b0 2024-03-26T08:27:20.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.480+0000 7efd37e00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd38105d80 0x7efd381a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:27:20.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.480+0000 7efd37e00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd38105d80 0x7efd381a5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49002/0 (socket says 172.21.15.52:49002) 2024-03-26T08:27:20.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.480+0000 7efd37e00700 1 -- 172.21.15.52:0/2158929319 learned_addr learned my addr 172.21.15.52:0/2158929319 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-26T08:27:20.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.480+0000 7efd37400700 1 --2- 172.21.15.52:0/2158929319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd38100a40 0x7efd3819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:27:20.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.480+0000 7efd2fe00700 1 --2- 172.21.15.52:0/2158929319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd381033b0 0x7efd3819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:27:20.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.480+0000 7efd37e00700 1 -- 172.21.15.52:0/2158929319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd381033b0 msgr2=0x7efd3819ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:27:20.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.480+0000 7efd37e00700 1 --2- 172.21.15.52:0/2158929319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd381033b0 0x7efd3819ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:27:20.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.480+0000 7efd37e00700 1 -- 172.21.15.52:0/2158929319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd38100a40 msgr2=0x7efd3819e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:27:20.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.480+0000 7efd37e00700 1 --2- 172.21.15.52:0/2158929319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd38100a40 0x7efd3819e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:27:20.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.480+0000 7efd37e00700 1 -- 172.21.15.52:0/2158929319 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd381a6530 con 0x7efd38105d80 2024-03-26T08:27:20.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.481+0000 7efd37e00700 1 --2- 172.21.15.52:0/2158929319 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd38105d80 0x7efd381a5e90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efd300062a0 tx=0x7efd3000cee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:27:20.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.481+0000 7efd34c00700 1 -- 172.21.15.52:0/2158929319 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd30021070 con 0x7efd38105d80 2024-03-26T08:27:20.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.481+0000 7efd34c00700 1 -- 172.21.15.52:0/2158929319 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efd30007d00 con 0x7efd38105d80 2024-03-26T08:27:20.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.481+0000 7efd34c00700 1 -- 172.21.15.52:0/2158929319 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3001d450 con 0x7efd38105d80 2024-03-26T08:27:20.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.481+0000 7efd3e600700 1 -- 172.21.15.52:0/2158929319 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd381a67c0 con 0x7efd38105d80 2024-03-26T08:27:20.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.481+0000 7efd3e600700 1 -- 172.21.15.52:0/2158929319 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd381a6cc0 con 0x7efd38105d80 2024-03-26T08:27:20.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.482+0000 7efd3e600700 1 -- 172.21.15.52:0/2158929319 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd38068e50 con 0x7efd38105d80 2024-03-26T08:27:20.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.486+0000 7efd34c00700 1 -- 172.21.15.52:0/2158929319 <== mon.2 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7efd30003a00 con 0x7efd38105d80 2024-03-26T08:27:20.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.487+0000 7efd34c00700 1 --2- 172.21.15.52:0/2158929319 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7efd1c0a2910 0x7efd1c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:27:20.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.487+0000 7efd34c00700 1 -- 172.21.15.52:0/2158929319 <== mon.2 v2:172.21.15.52:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4965+0+0 (secure 0 0 0) 0x7efd3000f020 con 0x7efd38105d80 2024-03-26T08:27:20.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.488+0000 7efd37400700 1 --2- 172.21.15.52:0/2158929319 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7efd1c0a2910 0x7efd1c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:27:20.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.489+0000 7efd37400700 1 --2- 172.21.15.52:0/2158929319 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7efd1c0a2910 0x7efd1c0a4dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efd28005950 tx=0x7efd2800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:27:20.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.490+0000 7efd34c00700 1 -- 172.21.15.52:0/2158929319 <== mon.2 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efd3002a020 con 0x7efd38105d80 2024-03-26T08:27:20.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:20.732+0000 7efd3e600700 1 -- 172.21.15.52:0/2158929319 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7efd38062690 con 0x7efd1c0a2910 2024-03-26T08:27:21.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:21 smithi052 ceph-mon[35071]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:21.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:21 smithi052 ceph-mon[35071]: from='client.34283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:27:21.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:21 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:27:21.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:21 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:27:21.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:21 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:27:21.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:21 smithi043 ceph-mon[31929]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:21.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:21 smithi043 ceph-mon[31929]: from='client.34283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:27:21.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:21 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:27:21.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:21 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:27:21.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:21 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:27:21.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:21 smithi183 ceph-mon[35818]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:21.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:21 smithi183 ceph-mon[35818]: from='client.34283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:27:21.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:21 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:27:21.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:21 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:27:21.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:21 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:27:23.517 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:23 smithi052 ceph-mon[35071]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:23.602 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:23 smithi183 ceph-mon[35818]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:23.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:23 smithi043 ceph-mon[31929]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:25.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:25 smithi052 ceph-mon[35071]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:25.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:25 smithi052 ceph-mon[35071]: from='client.? 172.21.15.52:0/681510712' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3b832f7-7d7d-4a62-b679-db6da98275d9"}]: dispatch 2024-03-26T08:27:25.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:25 smithi052 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3b832f7-7d7d-4a62-b679-db6da98275d9"}]: dispatch 2024-03-26T08:27:25.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:25 smithi052 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3b832f7-7d7d-4a62-b679-db6da98275d9"}]': finished 2024-03-26T08:27:25.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:25 smithi052 ceph-mon[35071]: osdmap e49: 8 total, 7 up, 8 in 2024-03-26T08:27:25.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:25 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:27:25.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:25 smithi043 ceph-mon[31929]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:25.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:25 smithi043 ceph-mon[31929]: from='client.? 172.21.15.52:0/681510712' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3b832f7-7d7d-4a62-b679-db6da98275d9"}]: dispatch 2024-03-26T08:27:25.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:25 smithi043 ceph-mon[31929]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3b832f7-7d7d-4a62-b679-db6da98275d9"}]: dispatch 2024-03-26T08:27:25.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:25 smithi043 ceph-mon[31929]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3b832f7-7d7d-4a62-b679-db6da98275d9"}]': finished 2024-03-26T08:27:25.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:25 smithi043 ceph-mon[31929]: osdmap e49: 8 total, 7 up, 8 in 2024-03-26T08:27:25.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:25 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:27:25.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:25 smithi183 ceph-mon[35818]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:25.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:25 smithi183 ceph-mon[35818]: from='client.? 172.21.15.52:0/681510712' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3b832f7-7d7d-4a62-b679-db6da98275d9"}]: dispatch 2024-03-26T08:27:25.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:25 smithi183 ceph-mon[35818]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3b832f7-7d7d-4a62-b679-db6da98275d9"}]: dispatch 2024-03-26T08:27:25.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:25 smithi183 ceph-mon[35818]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3b832f7-7d7d-4a62-b679-db6da98275d9"}]': finished 2024-03-26T08:27:25.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:25 smithi183 ceph-mon[35818]: osdmap e49: 8 total, 7 up, 8 in 2024-03-26T08:27:25.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:25 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:27:26.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:26 smithi043 ceph-mon[31929]: from='client.? 172.21.15.52:0/3681287858' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:27:26.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:26 smithi183 ceph-mon[35818]: from='client.? 172.21.15.52:0/3681287858' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:27:26.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:26 smithi052 ceph-mon[35071]: from='client.? 172.21.15.52:0/3681287858' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:27:27.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:27 smithi043 ceph-mon[31929]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:27.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:27 smithi183 ceph-mon[35818]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:27.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:27 smithi052 ceph-mon[35071]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:29.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:29 smithi052 ceph-mon[35071]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:29.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:29 smithi043 ceph-mon[31929]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:29.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:29 smithi183 ceph-mon[35818]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:31.602 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:31 smithi052 ceph-mon[35071]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:31.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:31 smithi043 ceph-mon[31929]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:31.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:31 smithi183 ceph-mon[35818]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:33.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:33 smithi043 ceph-mon[31929]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:33.774 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:33 smithi052 ceph-mon[35071]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:33.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:33 smithi183 ceph-mon[35818]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:34.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:34 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-26T08:27:34.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:34 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:27:34.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:34 smithi043 ceph-mon[31929]: Deploying daemon osd.7 on smithi052 2024-03-26T08:27:34.713 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:34 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-26T08:27:34.713 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:34 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:27:34.714 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:34 smithi052 ceph-mon[35071]: Deploying daemon osd.7 on smithi052 2024-03-26T08:27:34.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:34 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-26T08:27:34.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:34 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:27:34.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:34 smithi183 ceph-mon[35818]: Deploying daemon osd.7 on smithi052 2024-03-26T08:27:35.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:35 smithi052 ceph-mon[35071]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:35.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:35 smithi043 ceph-mon[31929]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:35.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:35 smithi183 ceph-mon[35818]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:37.538 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:37 smithi052 ceph-mon[35071]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:37.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:37 smithi043 ceph-mon[31929]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:37.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:37 smithi183 ceph-mon[35818]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:39.606 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:39 smithi052 ceph-mon[35071]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:39.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:39 smithi043 ceph-mon[31929]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:39.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:39 smithi183 ceph-mon[35818]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:41.563 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:41 smithi052 ceph-mon[35071]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:41.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:41 smithi043 ceph-mon[31929]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:41.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:41 smithi183 ceph-mon[35818]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:43.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:43 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:43.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:43 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:43.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:43 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:27:43.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:43 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:43.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:43 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:43.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:43 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:27:43.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:43 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:43.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:43 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:43.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:43 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:27:44.253 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:44 smithi052 ceph-mon[35071]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:44.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:44 smithi043 ceph-mon[31929]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:44.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:44 smithi183 ceph-mon[35818]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:46.235 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:46 smithi052 ceph-mon[35071]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:46.235 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:46 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:46.235 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:46 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:46.235 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:46 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:27:46.235 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:46 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:27:46.235 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:46 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:46.236 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:46 smithi052 ceph-mon[35071]: from='osd.7 [v2:172.21.15.52:6816/3452064146,v1:172.21.15.52:6817/3452064146]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T08:27:46.236 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:46 smithi052 ceph-mon[35071]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T08:27:46.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:46 smithi043 ceph-mon[31929]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:46.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:46 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:46.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:46 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:46.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:46 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:27:46.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:46 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:27:46.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:46 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:46.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:46 smithi043 ceph-mon[31929]: from='osd.7 [v2:172.21.15.52:6816/3452064146,v1:172.21.15.52:6817/3452064146]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T08:27:46.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:46 smithi043 ceph-mon[31929]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T08:27:46.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:46 smithi183 ceph-mon[35818]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:46.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:46 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:46.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:46 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:46.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:46 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:27:46.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:46 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:27:46.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:46 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:46.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:46 smithi183 ceph-mon[35818]: from='osd.7 [v2:172.21.15.52:6816/3452064146,v1:172.21.15.52:6817/3452064146]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T08:27:46.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:46 smithi183 ceph-mon[35818]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T08:27:47.091 INFO:teuthology.orchestra.run.smithi052.stdout:Created osd(s) 7 on host 'smithi052' 2024-03-26T08:27:47.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:47.096+0000 7efd34c00700 1 -- 172.21.15.52:0/2158929319 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7efd38062690 con 0x7efd1c0a2910 2024-03-26T08:27:47.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:47.099+0000 7efd3e600700 1 -- 172.21.15.52:0/2158929319 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7efd1c0a2910 msgr2=0x7efd1c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:27:47.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:47.099+0000 7efd3e600700 1 --2- 172.21.15.52:0/2158929319 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7efd1c0a2910 0x7efd1c0a4dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efd28005950 tx=0x7efd2800b410 comp rx=0 tx=0).stop 2024-03-26T08:27:47.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:47.099+0000 7efd3e600700 1 -- 172.21.15.52:0/2158929319 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd38105d80 msgr2=0x7efd381a5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:27:47.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:47.099+0000 7efd3e600700 1 --2- 172.21.15.52:0/2158929319 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd38105d80 0x7efd381a5e90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efd300062a0 tx=0x7efd3000cee0 comp rx=0 tx=0).stop 2024-03-26T08:27:47.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:47.099+0000 7efd3e600700 1 -- 172.21.15.52:0/2158929319 shutdown_connections 2024-03-26T08:27:47.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:47.099+0000 7efd3e600700 1 --2- 172.21.15.52:0/2158929319 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7efd1c0a2910 0x7efd1c0a4dd0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:27:47.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:47.100+0000 7efd3e600700 1 --2- 172.21.15.52:0/2158929319 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd38105d80 0x7efd381a5e90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:27:47.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:47.100+0000 7efd3e600700 1 --2- 172.21.15.52:0/2158929319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd38100a40 0x7efd3819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:27:47.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:47.100+0000 7efd3e600700 1 --2- 172.21.15.52:0/2158929319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd381033b0 0x7efd3819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:27:47.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:47.100+0000 7efd3e600700 1 -- 172.21.15.52:0/2158929319 >> 172.21.15.52:0/2158929319 conn(0x7efd380fc660 msgr2=0x7efd381013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:27:47.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:47.100+0000 7efd3e600700 1 -- 172.21.15.52:0/2158929319 shutdown_connections 2024-03-26T08:27:47.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:27:47.100+0000 7efd3e600700 1 -- 172.21.15.52:0/2158929319 wait complete. 2024-03-26T08:27:48.062 DEBUG:teuthology.orchestra.run.smithi052:osd.7> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.7.service 2024-03-26T08:27:48.067 INFO:tasks.cephadm:Deploying osd.8 on smithi183 with /dev/vg_nvme/lv_2... 2024-03-26T08:27:48.067 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-26T08:27:48.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:48 smithi183 ceph-mon[35818]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:48.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:48 smithi183 ceph-mon[35818]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-26T08:27:48.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:48 smithi183 ceph-mon[35818]: from='osd.7 [v2:172.21.15.52:6816/3452064146,v1:172.21.15.52:6817/3452064146]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-26T08:27:48.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:48 smithi183 ceph-mon[35818]: osdmap e50: 8 total, 7 up, 8 in 2024-03-26T08:27:48.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:48 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:27:48.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:48 smithi183 ceph-mon[35818]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-26T08:27:48.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:48 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:48.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:48 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:27:48.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:48 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:48.317 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:48 smithi052 ceph-mon[35071]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:48.317 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:48 smithi052 ceph-mon[35071]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-26T08:27:48.317 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:48 smithi052 ceph-mon[35071]: from='osd.7 [v2:172.21.15.52:6816/3452064146,v1:172.21.15.52:6817/3452064146]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-26T08:27:48.318 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:48 smithi052 ceph-mon[35071]: osdmap e50: 8 total, 7 up, 8 in 2024-03-26T08:27:48.318 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:48 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:27:48.318 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:48 smithi052 ceph-mon[35071]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-26T08:27:48.318 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:48 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:48.318 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:48 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:27:48.318 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:48 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:48.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:48 smithi043 ceph-mon[31929]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:48.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:48 smithi043 ceph-mon[31929]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-26T08:27:48.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:48 smithi043 ceph-mon[31929]: from='osd.7 [v2:172.21.15.52:6816/3452064146,v1:172.21.15.52:6817/3452064146]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-26T08:27:48.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:48 smithi043 ceph-mon[31929]: osdmap e50: 8 total, 7 up, 8 in 2024-03-26T08:27:48.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:48 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:27:48.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:48 smithi043 ceph-mon[31929]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-26T08:27:48.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:48 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:48.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:48 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:27:48.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:48 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:48.500 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:27:49.049 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:27:48 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:27:48.678+0000 7fdcd8800700 -1 osd.7 0 waiting for initial osdmap 2024-03-26T08:27:49.049 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:27:48 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:27:48.708+0000 7fdcce600700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T08:27:49.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:49 smithi183 ceph-mon[35818]: purged_snaps scrub starts 2024-03-26T08:27:49.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:49 smithi183 ceph-mon[35818]: purged_snaps scrub ok 2024-03-26T08:27:49.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:49 smithi183 ceph-mon[35818]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-03-26T08:27:49.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:49 smithi183 ceph-mon[35818]: osdmap e51: 8 total, 7 up, 8 in 2024-03-26T08:27:49.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:49 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:27:49.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:49 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:27:49.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:49 smithi183 ceph-mon[35818]: from='osd.7 ' entity='osd.7' 2024-03-26T08:27:49.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:49 smithi183 ceph-mon[35818]: osd.7 [v2:172.21.15.52:6816/3452064146,v1:172.21.15.52:6817/3452064146] boot 2024-03-26T08:27:49.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:49 smithi183 ceph-mon[35818]: osdmap e52: 8 total, 8 up, 8 in 2024-03-26T08:27:49.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:49 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:27:49.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:49 smithi052 ceph-mon[35071]: purged_snaps scrub starts 2024-03-26T08:27:49.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:49 smithi052 ceph-mon[35071]: purged_snaps scrub ok 2024-03-26T08:27:49.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:49 smithi052 ceph-mon[35071]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-03-26T08:27:49.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:49 smithi052 ceph-mon[35071]: osdmap e51: 8 total, 7 up, 8 in 2024-03-26T08:27:49.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:49 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:27:49.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:49 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:27:49.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:49 smithi052 ceph-mon[35071]: from='osd.7 ' entity='osd.7' 2024-03-26T08:27:49.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:49 smithi052 ceph-mon[35071]: osd.7 [v2:172.21.15.52:6816/3452064146,v1:172.21.15.52:6817/3452064146] boot 2024-03-26T08:27:49.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:49 smithi052 ceph-mon[35071]: osdmap e52: 8 total, 8 up, 8 in 2024-03-26T08:27:49.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:49 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:27:49.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:49 smithi043 ceph-mon[31929]: purged_snaps scrub starts 2024-03-26T08:27:49.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:49 smithi043 ceph-mon[31929]: purged_snaps scrub ok 2024-03-26T08:27:49.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:49 smithi043 ceph-mon[31929]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-03-26T08:27:49.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:49 smithi043 ceph-mon[31929]: osdmap e51: 8 total, 7 up, 8 in 2024-03-26T08:27:49.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:49 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:27:49.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:49 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:27:49.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:49 smithi043 ceph-mon[31929]: from='osd.7 ' entity='osd.7' 2024-03-26T08:27:49.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:49 smithi043 ceph-mon[31929]: osd.7 [v2:172.21.15.52:6816/3452064146,v1:172.21.15.52:6817/3452064146] boot 2024-03-26T08:27:49.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:49 smithi043 ceph-mon[31929]: osdmap e52: 8 total, 8 up, 8 in 2024-03-26T08:27:49.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:49 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:27:50.227 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:50 smithi052 ceph-mon[35071]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:50.228 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:50 smithi052 ceph-mon[35071]: osdmap e53: 8 total, 8 up, 8 in 2024-03-26T08:27:50.276 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:50 smithi183 ceph-mon[35818]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:50.276 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:50 smithi183 ceph-mon[35818]: osdmap e53: 8 total, 8 up, 8 in 2024-03-26T08:27:50.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:50 smithi043 ceph-mon[31929]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:27:50.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:50 smithi043 ceph-mon[31929]: osdmap e53: 8 total, 8 up, 8 in 2024-03-26T08:27:51.144 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-26T08:27:51.144 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-26T08:27:51.144 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-03-26T08:27:51.144 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-03-26T08:27:51.144 INFO:teuthology.orchestra.run.smithi183.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0362325 s, 289 MB/s 2024-03-26T08:27:51.145 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-03-26T08:27:51.991 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi183:vg_nvme/lv_2 2024-03-26T08:27:52.248 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:51 smithi052 ceph-mon[35071]: pgmap v230: 1 pgs: 1 unknown; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:27:52.248 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:51 smithi052 ceph-mon[35071]: osdmap e54: 8 total, 8 up, 8 in 2024-03-26T08:27:52.248 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:51 smithi052 ceph-mon[35071]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-03-26T08:27:52.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:51 smithi183 ceph-mon[35818]: pgmap v230: 1 pgs: 1 unknown; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:27:52.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:51 smithi183 ceph-mon[35818]: osdmap e54: 8 total, 8 up, 8 in 2024-03-26T08:27:52.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:51 smithi183 ceph-mon[35818]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-03-26T08:27:52.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:51 smithi043 ceph-mon[31929]: pgmap v230: 1 pgs: 1 unknown; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:27:52.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:51 smithi043 ceph-mon[31929]: osdmap e54: 8 total, 8 up, 8 in 2024-03-26T08:27:52.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:51 smithi043 ceph-mon[31929]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-03-26T08:27:52.422 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:27:53.285 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:52 smithi183 ceph-mon[35818]: osdmap e55: 8 total, 8 up, 8 in 2024-03-26T08:27:53.286 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:52 smithi183 ceph-mon[35818]: osdmap e56: 8 total, 8 up, 8 in 2024-03-26T08:27:53.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:52 smithi052 ceph-mon[35071]: osdmap e55: 8 total, 8 up, 8 in 2024-03-26T08:27:53.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:52 smithi052 ceph-mon[35071]: osdmap e56: 8 total, 8 up, 8 in 2024-03-26T08:27:53.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:52 smithi043 ceph-mon[31929]: osdmap e55: 8 total, 8 up, 8 in 2024-03-26T08:27:53.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:52 smithi043 ceph-mon[31929]: osdmap e56: 8 total, 8 up, 8 in 2024-03-26T08:27:54.257 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:53 smithi183 ceph-mon[35818]: pgmap v233: 1 pgs: 1 unknown; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:27:54.257 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:53 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:54.257 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:53 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:54.257 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:53 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:27:54.257 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:53 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:27:54.257 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:53 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:27:54.258 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:53 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:54.258 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:53 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:27:54.258 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:53 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:27:54.258 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:53 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:54.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:53 smithi052 ceph-mon[35071]: pgmap v233: 1 pgs: 1 unknown; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:27:54.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:53 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:54.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:53 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:54.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:53 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:27:54.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:53 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:27:54.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:53 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:27:54.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:53 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:54.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:53 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:27:54.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:53 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:27:54.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:53 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:54.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:53 smithi043 ceph-mon[31929]: pgmap v233: 1 pgs: 1 unknown; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:27:54.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:53 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:54.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:53 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:54.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:53 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:27:54.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:53 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:27:54.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:53 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:27:54.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:53 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:54.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:53 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:27:54.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:53 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:27:54.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:53 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:27:54.420 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.426+0000 7f42a6400700 1 -- 172.21.15.183:0/3304773472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a01021d0 msgr2=0x7f42a01025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:27:54.420 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.426+0000 7f42a6400700 1 --2- 172.21.15.183:0/3304773472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a01021d0 0x7f42a01025f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4290009990 tx=0x7f4290009ca0 comp rx=0 tx=0).stop 2024-03-26T08:27:54.420 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.426+0000 7f42a6400700 1 -- 172.21.15.183:0/3304773472 shutdown_connections 2024-03-26T08:27:54.420 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.426+0000 7f42a6400700 1 --2- 172.21.15.183:0/3304773472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42a0103d70 0x7f42a010a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:27:54.420 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.426+0000 7f42a6400700 1 --2- 172.21.15.183:0/3304773472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a01021d0 0x7f42a01025f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:27:54.421 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.426+0000 7f42a6400700 1 --2- 172.21.15.183:0/3304773472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42a01033d0 0x7f42a0103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:27:54.421 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.426+0000 7f42a6400700 1 -- 172.21.15.183:0/3304773472 >> 172.21.15.183:0/3304773472 conn(0x7f42a00fd730 msgr2=0x7f42a00ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:27:54.421 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.426+0000 7f42a6400700 1 -- 172.21.15.183:0/3304773472 shutdown_connections 2024-03-26T08:27:54.421 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.426+0000 7f42a6400700 1 -- 172.21.15.183:0/3304773472 wait complete. 2024-03-26T08:27:54.422 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.427+0000 7f42a6400700 1 Processor -- start 2024-03-26T08:27:54.423 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.427+0000 7f42a6400700 1 -- start start 2024-03-26T08:27:54.423 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.428+0000 7f42a6400700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42a01021d0 0x7f42a019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:27:54.423 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.428+0000 7f42a6400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42a01033d0 0x7f42a019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:27:54.423 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.428+0000 7f42a6400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a0103d70 0x7f42a01a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:27:54.423 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.428+0000 7f42a6400700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a010dcb0 con 0x7f42a01033d0 2024-03-26T08:27:54.423 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.428+0000 7f42a6400700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a010db30 con 0x7f42a01021d0 2024-03-26T08:27:54.423 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.428+0000 7f42a6400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a010de30 con 0x7f42a0103d70 2024-03-26T08:27:54.424 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.428+0000 7f429fe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a0103d70 0x7f42a01a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:27:54.424 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.428+0000 7f429fe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a0103d70 0x7f42a01a1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:42628/0 (socket says 172.21.15.183:42628) 2024-03-26T08:27:54.424 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.428+0000 7f429fe00700 1 -- 172.21.15.183:0/2062412312 learned_addr learned my addr 172.21.15.183:0/2062412312 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-03-26T08:27:54.424 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.428+0000 7f429f400700 1 --2- 172.21.15.183:0/2062412312 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42a01021d0 0x7f42a019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:27:54.424 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.428+0000 7f429ea00700 1 --2- 172.21.15.183:0/2062412312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42a01033d0 0x7f42a019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:27:54.424 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.429+0000 7f429fe00700 1 -- 172.21.15.183:0/2062412312 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42a01021d0 msgr2=0x7f42a019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:27:54.424 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.429+0000 7f429fe00700 1 --2- 172.21.15.183:0/2062412312 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42a01021d0 0x7f42a019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:27:54.424 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.429+0000 7f429fe00700 1 -- 172.21.15.183:0/2062412312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42a01033d0 msgr2=0x7f42a019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:27:54.424 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.429+0000 7f429fe00700 1 --2- 172.21.15.183:0/2062412312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42a01033d0 0x7f42a019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:27:54.425 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.429+0000 7f429fe00700 1 -- 172.21.15.183:0/2062412312 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a01a21f0 con 0x7f42a0103d70 2024-03-26T08:27:54.426 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.429+0000 7f429fe00700 1 --2- 172.21.15.183:0/2062412312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a0103d70 0x7f42a01a1b50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4298007f10 tx=0x7f429800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:27:54.426 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.429+0000 7f4297e00700 1 -- 172.21.15.183:0/2062412312 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4298004bb0 con 0x7f42a0103d70 2024-03-26T08:27:54.426 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.429+0000 7f4297e00700 1 -- 172.21.15.183:0/2062412312 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4298004d50 con 0x7f42a0103d70 2024-03-26T08:27:54.426 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.429+0000 7f42a6400700 1 -- 172.21.15.183:0/2062412312 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a01a24e0 con 0x7f42a0103d70 2024-03-26T08:27:54.426 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.429+0000 7f4297e00700 1 -- 172.21.15.183:0/2062412312 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4298005990 con 0x7f42a0103d70 2024-03-26T08:27:54.426 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.429+0000 7f42a6400700 1 -- 172.21.15.183:0/2062412312 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a01a2a20 con 0x7f42a0103d70 2024-03-26T08:27:54.427 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.431+0000 7f42a6400700 1 -- 172.21.15.183:0/2062412312 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a0068e50 con 0x7f42a0103d70 2024-03-26T08:27:54.427 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.433+0000 7f4297e00700 1 -- 172.21.15.183:0/2062412312 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f429801a020 con 0x7f42a0103d70 2024-03-26T08:27:54.428 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.434+0000 7f4297e00700 1 --2- 172.21.15.183:0/2062412312 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f42840a2910 0x7f42840a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:27:54.428 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.434+0000 7f4297e00700 1 -- 172.21.15.183:0/2062412312 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5397+0+0 (secure 0 0 0) 0x7f42980cddb0 con 0x7f42a0103d70 2024-03-26T08:27:54.429 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.434+0000 7f429f400700 1 --2- 172.21.15.183:0/2062412312 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f42840a2910 0x7f42840a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:27:54.429 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.435+0000 7f429f400700 1 --2- 172.21.15.183:0/2062412312 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f42840a2910 0x7f42840a4dd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f42900037f0 tx=0x7f4290005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:27:54.435 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.441+0000 7f4297e00700 1 -- 172.21.15.183:0/2062412312 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4298010070 con 0x7f42a0103d70 2024-03-26T08:27:54.674 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:27:54.679+0000 7f42a6400700 1 -- 172.21.15.183:0/2062412312 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f42a01a2d40 con 0x7f42840a2910 2024-03-26T08:27:55.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:54 smithi183 ceph-mon[35818]: Detected new or changed devices on smithi052 2024-03-26T08:27:55.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:54 smithi183 ceph-mon[35818]: Adjusting osd_memory_target on smithi052 to 5698M 2024-03-26T08:27:55.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:54 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:27:55.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:54 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:27:55.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:54 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:27:55.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:54 smithi052 ceph-mon[35071]: Detected new or changed devices on smithi052 2024-03-26T08:27:55.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:54 smithi052 ceph-mon[35071]: Adjusting osd_memory_target on smithi052 to 5698M 2024-03-26T08:27:55.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:54 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:27:55.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:54 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:27:55.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:54 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:27:55.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:54 smithi043 ceph-mon[31929]: Detected new or changed devices on smithi052 2024-03-26T08:27:55.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:54 smithi043 ceph-mon[31929]: Adjusting osd_memory_target on smithi052 to 5698M 2024-03-26T08:27:55.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:54 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:27:55.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:54 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:27:55.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:54 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:27:56.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:55 smithi183 ceph-mon[35818]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:27:56.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:55 smithi183 ceph-mon[35818]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:27:56.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:55 smithi183 ceph-mon[35818]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-03-26T08:27:56.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:55 smithi183 ceph-mon[35818]: Cluster is now healthy 2024-03-26T08:27:56.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:55 smithi052 ceph-mon[35071]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:27:56.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:55 smithi052 ceph-mon[35071]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:27:56.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:55 smithi052 ceph-mon[35071]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-03-26T08:27:56.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:55 smithi052 ceph-mon[35071]: Cluster is now healthy 2024-03-26T08:27:56.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:55 smithi043 ceph-mon[31929]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:27:56.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:55 smithi043 ceph-mon[31929]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:27:56.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:55 smithi043 ceph-mon[31929]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-03-26T08:27:56.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:55 smithi043 ceph-mon[31929]: Cluster is now healthy 2024-03-26T08:27:58.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:58 smithi183 ceph-mon[35818]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:27:58.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:58 smithi052 ceph-mon[35071]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:27:58.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:58 smithi043 ceph-mon[31929]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:27:59.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:59 smithi183 ceph-mon[35818]: from='client.? 172.21.15.183:0/283822389' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ddc6b07f-d703-414d-a59a-02316376b13d"}]: dispatch 2024-03-26T08:27:59.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:59 smithi183 ceph-mon[35818]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ddc6b07f-d703-414d-a59a-02316376b13d"}]: dispatch 2024-03-26T08:27:59.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:59 smithi183 ceph-mon[35818]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ddc6b07f-d703-414d-a59a-02316376b13d"}]': finished 2024-03-26T08:27:59.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:59 smithi183 ceph-mon[35818]: osdmap e57: 9 total, 8 up, 9 in 2024-03-26T08:27:59.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:27:59 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:27:59.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:59 smithi052 ceph-mon[35071]: from='client.? 172.21.15.183:0/283822389' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ddc6b07f-d703-414d-a59a-02316376b13d"}]: dispatch 2024-03-26T08:27:59.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:59 smithi052 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ddc6b07f-d703-414d-a59a-02316376b13d"}]: dispatch 2024-03-26T08:27:59.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:59 smithi052 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ddc6b07f-d703-414d-a59a-02316376b13d"}]': finished 2024-03-26T08:27:59.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:59 smithi052 ceph-mon[35071]: osdmap e57: 9 total, 8 up, 9 in 2024-03-26T08:27:59.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:27:59 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:27:59.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:59 smithi043 ceph-mon[31929]: from='client.? 172.21.15.183:0/283822389' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ddc6b07f-d703-414d-a59a-02316376b13d"}]: dispatch 2024-03-26T08:27:59.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:59 smithi043 ceph-mon[31929]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ddc6b07f-d703-414d-a59a-02316376b13d"}]: dispatch 2024-03-26T08:27:59.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:59 smithi043 ceph-mon[31929]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ddc6b07f-d703-414d-a59a-02316376b13d"}]': finished 2024-03-26T08:27:59.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:59 smithi043 ceph-mon[31929]: osdmap e57: 9 total, 8 up, 9 in 2024-03-26T08:27:59.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:27:59 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:28:00.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:00 smithi183 ceph-mon[35818]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:00.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:00 smithi183 ceph-mon[35818]: from='client.? 172.21.15.183:0/3432959326' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:28:00.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:00 smithi052 ceph-mon[35071]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:00.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:00 smithi052 ceph-mon[35071]: from='client.? 172.21.15.183:0/3432959326' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:28:00.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:00 smithi043 ceph-mon[31929]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:00.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:00 smithi043 ceph-mon[31929]: from='client.? 172.21.15.183:0/3432959326' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:28:02.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:02 smithi183 ceph-mon[35818]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:02.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:02 smithi052 ceph-mon[35071]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:02.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:02 smithi043 ceph-mon[31929]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:04.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:04 smithi183 ceph-mon[35818]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:04.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:04 smithi052 ceph-mon[35071]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:04.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:04 smithi043 ceph-mon[31929]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:06.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:06 smithi183 ceph-mon[35818]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:06.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:06 smithi052 ceph-mon[35071]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:06.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:06 smithi043 ceph-mon[31929]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:08.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:08 smithi183 ceph-mon[35818]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:08.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:08 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-26T08:28:08.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:08 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:28:08.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:08 smithi052 ceph-mon[35071]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:08.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:08 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-26T08:28:08.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:08 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:28:08.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:08 smithi043 ceph-mon[31929]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:08.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:08 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-26T08:28:08.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:08 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:28:09.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:09 smithi183 ceph-mon[35818]: Deploying daemon osd.8 on smithi183 2024-03-26T08:28:09.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:09 smithi052 ceph-mon[35071]: Deploying daemon osd.8 on smithi183 2024-03-26T08:28:09.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:09 smithi043 ceph-mon[31929]: Deploying daemon osd.8 on smithi183 2024-03-26T08:28:10.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:10 smithi052 ceph-mon[35071]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:10.338 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:10 smithi183 ceph-mon[35818]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:10.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:10 smithi043 ceph-mon[31929]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:12.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:12 smithi183 ceph-mon[35818]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:12.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:12 smithi052 ceph-mon[35071]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:12.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:12 smithi043 ceph-mon[31929]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:14.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:14 smithi052 ceph-mon[35071]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:14.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:14 smithi043 ceph-mon[31929]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:14.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:14 smithi183 ceph-mon[35818]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:16.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:16 smithi043 ceph-mon[31929]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:16.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:16 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:16.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:16 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:28:16.404 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:16 smithi183 ceph-mon[35818]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:16.404 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:16 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:16.404 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:16 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:28:16.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:16 smithi052 ceph-mon[35071]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:16.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:16 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:16.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:16 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:28:17.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:17 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:17.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:17 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:17.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:17 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:18.400 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:18 smithi183 ceph-mon[35818]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:18.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:18 smithi043 ceph-mon[31929]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:18.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:18 smithi052 ceph-mon[35071]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:20.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:20 smithi183 ceph-mon[35818]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:20.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:20 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:20.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:20 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:20.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:20 smithi183 ceph-mon[35818]: from='osd.8 [v2:172.21.15.183:6816/1023131725,v1:172.21.15.183:6817/1023131725]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T08:28:20.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:20 smithi183 ceph-mon[35818]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T08:28:20.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:20 smithi043 ceph-mon[31929]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:20.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:20 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:20.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:20 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:20.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:20 smithi043 ceph-mon[31929]: from='osd.8 [v2:172.21.15.183:6816/1023131725,v1:172.21.15.183:6817/1023131725]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T08:28:20.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:20 smithi043 ceph-mon[31929]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T08:28:20.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:20 smithi052 ceph-mon[35071]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:20.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:20 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:20.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:20 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:20.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:20 smithi052 ceph-mon[35071]: from='osd.8 [v2:172.21.15.183:6816/1023131725,v1:172.21.15.183:6817/1023131725]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T08:28:20.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:20 smithi052 ceph-mon[35071]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T08:28:20.795 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:28:20.801+0000 7f4297e00700 1 -- 172.21.15.183:0/2062412312 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f42a01a2d40 con 0x7f42840a2910 2024-03-26T08:28:20.795 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 8 on host 'smithi183' 2024-03-26T08:28:20.801 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:28:20.806+0000 7f42a6400700 1 -- 172.21.15.183:0/2062412312 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f42840a2910 msgr2=0x7f42840a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:28:20.801 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:28:20.806+0000 7f42a6400700 1 --2- 172.21.15.183:0/2062412312 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f42840a2910 0x7f42840a4dd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f42900037f0 tx=0x7f4290005fd0 comp rx=0 tx=0).stop 2024-03-26T08:28:20.801 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:28:20.806+0000 7f42a6400700 1 -- 172.21.15.183:0/2062412312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a0103d70 msgr2=0x7f42a01a1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:28:20.801 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:28:20.806+0000 7f42a6400700 1 --2- 172.21.15.183:0/2062412312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a0103d70 0x7f42a01a1b50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4298007f10 tx=0x7f429800d3b0 comp rx=0 tx=0).stop 2024-03-26T08:28:20.801 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:28:20.806+0000 7f42a6400700 1 -- 172.21.15.183:0/2062412312 shutdown_connections 2024-03-26T08:28:20.802 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:28:20.806+0000 7f42a6400700 1 --2- 172.21.15.183:0/2062412312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a0103d70 0x7f42a01a1b50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:28:20.802 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:28:20.806+0000 7f42a6400700 1 --2- 172.21.15.183:0/2062412312 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f42840a2910 0x7f42840a4dd0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:28:20.802 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:28:20.806+0000 7f42a6400700 1 --2- 172.21.15.183:0/2062412312 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42a01021d0 0x7f42a019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:28:20.802 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:28:20.806+0000 7f42a6400700 1 --2- 172.21.15.183:0/2062412312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42a01033d0 0x7f42a019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:28:20.802 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:28:20.806+0000 7f42a6400700 1 -- 172.21.15.183:0/2062412312 >> 172.21.15.183:0/2062412312 conn(0x7f42a00fd730 msgr2=0x7f42a00743c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:28:20.802 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:28:20.806+0000 7f42a6400700 1 -- 172.21.15.183:0/2062412312 shutdown_connections 2024-03-26T08:28:20.802 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:28:20.806+0000 7f42a6400700 1 -- 172.21.15.183:0/2062412312 wait complete. 2024-03-26T08:28:21.667 DEBUG:teuthology.orchestra.run.smithi183:osd.8> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.8.service 2024-03-26T08:28:21.673 INFO:tasks.cephadm:Deploying osd.9 on smithi043 with /dev/vg_nvme/lv_1... 2024-03-26T08:28:21.673 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-26T08:28:21.919 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:21 smithi183 ceph-mon[35818]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:21.920 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:21 smithi183 ceph-mon[35818]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-26T08:28:21.920 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:21 smithi183 ceph-mon[35818]: from='osd.8 [v2:172.21.15.183:6816/1023131725,v1:172.21.15.183:6817/1023131725]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-26T08:28:21.920 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:21 smithi183 ceph-mon[35818]: osdmap e58: 9 total, 8 up, 9 in 2024-03-26T08:28:21.920 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:21 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:28:21.920 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:21 smithi183 ceph-mon[35818]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-26T08:28:21.920 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:21 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:21.920 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:21 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:22.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:21 smithi052 ceph-mon[35071]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:22.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:21 smithi052 ceph-mon[35071]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-26T08:28:22.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:21 smithi052 ceph-mon[35071]: from='osd.8 [v2:172.21.15.183:6816/1023131725,v1:172.21.15.183:6817/1023131725]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-26T08:28:22.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:21 smithi052 ceph-mon[35071]: osdmap e58: 9 total, 8 up, 9 in 2024-03-26T08:28:22.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:21 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:28:22.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:21 smithi052 ceph-mon[35071]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-26T08:28:22.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:21 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:22.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:21 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:22.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:21 smithi043 ceph-mon[31929]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:22.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:21 smithi043 ceph-mon[31929]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-26T08:28:22.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:21 smithi043 ceph-mon[31929]: from='osd.8 [v2:172.21.15.183:6816/1023131725,v1:172.21.15.183:6817/1023131725]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-26T08:28:22.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:21 smithi043 ceph-mon[31929]: osdmap e58: 9 total, 8 up, 9 in 2024-03-26T08:28:22.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:21 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:28:22.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:21 smithi043 ceph-mon[31929]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-26T08:28:22.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:21 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:22.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:21 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:22.163 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:28:23.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:22 smithi183 ceph-mon[35818]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-03-26T08:28:23.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:22 smithi183 ceph-mon[35818]: osdmap e59: 9 total, 8 up, 9 in 2024-03-26T08:28:23.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:22 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:28:23.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:22 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:28:23.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:22 smithi052 ceph-mon[35071]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-03-26T08:28:23.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:22 smithi052 ceph-mon[35071]: osdmap e59: 9 total, 8 up, 9 in 2024-03-26T08:28:23.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:22 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:28:23.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:22 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:28:23.106 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:22 smithi043 ceph-mon[31929]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-03-26T08:28:23.106 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:22 smithi043 ceph-mon[31929]: osdmap e59: 9 total, 8 up, 9 in 2024-03-26T08:28:23.106 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:22 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:28:23.107 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:22 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:28:23.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:23 smithi043 ceph-mon[31929]: purged_snaps scrub starts 2024-03-26T08:28:23.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:23 smithi043 ceph-mon[31929]: purged_snaps scrub ok 2024-03-26T08:28:23.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:23 smithi043 ceph-mon[31929]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:23.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:23 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:28:23.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:23 smithi043 ceph-mon[31929]: osd.8 [v2:172.21.15.183:6816/1023131725,v1:172.21.15.183:6817/1023131725] boot 2024-03-26T08:28:23.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:23 smithi043 ceph-mon[31929]: osdmap e60: 9 total, 9 up, 9 in 2024-03-26T08:28:23.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:23 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:28:24.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:23 smithi183 ceph-mon[35818]: purged_snaps scrub starts 2024-03-26T08:28:24.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:23 smithi183 ceph-mon[35818]: purged_snaps scrub ok 2024-03-26T08:28:24.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:23 smithi183 ceph-mon[35818]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:24.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:23 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:28:24.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:23 smithi183 ceph-mon[35818]: osd.8 [v2:172.21.15.183:6816/1023131725,v1:172.21.15.183:6817/1023131725] boot 2024-03-26T08:28:24.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:23 smithi183 ceph-mon[35818]: osdmap e60: 9 total, 9 up, 9 in 2024-03-26T08:28:24.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:23 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:28:24.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:23 smithi052 ceph-mon[35071]: purged_snaps scrub starts 2024-03-26T08:28:24.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:23 smithi052 ceph-mon[35071]: purged_snaps scrub ok 2024-03-26T08:28:24.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:23 smithi052 ceph-mon[35071]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:28:24.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:23 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:28:24.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:23 smithi052 ceph-mon[35071]: osd.8 [v2:172.21.15.183:6816/1023131725,v1:172.21.15.183:6817/1023131725] boot 2024-03-26T08:28:24.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:23 smithi052 ceph-mon[35071]: osdmap e60: 9 total, 9 up, 9 in 2024-03-26T08:28:24.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:23 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:28:25.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:24 smithi183 ceph-mon[35818]: osdmap e61: 9 total, 9 up, 9 in 2024-03-26T08:28:25.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:24 smithi052 ceph-mon[35071]: osdmap e61: 9 total, 9 up, 9 in 2024-03-26T08:28:25.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:24 smithi043 ceph-mon[31929]: osdmap e61: 9 total, 9 up, 9 in 2024-03-26T08:28:25.166 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-26T08:28:25.166 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-26T08:28:25.166 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2024-03-26T08:28:25.166 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2024-03-26T08:28:25.167 INFO:teuthology.orchestra.run.smithi043.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0382999 s, 274 MB/s 2024-03-26T08:28:25.167 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2024-03-26T08:28:25.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:25 smithi043 ceph-mon[31929]: pgmap v255: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:25.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:25 smithi043 ceph-mon[31929]: osdmap e62: 9 total, 9 up, 9 in 2024-03-26T08:28:26.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:25 smithi183 ceph-mon[35818]: pgmap v255: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:26.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:25 smithi183 ceph-mon[35818]: osdmap e62: 9 total, 9 up, 9 in 2024-03-26T08:28:26.075 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:25 smithi052 ceph-mon[35071]: pgmap v255: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:26.075 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:25 smithi052 ceph-mon[35071]: osdmap e62: 9 total, 9 up, 9 in 2024-03-26T08:28:26.092 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi043:vg_nvme/lv_1 2024-03-26T08:28:26.563 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:28:27.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:26 smithi183 ceph-mon[35818]: osdmap e63: 9 total, 9 up, 9 in 2024-03-26T08:28:27.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:26 smithi183 ceph-mon[35818]: Detected new or changed devices on smithi183 2024-03-26T08:28:27.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:26 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:27.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:26 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:27.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:26 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:28:27.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:26 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:28:27.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:26 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:28:27.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:26 smithi183 ceph-mon[35818]: Adjusting osd_memory_target on smithi183 to 5697M 2024-03-26T08:28:27.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:26 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:27.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:26 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:28:27.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:26 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:28:27.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:26 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:27.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:26 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:28:27.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:26 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:28:27.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:26 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:28:27.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:26 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:27.056 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:26 smithi043 ceph-mon[31929]: osdmap e63: 9 total, 9 up, 9 in 2024-03-26T08:28:27.056 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:26 smithi043 ceph-mon[31929]: Detected new or changed devices on smithi183 2024-03-26T08:28:27.056 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:26 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:27.056 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:26 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:27.056 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:26 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:28:27.056 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:26 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:28:27.057 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:26 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:28:27.057 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:26 smithi043 ceph-mon[31929]: Adjusting osd_memory_target on smithi183 to 5697M 2024-03-26T08:28:27.057 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:26 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:27.057 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:26 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:28:27.057 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:26 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:28:27.057 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:26 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:27.057 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:26 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:28:27.057 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:26 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:28:27.057 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:26 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:28:27.057 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:26 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:27.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:26 smithi052 ceph-mon[35071]: osdmap e63: 9 total, 9 up, 9 in 2024-03-26T08:28:27.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:26 smithi052 ceph-mon[35071]: Detected new or changed devices on smithi183 2024-03-26T08:28:27.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:26 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:27.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:26 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:27.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:26 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:28:27.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:26 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:28:27.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:26 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:28:27.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:26 smithi052 ceph-mon[35071]: Adjusting osd_memory_target on smithi183 to 5697M 2024-03-26T08:28:27.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:26 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:27.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:26 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:28:27.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:26 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:28:27.075 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:26 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:27.075 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:26 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:28:27.075 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:26 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:28:27.075 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:26 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:28:27.075 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:26 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:27.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:27 smithi043 ceph-mon[31929]: pgmap v258: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:27.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:27 smithi043 ceph-mon[31929]: osdmap e64: 9 total, 9 up, 9 in 2024-03-26T08:28:28.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:27 smithi183 ceph-mon[35818]: pgmap v258: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:28.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:27 smithi183 ceph-mon[35818]: osdmap e64: 9 total, 9 up, 9 in 2024-03-26T08:28:28.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:27 smithi052 ceph-mon[35071]: pgmap v258: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:28.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:27 smithi052 ceph-mon[35071]: osdmap e64: 9 total, 9 up, 9 in 2024-03-26T08:28:28.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.882+0000 7f51f5e00700 1 -- 172.21.15.43:0/1898965477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51f01055d0 msgr2=0x7f51f0105a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:28:28.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.882+0000 7f51f5e00700 1 --2- 172.21.15.43:0/1898965477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51f01055d0 0x7f51f0105a30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f51dc0099c0 tx=0x7f51dc009cd0 comp rx=0 tx=0).stop 2024-03-26T08:28:28.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.882+0000 7f51f5e00700 1 -- 172.21.15.43:0/1898965477 shutdown_connections 2024-03-26T08:28:28.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.882+0000 7f51f5e00700 1 --2- 172.21.15.43:0/1898965477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51f0105f70 0x7f51f010c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:28:28.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.882+0000 7f51f5e00700 1 --2- 172.21.15.43:0/1898965477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51f01043d0 0x7f51f01047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:28:28.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.882+0000 7f51f5e00700 1 --2- 172.21.15.43:0/1898965477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51f01055d0 0x7f51f0105a30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:28:28.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.882+0000 7f51f5e00700 1 -- 172.21.15.43:0/1898965477 >> 172.21.15.43:0/1898965477 conn(0x7f51f00ff950 msgr2=0x7f51f0101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:28:28.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.882+0000 7f51f5e00700 1 -- 172.21.15.43:0/1898965477 shutdown_connections 2024-03-26T08:28:28.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.883+0000 7f51f5e00700 1 -- 172.21.15.43:0/1898965477 wait complete. 2024-03-26T08:28:28.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.884+0000 7f51f5e00700 1 Processor -- start 2024-03-26T08:28:28.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.884+0000 7f51f5e00700 1 -- start start 2024-03-26T08:28:28.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.885+0000 7f51f5e00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51f01043d0 0x7f51f019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:28:28.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.885+0000 7f51f5e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51f01055d0 0x7f51f019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:28:28.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.885+0000 7f51f5e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51f0105f70 0x7f51f01a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:28:28.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.885+0000 7f51f5e00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f010fe50 con 0x7f51f01055d0 2024-03-26T08:28:28.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.885+0000 7f51f5e00700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f010fcd0 con 0x7f51f01043d0 2024-03-26T08:28:28.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.885+0000 7f51f5e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f010ffd0 con 0x7f51f0105f70 2024-03-26T08:28:28.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.885+0000 7f51ee000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51f01055d0 0x7f51f019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:28:28.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.885+0000 7f51ee000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51f01055d0 0x7f51f019a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53952/0 (socket says 172.21.15.43:53952) 2024-03-26T08:28:28.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.885+0000 7f51ee000700 1 -- 172.21.15.43:0/3333319892 learned_addr learned my addr 172.21.15.43:0/3333319892 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:28:28.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.885+0000 7f51eea00700 1 --2- 172.21.15.43:0/3333319892 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51f01043d0 0x7f51f019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:28:28.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.886+0000 7f51ef400700 1 --2- 172.21.15.43:0/3333319892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51f0105f70 0x7f51f01a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:28:28.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.886+0000 7f51ee000700 1 -- 172.21.15.43:0/3333319892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51f0105f70 msgr2=0x7f51f01a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:28:28.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.886+0000 7f51ee000700 1 --2- 172.21.15.43:0/3333319892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51f0105f70 0x7f51f01a19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:28:28.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.886+0000 7f51ee000700 1 -- 172.21.15.43:0/3333319892 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51f01043d0 msgr2=0x7f51f019a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:28:28.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.886+0000 7f51ee000700 1 --2- 172.21.15.43:0/3333319892 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51f01043d0 0x7f51f019a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:28:28.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.886+0000 7f51ee000700 1 -- 172.21.15.43:0/3333319892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f01a2090 con 0x7f51f01055d0 2024-03-26T08:28:28.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.886+0000 7f51ee000700 1 --2- 172.21.15.43:0/3333319892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51f01055d0 0x7f51f019a970 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f51dc000c00 tx=0x7f51dc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:28:28.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.887+0000 7f51f4a00700 1 -- 172.21.15.43:0/3333319892 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc027070 con 0x7f51f01055d0 2024-03-26T08:28:28.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.887+0000 7f51f4a00700 1 -- 172.21.15.43:0/3333319892 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f51dc004050 con 0x7f51f01055d0 2024-03-26T08:28:28.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.887+0000 7f51f5e00700 1 -- 172.21.15.43:0/3333319892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f01a2320 con 0x7f51f01055d0 2024-03-26T08:28:28.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.887+0000 7f51f4a00700 1 -- 172.21.15.43:0/3333319892 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc022dd0 con 0x7f51f01055d0 2024-03-26T08:28:28.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.887+0000 7f51f5e00700 1 -- 172.21.15.43:0/3333319892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f01a27d0 con 0x7f51f01055d0 2024-03-26T08:28:28.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.888+0000 7f51f5e00700 1 -- 172.21.15.43:0/3333319892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f0068e50 con 0x7f51f01055d0 2024-03-26T08:28:28.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.890+0000 7f51f4a00700 1 -- 172.21.15.43:0/3333319892 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f51dc02a070 con 0x7f51f01055d0 2024-03-26T08:28:28.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.891+0000 7f51f4a00700 1 --2- 172.21.15.43:0/3333319892 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f51d40a2910 0x7f51d40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:28:28.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.892+0000 7f51f4a00700 1 -- 172.21.15.43:0/3333319892 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5829+0+0 (secure 0 0 0) 0x7f51dc0cf400 con 0x7f51f01055d0 2024-03-26T08:28:28.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.892+0000 7f51eea00700 1 --2- 172.21.15.43:0/3333319892 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f51d40a2910 0x7f51d40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:28:28.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.893+0000 7f51eea00700 1 --2- 172.21.15.43:0/3333319892 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f51d40a2910 0x7f51d40a4dd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f51f0105430 tx=0x7f51e000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:28:28.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:28.895+0000 7f51f4a00700 1 -- 172.21.15.43:0/3333319892 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f51dc031070 con 0x7f51f01055d0 2024-03-26T08:28:29.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:29.132+0000 7f51f5e00700 1 -- 172.21.15.43:0/3333319892 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f51f00625d0 con 0x7f51d40a2910 2024-03-26T08:28:30.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:29 smithi183 ceph-mon[35818]: pgmap v260: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:30.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:29 smithi183 ceph-mon[35818]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:28:30.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:29 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:28:30.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:29 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:28:30.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:29 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:28:30.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:29 smithi052 ceph-mon[35071]: pgmap v260: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:30.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:29 smithi052 ceph-mon[35071]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:28:30.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:29 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:28:30.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:29 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:28:30.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:29 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:28:30.085 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:29 smithi043 ceph-mon[31929]: pgmap v260: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:30.086 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:29 smithi043 ceph-mon[31929]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:28:30.086 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:29 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:28:30.086 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:29 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:28:30.086 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:29 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:28:32.106 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:31 smithi043 ceph-mon[31929]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T08:28:32.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:31 smithi183 ceph-mon[35818]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T08:28:32.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:31 smithi052 ceph-mon[35071]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T08:28:34.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:33 smithi183 ceph-mon[35818]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-26T08:28:34.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:33 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2424325529' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac16bb0e-6232-45c8-b16a-4d878874c588"}]: dispatch 2024-03-26T08:28:34.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:33 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2424325529' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac16bb0e-6232-45c8-b16a-4d878874c588"}]': finished 2024-03-26T08:28:34.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:33 smithi183 ceph-mon[35818]: osdmap e65: 10 total, 9 up, 10 in 2024-03-26T08:28:34.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:33 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:28:34.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:33 smithi052 ceph-mon[35071]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-26T08:28:34.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:33 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2424325529' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac16bb0e-6232-45c8-b16a-4d878874c588"}]: dispatch 2024-03-26T08:28:34.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:33 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2424325529' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac16bb0e-6232-45c8-b16a-4d878874c588"}]': finished 2024-03-26T08:28:34.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:33 smithi052 ceph-mon[35071]: osdmap e65: 10 total, 9 up, 10 in 2024-03-26T08:28:34.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:33 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:28:34.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:33 smithi043 ceph-mon[31929]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-26T08:28:34.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:33 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2424325529' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac16bb0e-6232-45c8-b16a-4d878874c588"}]: dispatch 2024-03-26T08:28:34.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:33 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2424325529' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac16bb0e-6232-45c8-b16a-4d878874c588"}]': finished 2024-03-26T08:28:34.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:33 smithi043 ceph-mon[31929]: osdmap e65: 10 total, 9 up, 10 in 2024-03-26T08:28:34.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:33 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:28:35.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:34 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2008112842' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:28:35.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:34 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2008112842' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:28:35.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:34 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2008112842' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:28:36.137 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:35 smithi043 ceph-mon[31929]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T08:28:36.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:35 smithi183 ceph-mon[35818]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T08:28:36.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:35 smithi052 ceph-mon[35071]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T08:28:38.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:38 smithi043 ceph-mon[31929]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-03-26T08:28:38.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:38 smithi183 ceph-mon[35818]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-03-26T08:28:38.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:38 smithi052 ceph-mon[35071]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-03-26T08:28:40.258 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:40 smithi043 ceph-mon[31929]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T08:28:40.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:40 smithi183 ceph-mon[35818]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T08:28:40.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:40 smithi052 ceph-mon[35071]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T08:28:42.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:42 smithi183 ceph-mon[35818]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:42.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:42 smithi052 ceph-mon[35071]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:42.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:42 smithi043 ceph-mon[31929]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:44.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:44 smithi183 ceph-mon[35818]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:44.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:44 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-26T08:28:44.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:44 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:28:44.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:44 smithi052 ceph-mon[35071]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:44.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:44 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-26T08:28:44.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:44 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:28:44.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:44 smithi043 ceph-mon[31929]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:44.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:44 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-26T08:28:44.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:44 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:28:45.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:45 smithi043 ceph-mon[31929]: Deploying daemon osd.9 on smithi043 2024-03-26T08:28:45.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:45 smithi183 ceph-mon[35818]: Deploying daemon osd.9 on smithi043 2024-03-26T08:28:45.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:45 smithi052 ceph-mon[35071]: Deploying daemon osd.9 on smithi043 2024-03-26T08:28:46.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:46 smithi043 ceph-mon[31929]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:46.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:46 smithi183 ceph-mon[35818]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:46.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:46 smithi052 ceph-mon[35071]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:48.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:48 smithi183 ceph-mon[35818]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:48.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:48 smithi052 ceph-mon[35071]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:48.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:48 smithi043 ceph-mon[31929]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:50.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:50 smithi183 ceph-mon[35818]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:50.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:50 smithi052 ceph-mon[35071]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:50.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:50 smithi043 ceph-mon[31929]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:52.221 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:52 smithi043 ceph-mon[31929]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:52.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:52 smithi183 ceph-mon[35818]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:52.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:52 smithi052 ceph-mon[35071]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:53.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:53 smithi183 ceph-mon[35818]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:53.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:53 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:53.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:53 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:28:53.801 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:53 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:53.819 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:53 smithi043 ceph-mon[31929]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:53.819 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:53 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:53.819 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:53 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:28:53.819 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:53 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:53.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:53 smithi052 ceph-mon[35071]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:53.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:53 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:53.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:53 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:28:53.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:53 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:55.787 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:55 smithi043 ceph-mon[31929]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:55.801 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:55 smithi183 ceph-mon[35818]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:55.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:55 smithi052 ceph-mon[35071]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:57.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:57 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:57.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:57 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:57.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:57 smithi043 ceph-mon[31929]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:57.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:57 smithi043 ceph-mon[31929]: from='osd.9 [v2:172.21.15.43:6826/3935195657,v1:172.21.15.43:6827/3935195657]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-26T08:28:57.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:57 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:57.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:57 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:57.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:57 smithi183 ceph-mon[35818]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:57.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:57 smithi183 ceph-mon[35818]: from='osd.9 [v2:172.21.15.43:6826/3935195657,v1:172.21.15.43:6827/3935195657]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-26T08:28:57.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:57 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:57.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:57 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:57.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:57 smithi052 ceph-mon[35071]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:57.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:57 smithi052 ceph-mon[35071]: from='osd.9 [v2:172.21.15.43:6826/3935195657,v1:172.21.15.43:6827/3935195657]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-26T08:28:57.574 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 9 on host 'smithi043' 2024-03-26T08:28:57.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:57.580+0000 7f51f4a00700 1 -- 172.21.15.43:0/3333319892 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f51f00625d0 con 0x7f51d40a2910 2024-03-26T08:28:57.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:57.583+0000 7f51f5e00700 1 -- 172.21.15.43:0/3333319892 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f51d40a2910 msgr2=0x7f51d40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:28:57.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:57.583+0000 7f51f5e00700 1 --2- 172.21.15.43:0/3333319892 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f51d40a2910 0x7f51d40a4dd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f51f0105430 tx=0x7f51e000b410 comp rx=0 tx=0).stop 2024-03-26T08:28:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:57.584+0000 7f51f5e00700 1 -- 172.21.15.43:0/3333319892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51f01055d0 msgr2=0x7f51f019a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:28:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:57.584+0000 7f51f5e00700 1 --2- 172.21.15.43:0/3333319892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51f01055d0 0x7f51f019a970 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f51dc000c00 tx=0x7f51dc0047b0 comp rx=0 tx=0).stop 2024-03-26T08:28:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:57.584+0000 7f51f5e00700 1 -- 172.21.15.43:0/3333319892 shutdown_connections 2024-03-26T08:28:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:57.584+0000 7f51f5e00700 1 --2- 172.21.15.43:0/3333319892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51f0105f70 0x7f51f01a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:28:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:57.584+0000 7f51f5e00700 1 --2- 172.21.15.43:0/3333319892 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f51d40a2910 0x7f51d40a4dd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:28:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:57.584+0000 7f51f5e00700 1 --2- 172.21.15.43:0/3333319892 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51f01043d0 0x7f51f019a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:28:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:57.584+0000 7f51f5e00700 1 --2- 172.21.15.43:0/3333319892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51f01055d0 0x7f51f019a970 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:28:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:57.584+0000 7f51f5e00700 1 -- 172.21.15.43:0/3333319892 >> 172.21.15.43:0/3333319892 conn(0x7f51f00ff950 msgr2=0x7f51f01088d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:28:57.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:57.584+0000 7f51f5e00700 1 -- 172.21.15.43:0/3333319892 shutdown_connections 2024-03-26T08:28:57.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:28:57.584+0000 7f51f5e00700 1 -- 172.21.15.43:0/3333319892 wait complete. 2024-03-26T08:28:58.317 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:58 smithi043 ceph-mon[31929]: from='osd.9 [v2:172.21.15.43:6826/3935195657,v1:172.21.15.43:6827/3935195657]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-26T08:28:58.317 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:58 smithi043 ceph-mon[31929]: osdmap e66: 10 total, 9 up, 10 in 2024-03-26T08:28:58.317 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:58 smithi043 ceph-mon[31929]: from='osd.9 [v2:172.21.15.43:6826/3935195657,v1:172.21.15.43:6827/3935195657]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-26T08:28:58.317 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:58 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:28:58.317 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:58 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:58.317 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:58 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:58.488 DEBUG:teuthology.orchestra.run.smithi043:osd.9> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.9.service 2024-03-26T08:28:58.493 INFO:tasks.cephadm:Deploying osd.10 on smithi052 with /dev/vg_nvme/lv_1... 2024-03-26T08:28:58.493 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-26T08:28:58.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:58 smithi183 ceph-mon[35818]: from='osd.9 [v2:172.21.15.43:6826/3935195657,v1:172.21.15.43:6827/3935195657]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-26T08:28:58.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:58 smithi183 ceph-mon[35818]: osdmap e66: 10 total, 9 up, 10 in 2024-03-26T08:28:58.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:58 smithi183 ceph-mon[35818]: from='osd.9 [v2:172.21.15.43:6826/3935195657,v1:172.21.15.43:6827/3935195657]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-26T08:28:58.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:58 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:28:58.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:58 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:58.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:58 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:58.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:58 smithi052 ceph-mon[35071]: from='osd.9 [v2:172.21.15.43:6826/3935195657,v1:172.21.15.43:6827/3935195657]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-26T08:28:58.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:58 smithi052 ceph-mon[35071]: osdmap e66: 10 total, 9 up, 10 in 2024-03-26T08:28:58.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:58 smithi052 ceph-mon[35071]: from='osd.9 [v2:172.21.15.43:6826/3935195657,v1:172.21.15.43:6827/3935195657]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-26T08:28:58.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:58 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:28:58.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:58 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:58.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:58 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:28:58.954 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T08:28:59.270 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:59 smithi043 ceph-mon[31929]: from='osd.9 [v2:172.21.15.43:6826/3935195657,v1:172.21.15.43:6827/3935195657]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-03-26T08:28:59.270 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:59 smithi043 ceph-mon[31929]: osdmap e67: 10 total, 9 up, 10 in 2024-03-26T08:28:59.270 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:59 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:28:59.270 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:59 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:28:59.270 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:28:59 smithi043 ceph-mon[31929]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:59.431 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:59 smithi052 ceph-mon[35071]: from='osd.9 [v2:172.21.15.43:6826/3935195657,v1:172.21.15.43:6827/3935195657]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-03-26T08:28:59.432 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:59 smithi052 ceph-mon[35071]: osdmap e67: 10 total, 9 up, 10 in 2024-03-26T08:28:59.432 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:59 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:28:59.432 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:59 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:28:59.432 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:28:59 smithi052 ceph-mon[35071]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:28:59.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:59 smithi183 ceph-mon[35818]: from='osd.9 [v2:172.21.15.43:6826/3935195657,v1:172.21.15.43:6827/3935195657]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-03-26T08:28:59.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:59 smithi183 ceph-mon[35818]: osdmap e67: 10 total, 9 up, 10 in 2024-03-26T08:28:59.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:59 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:28:59.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:59 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:28:59.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:28:59 smithi183 ceph-mon[35818]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:29:00.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:00 smithi183 ceph-mon[35818]: purged_snaps scrub starts 2024-03-26T08:29:00.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:00 smithi183 ceph-mon[35818]: purged_snaps scrub ok 2024-03-26T08:29:00.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:00 smithi183 ceph-mon[35818]: osd.9 [v2:172.21.15.43:6826/3935195657,v1:172.21.15.43:6827/3935195657] boot 2024-03-26T08:29:00.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:00 smithi183 ceph-mon[35818]: osdmap e68: 10 total, 10 up, 10 in 2024-03-26T08:29:00.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:00 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:29:00.550 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:00 smithi043 ceph-mon[31929]: purged_snaps scrub starts 2024-03-26T08:29:00.550 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:00 smithi043 ceph-mon[31929]: purged_snaps scrub ok 2024-03-26T08:29:00.550 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:00 smithi043 ceph-mon[31929]: osd.9 [v2:172.21.15.43:6826/3935195657,v1:172.21.15.43:6827/3935195657] boot 2024-03-26T08:29:00.550 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:00 smithi043 ceph-mon[31929]: osdmap e68: 10 total, 10 up, 10 in 2024-03-26T08:29:00.550 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:00 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:29:00.568 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:00 smithi052 ceph-mon[35071]: purged_snaps scrub starts 2024-03-26T08:29:00.569 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:00 smithi052 ceph-mon[35071]: purged_snaps scrub ok 2024-03-26T08:29:00.569 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:00 smithi052 ceph-mon[35071]: osd.9 [v2:172.21.15.43:6826/3935195657,v1:172.21.15.43:6827/3935195657] boot 2024-03-26T08:29:00.569 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:00 smithi052 ceph-mon[35071]: osdmap e68: 10 total, 10 up, 10 in 2024-03-26T08:29:00.569 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:00 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:29:01.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:01 smithi183 ceph-mon[35818]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:01.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:01 smithi183 ceph-mon[35818]: osdmap e69: 10 total, 10 up, 10 in 2024-03-26T08:29:01.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:01 smithi052 ceph-mon[35071]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:01.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:01 smithi052 ceph-mon[35071]: osdmap e69: 10 total, 10 up, 10 in 2024-03-26T08:29:01.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:01 smithi043 ceph-mon[31929]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:01.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:01 smithi043 ceph-mon[31929]: osdmap e69: 10 total, 10 up, 10 in 2024-03-26T08:29:02.112 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-26T08:29:02.113 INFO:teuthology.orchestra.run.smithi052.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-26T08:29:02.113 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10+0 records in 2024-03-26T08:29:02.113 INFO:teuthology.orchestra.run.smithi052.stderr:10+0 records out 2024-03-26T08:29:02.113 INFO:teuthology.orchestra.run.smithi052.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0369018 s, 284 MB/s 2024-03-26T08:29:02.113 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping successful for: 2024-03-26T08:29:03.092 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi052:vg_nvme/lv_1 2024-03-26T08:29:03.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:03 smithi043 ceph-mon[31929]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:03.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:03 smithi183 ceph-mon[35818]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:03.567 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T08:29:03.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:03 smithi052 ceph-mon[35071]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:04.856 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:04 smithi043 ceph-mon[31929]: Detected new or changed devices on smithi043 2024-03-26T08:29:04.856 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:04 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:04.856 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:04 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:04.856 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:04 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:29:04.857 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:04 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:29:04.857 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:04 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:29:04.857 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:04 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:29:04.857 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:04 smithi043 ceph-mon[31929]: Adjusting osd_memory_target on smithi043 to 4273M 2024-03-26T08:29:04.857 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:04 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:04.857 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:04 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:29:04.857 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:04 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:29:04.857 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:04 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:04.857 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:04 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:29:04.857 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:04 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:29:04.857 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:04 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:29:04.857 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:04 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:05.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:04 smithi183 ceph-mon[35818]: Detected new or changed devices on smithi043 2024-03-26T08:29:05.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:04 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:05.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:04 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:05.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:04 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:29:05.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:04 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:29:05.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:04 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:29:05.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:04 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:29:05.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:04 smithi183 ceph-mon[35818]: Adjusting osd_memory_target on smithi043 to 4273M 2024-03-26T08:29:05.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:04 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:05.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:04 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:29:05.051 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:04 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:29:05.051 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:04 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:05.051 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:04 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:29:05.051 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:04 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:29:05.051 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:04 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:29:05.051 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:04 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:05.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:04 smithi052 ceph-mon[35071]: Detected new or changed devices on smithi043 2024-03-26T08:29:05.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:04 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:05.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:04 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:05.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:04 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:29:05.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:04 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:29:05.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:04 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:29:05.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:04 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:29:05.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:04 smithi052 ceph-mon[35071]: Adjusting osd_memory_target on smithi043 to 4273M 2024-03-26T08:29:05.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:04 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:05.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:04 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:29:05.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:04 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:29:05.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:04 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:05.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:04 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:29:05.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:04 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:29:05.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:04 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:29:05.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:04 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:06.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.129+0000 7fc941c00700 1 -- 172.21.15.52:0/3923726397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc93c105fb0 msgr2=0x7fc93c10c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:29:06.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.129+0000 7fc941c00700 1 --2- 172.21.15.52:0/3923726397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc93c105fb0 0x7fc93c10c860 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc9340099c0 tx=0x7fc934009cd0 comp rx=0 tx=0).stop 2024-03-26T08:29:06.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.130+0000 7fc941c00700 1 -- 172.21.15.52:0/3923726397 shutdown_connections 2024-03-26T08:29:06.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.130+0000 7fc941c00700 1 --2- 172.21.15.52:0/3923726397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc93c105fb0 0x7fc93c10c860 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:29:06.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.130+0000 7fc941c00700 1 --2- 172.21.15.52:0/3923726397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc93c104410 0x7fc93c104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:29:06.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.130+0000 7fc941c00700 1 --2- 172.21.15.52:0/3923726397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc93c105610 0x7fc93c105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:29:06.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.130+0000 7fc941c00700 1 -- 172.21.15.52:0/3923726397 >> 172.21.15.52:0/3923726397 conn(0x7fc93c0ff9b0 msgr2=0x7fc93c101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:29:06.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.130+0000 7fc941c00700 1 -- 172.21.15.52:0/3923726397 shutdown_connections 2024-03-26T08:29:06.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.130+0000 7fc941c00700 1 -- 172.21.15.52:0/3923726397 wait complete. 2024-03-26T08:29:06.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.131+0000 7fc941c00700 1 Processor -- start 2024-03-26T08:29:06.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.131+0000 7fc941c00700 1 -- start start 2024-03-26T08:29:06.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.132+0000 7fc941c00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc93c104410 0x7fc93c19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:29:06.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.132+0000 7fc941c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc93c105610 0x7fc93c19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:29:06.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.132+0000 7fc941c00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc93c105fb0 0x7fc93c1a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:29:06.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.132+0000 7fc941c00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c10fee0 con 0x7fc93c105610 2024-03-26T08:29:06.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.132+0000 7fc941c00700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c10fd60 con 0x7fc93c105fb0 2024-03-26T08:29:06.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.132+0000 7fc941c00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c110060 con 0x7fc93c104410 2024-03-26T08:29:06.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.132+0000 7fc93b400700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc93c105fb0 0x7fc93c1a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:29:06.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.132+0000 7fc93b400700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc93c105fb0 0x7fc93c1a1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56576/0 (socket says 172.21.15.52:56576) 2024-03-26T08:29:06.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.132+0000 7fc93b400700 1 -- 172.21.15.52:0/2934790984 learned_addr learned my addr 172.21.15.52:0/2934790984 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-26T08:29:06.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.132+0000 7fc932000700 1 --2- 172.21.15.52:0/2934790984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc93c105610 0x7fc93c19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:29:06.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.132+0000 7fc93aa00700 1 --2- 172.21.15.52:0/2934790984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc93c104410 0x7fc93c19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:29:06.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.132+0000 7fc93b400700 1 -- 172.21.15.52:0/2934790984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc93c104410 msgr2=0x7fc93c19a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:29:06.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.132+0000 7fc93b400700 1 --2- 172.21.15.52:0/2934790984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc93c104410 0x7fc93c19a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:29:06.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.132+0000 7fc93b400700 1 -- 172.21.15.52:0/2934790984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc93c105610 msgr2=0x7fc93c19a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:29:06.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.132+0000 7fc93b400700 1 --2- 172.21.15.52:0/2934790984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc93c105610 0x7fc93c19a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:29:06.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.132+0000 7fc93b400700 1 -- 172.21.15.52:0/2934790984 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93c1a20a0 con 0x7fc93c105fb0 2024-03-26T08:29:06.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.132+0000 7fc93aa00700 1 --2- 172.21.15.52:0/2934790984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc93c104410 0x7fc93c19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:29:06.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.133+0000 7fc93b400700 1 --2- 172.21.15.52:0/2934790984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc93c105fb0 0x7fc93c1a1a00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc93400c010 tx=0x7fc9340047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:29:06.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.133+0000 7fc933e00700 1 -- 172.21.15.52:0/2934790984 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc934003dc0 con 0x7fc93c105fb0 2024-03-26T08:29:06.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.133+0000 7fc933e00700 1 -- 172.21.15.52:0/2934790984 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc934004ce0 con 0x7fc93c105fb0 2024-03-26T08:29:06.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.133+0000 7fc941c00700 1 -- 172.21.15.52:0/2934790984 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc93c1a2330 con 0x7fc93c105fb0 2024-03-26T08:29:06.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.133+0000 7fc933e00700 1 -- 172.21.15.52:0/2934790984 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9340055b0 con 0x7fc93c105fb0 2024-03-26T08:29:06.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.133+0000 7fc941c00700 1 -- 172.21.15.52:0/2934790984 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc93c1a2780 con 0x7fc93c105fb0 2024-03-26T08:29:06.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.134+0000 7fc941c00700 1 -- 172.21.15.52:0/2934790984 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc93c068e50 con 0x7fc93c105fb0 2024-03-26T08:29:06.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.137+0000 7fc933e00700 1 -- 172.21.15.52:0/2934790984 <== mon.2 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc934003f60 con 0x7fc93c105fb0 2024-03-26T08:29:06.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.139+0000 7fc933e00700 1 --2- 172.21.15.52:0/2934790984 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fc9200a2910 0x7fc9200a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:29:06.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.139+0000 7fc933e00700 1 -- 172.21.15.52:0/2934790984 <== mon.2 v2:172.21.15.52:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fc9340cf870 con 0x7fc93c105fb0 2024-03-26T08:29:06.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.139+0000 7fc93aa00700 1 --2- 172.21.15.52:0/2934790984 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fc9200a2910 0x7fc9200a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:29:06.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.140+0000 7fc93aa00700 1 --2- 172.21.15.52:0/2934790984 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fc9200a2910 0x7fc9200a4dd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc92c005fd0 tx=0x7fc92c005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:29:06.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.141+0000 7fc933e00700 1 -- 172.21.15.52:0/2934790984 <== mon.2 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc934013070 con 0x7fc93c105fb0 2024-03-26T08:29:06.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:06 smithi052 ceph-mon[35071]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:06.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:06.379+0000 7fc941c00700 1 -- 172.21.15.52:0/2934790984 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc93c062690 con 0x7fc9200a2910 2024-03-26T08:29:06.459 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:06 smithi183 ceph-mon[35818]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:06.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:06 smithi043 ceph-mon[31929]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:07.546 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:07 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:29:07.546 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:07 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:29:07.546 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:07 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:29:07.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:07 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:29:07.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:07 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:29:07.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:07 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:29:07.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:07 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:29:07.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:07 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:29:07.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:07 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:29:08.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:08 smithi183 ceph-mon[35818]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:08.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:08 smithi183 ceph-mon[35818]: from='client.34358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:29:08.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:08 smithi052 ceph-mon[35071]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:08.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:08 smithi052 ceph-mon[35071]: from='client.34358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:29:08.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:08 smithi043 ceph-mon[31929]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:08.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:08 smithi043 ceph-mon[31929]: from='client.34358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:29:09.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:09 smithi052 ceph-mon[35071]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:09.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:09 smithi183 ceph-mon[35818]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:09.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:09 smithi043 ceph-mon[31929]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:11.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:11 smithi052 ceph-mon[35071]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:11.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:11 smithi052 ceph-mon[35071]: from='client.? 172.21.15.52:0/3130587631' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd839188-f398-4cff-aa7b-36522ff1a8a9"}]: dispatch 2024-03-26T08:29:11.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:11 smithi052 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd839188-f398-4cff-aa7b-36522ff1a8a9"}]: dispatch 2024-03-26T08:29:11.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:11 smithi052 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd839188-f398-4cff-aa7b-36522ff1a8a9"}]': finished 2024-03-26T08:29:11.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:11 smithi052 ceph-mon[35071]: osdmap e70: 11 total, 10 up, 11 in 2024-03-26T08:29:11.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:11 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:29:11.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:11 smithi043 ceph-mon[31929]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:11.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:11 smithi043 ceph-mon[31929]: from='client.? 172.21.15.52:0/3130587631' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd839188-f398-4cff-aa7b-36522ff1a8a9"}]: dispatch 2024-03-26T08:29:11.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:11 smithi043 ceph-mon[31929]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd839188-f398-4cff-aa7b-36522ff1a8a9"}]: dispatch 2024-03-26T08:29:11.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:11 smithi043 ceph-mon[31929]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd839188-f398-4cff-aa7b-36522ff1a8a9"}]': finished 2024-03-26T08:29:11.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:11 smithi043 ceph-mon[31929]: osdmap e70: 11 total, 10 up, 11 in 2024-03-26T08:29:11.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:11 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:29:11.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:11 smithi183 ceph-mon[35818]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:11.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:11 smithi183 ceph-mon[35818]: from='client.? 172.21.15.52:0/3130587631' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd839188-f398-4cff-aa7b-36522ff1a8a9"}]: dispatch 2024-03-26T08:29:11.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:11 smithi183 ceph-mon[35818]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd839188-f398-4cff-aa7b-36522ff1a8a9"}]: dispatch 2024-03-26T08:29:11.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:11 smithi183 ceph-mon[35818]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd839188-f398-4cff-aa7b-36522ff1a8a9"}]': finished 2024-03-26T08:29:11.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:11 smithi183 ceph-mon[35818]: osdmap e70: 11 total, 10 up, 11 in 2024-03-26T08:29:11.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:11 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:29:12.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:12 smithi043 ceph-mon[31929]: from='client.? 172.21.15.52:0/2635371388' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:29:12.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:12 smithi183 ceph-mon[35818]: from='client.? 172.21.15.52:0/2635371388' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:29:12.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:12 smithi052 ceph-mon[35071]: from='client.? 172.21.15.52:0/2635371388' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:29:13.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:13 smithi043 ceph-mon[31929]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:13.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:13 smithi183 ceph-mon[35818]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:13.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:13 smithi052 ceph-mon[35071]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:15.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:15 smithi052 ceph-mon[35071]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:15.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:15 smithi043 ceph-mon[31929]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:15.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:15 smithi183 ceph-mon[35818]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:17.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:17 smithi043 ceph-mon[31929]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:17.689 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:17 smithi052 ceph-mon[35071]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:17.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:17 smithi183 ceph-mon[35818]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:19.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:19 smithi052 ceph-mon[35071]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:19.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:19 smithi043 ceph-mon[31929]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:19.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:19 smithi183 ceph-mon[35818]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:20.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:20 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-26T08:29:20.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:20 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:29:20.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:20 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-26T08:29:20.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:20 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:29:20.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:20 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-26T08:29:20.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:20 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:29:21.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:21 smithi052 ceph-mon[35071]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:21.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:21 smithi052 ceph-mon[35071]: Deploying daemon osd.10 on smithi052 2024-03-26T08:29:21.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:21 smithi043 ceph-mon[31929]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:21.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:21 smithi043 ceph-mon[31929]: Deploying daemon osd.10 on smithi052 2024-03-26T08:29:21.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:21 smithi183 ceph-mon[35818]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:21.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:21 smithi183 ceph-mon[35818]: Deploying daemon osd.10 on smithi052 2024-03-26T08:29:23.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:23 smithi043 ceph-mon[31929]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:23.765 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:23 smithi052 ceph-mon[35071]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:23.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:23 smithi183 ceph-mon[35818]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:25.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:25 smithi183 ceph-mon[35818]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:25.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:25 smithi043 ceph-mon[31929]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:25.736 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:25 smithi052 ceph-mon[35071]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:27.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:27 smithi043 ceph-mon[31929]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:27.791 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:27 smithi052 ceph-mon[35071]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:27.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:27 smithi183 ceph-mon[35818]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:30.401 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:30 smithi052 ceph-mon[35071]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:30.402 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:30 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:30.402 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:30 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:30.402 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:30 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:29:30.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:30 smithi043 ceph-mon[31929]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:30.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:30 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:30.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:30 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:30.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:30 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:29:30.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:30 smithi183 ceph-mon[35818]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:30.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:30 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:30.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:30 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:30.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:30 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:29:32.305 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:32 smithi052 ceph-mon[35071]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:32.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:32 smithi043 ceph-mon[31929]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:32.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:32 smithi183 ceph-mon[35818]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:33.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:33 smithi052 ceph-mon[35071]: from='osd.10 [v2:172.21.15.52:6824/377628320,v1:172.21.15.52:6825/377628320]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T08:29:33.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:33 smithi052 ceph-mon[35071]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T08:29:33.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:33 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:33.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:33 smithi043 ceph-mon[31929]: from='osd.10 [v2:172.21.15.52:6824/377628320,v1:172.21.15.52:6825/377628320]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T08:29:33.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:33 smithi043 ceph-mon[31929]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T08:29:33.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:33 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:33.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:33 smithi183 ceph-mon[35818]: from='osd.10 [v2:172.21.15.52:6824/377628320,v1:172.21.15.52:6825/377628320]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T08:29:33.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:33 smithi183 ceph-mon[35818]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T08:29:33.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:33 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:34.505 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:34 smithi052 ceph-mon[35071]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:34.505 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:34 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:34.505 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:34 smithi052 ceph-mon[35071]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-26T08:29:34.505 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:34 smithi052 ceph-mon[35071]: from='osd.10 [v2:172.21.15.52:6824/377628320,v1:172.21.15.52:6825/377628320]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-26T08:29:34.505 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:34 smithi052 ceph-mon[35071]: osdmap e71: 11 total, 10 up, 11 in 2024-03-26T08:29:34.505 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:34 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:29:34.505 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:34 smithi052 ceph-mon[35071]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-26T08:29:34.505 INFO:teuthology.orchestra.run.smithi052.stdout:Created osd(s) 10 on host 'smithi052' 2024-03-26T08:29:34.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:34.510+0000 7fc933e00700 1 -- 172.21.15.52:0/2934790984 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fc93c062690 con 0x7fc9200a2910 2024-03-26T08:29:34.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:34.513+0000 7fc941c00700 1 -- 172.21.15.52:0/2934790984 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fc9200a2910 msgr2=0x7fc9200a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:29:34.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:34.513+0000 7fc941c00700 1 --2- 172.21.15.52:0/2934790984 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fc9200a2910 0x7fc9200a4dd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc92c005fd0 tx=0x7fc92c005dc0 comp rx=0 tx=0).stop 2024-03-26T08:29:34.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:34.513+0000 7fc941c00700 1 -- 172.21.15.52:0/2934790984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc93c105fb0 msgr2=0x7fc93c1a1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:29:34.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:34.513+0000 7fc941c00700 1 --2- 172.21.15.52:0/2934790984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc93c105fb0 0x7fc93c1a1a00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc93400c010 tx=0x7fc9340047b0 comp rx=0 tx=0).stop 2024-03-26T08:29:34.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:34.513+0000 7fc941c00700 1 -- 172.21.15.52:0/2934790984 shutdown_connections 2024-03-26T08:29:34.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:34.513+0000 7fc941c00700 1 --2- 172.21.15.52:0/2934790984 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fc9200a2910 0x7fc9200a4dd0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:29:34.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:34.513+0000 7fc941c00700 1 --2- 172.21.15.52:0/2934790984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc93c105fb0 0x7fc93c1a1a00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:29:34.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:34.513+0000 7fc941c00700 1 --2- 172.21.15.52:0/2934790984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc93c104410 0x7fc93c19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:29:34.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:34.513+0000 7fc941c00700 1 --2- 172.21.15.52:0/2934790984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc93c105610 0x7fc93c19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:29:34.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:34.513+0000 7fc941c00700 1 -- 172.21.15.52:0/2934790984 >> 172.21.15.52:0/2934790984 conn(0x7fc93c0ff9b0 msgr2=0x7fc93c1088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:29:34.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:34.514+0000 7fc941c00700 1 -- 172.21.15.52:0/2934790984 shutdown_connections 2024-03-26T08:29:34.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-26T08:29:34.514+0000 7fc941c00700 1 -- 172.21.15.52:0/2934790984 wait complete. 2024-03-26T08:29:34.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:34 smithi183 ceph-mon[35818]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:34.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:34 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:34.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:34 smithi183 ceph-mon[35818]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-26T08:29:34.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:34 smithi183 ceph-mon[35818]: from='osd.10 [v2:172.21.15.52:6824/377628320,v1:172.21.15.52:6825/377628320]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-26T08:29:34.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:34 smithi183 ceph-mon[35818]: osdmap e71: 11 total, 10 up, 11 in 2024-03-26T08:29:34.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:34 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:29:34.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:34 smithi183 ceph-mon[35818]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-26T08:29:34.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:34 smithi043 ceph-mon[31929]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:34.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:34 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:34.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:34 smithi043 ceph-mon[31929]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-26T08:29:34.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:34 smithi043 ceph-mon[31929]: from='osd.10 [v2:172.21.15.52:6824/377628320,v1:172.21.15.52:6825/377628320]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-26T08:29:34.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:34 smithi043 ceph-mon[31929]: osdmap e71: 11 total, 10 up, 11 in 2024-03-26T08:29:34.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:34 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:29:34.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:34 smithi043 ceph-mon[31929]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-26T08:29:35.317 DEBUG:teuthology.orchestra.run.smithi052:osd.10> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.10.service 2024-03-26T08:29:35.323 INFO:tasks.cephadm:Deploying osd.11 on smithi183 with /dev/vg_nvme/lv_1... 2024-03-26T08:29:35.323 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-26T08:29:35.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:35 smithi183 ceph-mon[35818]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-03-26T08:29:35.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:35 smithi183 ceph-mon[35818]: osdmap e72: 11 total, 10 up, 11 in 2024-03-26T08:29:35.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:35 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:29:35.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:35 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:29:35.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:35 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:35.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:35 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:35.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:35 smithi183 ceph-mon[35818]: from='osd.10 ' entity='osd.10' 2024-03-26T08:29:35.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:35 smithi183 ceph-mon[35818]: osd.10 [v2:172.21.15.52:6824/377628320,v1:172.21.15.52:6825/377628320] boot 2024-03-26T08:29:35.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:35 smithi183 ceph-mon[35818]: osdmap e73: 11 total, 11 up, 11 in 2024-03-26T08:29:35.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:35 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:29:35.551 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:35 smithi052 ceph-mon[35071]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-03-26T08:29:35.552 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:35 smithi052 ceph-mon[35071]: osdmap e72: 11 total, 10 up, 11 in 2024-03-26T08:29:35.552 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:35 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:29:35.552 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:35 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:29:35.552 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:35 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:35.552 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:35 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:35.552 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:35 smithi052 ceph-mon[35071]: from='osd.10 ' entity='osd.10' 2024-03-26T08:29:35.552 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:35 smithi052 ceph-mon[35071]: osd.10 [v2:172.21.15.52:6824/377628320,v1:172.21.15.52:6825/377628320] boot 2024-03-26T08:29:35.552 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:35 smithi052 ceph-mon[35071]: osdmap e73: 11 total, 11 up, 11 in 2024-03-26T08:29:35.552 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:35 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:29:35.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:35 smithi043 ceph-mon[31929]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-03-26T08:29:35.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:35 smithi043 ceph-mon[31929]: osdmap e72: 11 total, 10 up, 11 in 2024-03-26T08:29:35.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:35 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:29:35.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:35 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:29:35.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:35 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:35.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:35 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:35.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:35 smithi043 ceph-mon[31929]: from='osd.10 ' entity='osd.10' 2024-03-26T08:29:35.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:35 smithi043 ceph-mon[31929]: osd.10 [v2:172.21.15.52:6824/377628320,v1:172.21.15.52:6825/377628320] boot 2024-03-26T08:29:35.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:35 smithi043 ceph-mon[31929]: osdmap e73: 11 total, 11 up, 11 in 2024-03-26T08:29:35.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:35 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:29:35.752 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:29:36.437 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:36 smithi052 ceph-mon[35071]: purged_snaps scrub starts 2024-03-26T08:29:36.438 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:36 smithi052 ceph-mon[35071]: purged_snaps scrub ok 2024-03-26T08:29:36.438 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:36 smithi052 ceph-mon[35071]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:36.438 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:36 smithi052 ceph-mon[35071]: osdmap e74: 11 total, 11 up, 11 in 2024-03-26T08:29:36.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:36 smithi183 ceph-mon[35818]: purged_snaps scrub starts 2024-03-26T08:29:36.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:36 smithi183 ceph-mon[35818]: purged_snaps scrub ok 2024-03-26T08:29:36.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:36 smithi183 ceph-mon[35818]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:36.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:36 smithi183 ceph-mon[35818]: osdmap e74: 11 total, 11 up, 11 in 2024-03-26T08:29:36.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:36 smithi043 ceph-mon[31929]: purged_snaps scrub starts 2024-03-26T08:29:36.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:36 smithi043 ceph-mon[31929]: purged_snaps scrub ok 2024-03-26T08:29:36.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:36 smithi043 ceph-mon[31929]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:29:36.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:36 smithi043 ceph-mon[31929]: osdmap e74: 11 total, 11 up, 11 in 2024-03-26T08:29:38.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:38 smithi183 ceph-mon[35818]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:29:38.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:38 smithi183 ceph-mon[35818]: osdmap e75: 11 total, 11 up, 11 in 2024-03-26T08:29:38.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:38 smithi052 ceph-mon[35071]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:29:38.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:38 smithi052 ceph-mon[35071]: osdmap e75: 11 total, 11 up, 11 in 2024-03-26T08:29:38.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:38 smithi043 ceph-mon[31929]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:29:38.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:38 smithi043 ceph-mon[31929]: osdmap e75: 11 total, 11 up, 11 in 2024-03-26T08:29:38.673 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-26T08:29:38.673 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-26T08:29:38.673 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-03-26T08:29:38.673 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-03-26T08:29:38.673 INFO:teuthology.orchestra.run.smithi183.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0335051 s, 313 MB/s 2024-03-26T08:29:38.673 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-03-26T08:29:39.297 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:39 smithi052 ceph-mon[35071]: osdmap e76: 11 total, 11 up, 11 in 2024-03-26T08:29:39.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:39 smithi183 ceph-mon[35818]: osdmap e76: 11 total, 11 up, 11 in 2024-03-26T08:29:39.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:39 smithi043 ceph-mon[31929]: osdmap e76: 11 total, 11 up, 11 in 2024-03-26T08:29:39.611 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi183:vg_nvme/lv_1 2024-03-26T08:29:40.048 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:29:40.259 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:40 smithi052 ceph-mon[35071]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:29:40.259 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:40 smithi052 ceph-mon[35071]: osdmap e77: 11 total, 11 up, 11 in 2024-03-26T08:29:40.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:40 smithi183 ceph-mon[35818]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:29:40.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:40 smithi183 ceph-mon[35818]: osdmap e77: 11 total, 11 up, 11 in 2024-03-26T08:29:40.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:40 smithi043 ceph-mon[31929]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:29:40.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:40 smithi043 ceph-mon[31929]: osdmap e77: 11 total, 11 up, 11 in 2024-03-26T08:29:41.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:41 smithi043 ceph-mon[31929]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-03-26T08:29:41.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:41 smithi043 ceph-mon[31929]: Detected new or changed devices on smithi052 2024-03-26T08:29:41.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:41.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:41.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:29:41.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:29:41.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:29:41.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:29:41.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:41 smithi043 ceph-mon[31929]: Adjusting osd_memory_target on smithi052 to 4273M 2024-03-26T08:29:41.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:41.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:29:41.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:29:41.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:41.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:29:41.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:29:41.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:29:41.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:41.690 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:41 smithi183 ceph-mon[35818]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-03-26T08:29:41.690 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:41 smithi183 ceph-mon[35818]: Detected new or changed devices on smithi052 2024-03-26T08:29:41.690 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:41.690 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:41.691 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:29:41.691 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:29:41.691 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:29:41.691 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:29:41.691 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:41 smithi183 ceph-mon[35818]: Adjusting osd_memory_target on smithi052 to 4273M 2024-03-26T08:29:41.691 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:41.691 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:29:41.691 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:29:41.691 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:41.691 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:29:41.691 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:29:41.691 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:29:41.691 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:41.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:41 smithi052 ceph-mon[35071]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-03-26T08:29:41.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:41 smithi052 ceph-mon[35071]: Detected new or changed devices on smithi052 2024-03-26T08:29:41.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:41.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:41.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:29:41.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:29:41.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:29:41.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:29:41.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:41 smithi052 ceph-mon[35071]: Adjusting osd_memory_target on smithi052 to 4273M 2024-03-26T08:29:41.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:41.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:29:41.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:29:41.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:41.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:29:41.825 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:29:41.825 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:29:41.825 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:29:42.188 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.194+0000 7f71fe200700 1 -- 172.21.15.183:0/3285386373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f8074b90 msgr2=0x7f71f8075010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:29:42.188 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.194+0000 7f71fe200700 1 --2- 172.21.15.183:0/3285386373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f8074b90 0x7f71f8075010 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f71e4009990 tx=0x7f71e4009ca0 comp rx=0 tx=0).stop 2024-03-26T08:29:42.188 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.194+0000 7f71fe200700 1 -- 172.21.15.183:0/3285386373 shutdown_connections 2024-03-26T08:29:42.189 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.194+0000 7f71fe200700 1 --2- 172.21.15.183:0/3285386373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71f810a0a0 0x7f71f810c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:29:42.189 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.194+0000 7f71fe200700 1 --2- 172.21.15.183:0/3285386373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71f8076160 0x7f71f80745c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:29:42.189 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.194+0000 7f71fe200700 1 --2- 172.21.15.183:0/3285386373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f8074b90 0x7f71f8075010 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:29:42.189 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.194+0000 7f71fe200700 1 -- 172.21.15.183:0/3285386373 >> 172.21.15.183:0/3285386373 conn(0x7f71f80fd7a0 msgr2=0x7f71f80ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:29:42.189 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.194+0000 7f71fe200700 1 -- 172.21.15.183:0/3285386373 shutdown_connections 2024-03-26T08:29:42.189 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.194+0000 7f71fe200700 1 -- 172.21.15.183:0/3285386373 wait complete. 2024-03-26T08:29:42.191 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.195+0000 7f71fe200700 1 Processor -- start 2024-03-26T08:29:42.192 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.195+0000 7f71fe200700 1 -- start start 2024-03-26T08:29:42.192 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.196+0000 7f71fe200700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71f8074b90 0x7f71f819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:29:42.192 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.196+0000 7f71fe200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71f8076160 0x7f71f819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:29:42.192 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.196+0000 7f71fe200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f810a0a0 0x7f71f81a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:29:42.192 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.196+0000 7f71fe200700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f810ff60 con 0x7f71f8076160 2024-03-26T08:29:42.192 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.196+0000 7f71fe200700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f810fde0 con 0x7f71f8074b90 2024-03-26T08:29:42.192 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.196+0000 7f71fe200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f81100e0 con 0x7f71f810a0a0 2024-03-26T08:29:42.192 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.196+0000 7f71f7e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f810a0a0 0x7f71f81a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:29:42.192 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.196+0000 7f71f7e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f810a0a0 0x7f71f81a1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:36458/0 (socket says 172.21.15.183:36458) 2024-03-26T08:29:42.192 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.196+0000 7f71f7e00700 1 -- 172.21.15.183:0/3471620699 learned_addr learned my addr 172.21.15.183:0/3471620699 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-03-26T08:29:42.192 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.196+0000 7f71f6a00700 1 --2- 172.21.15.183:0/3471620699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71f8076160 0x7f71f819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:29:42.193 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.196+0000 7f71f7400700 1 --2- 172.21.15.183:0/3471620699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71f8074b90 0x7f71f819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:29:42.193 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.196+0000 7f71f7e00700 1 -- 172.21.15.183:0/3471620699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71f8074b90 msgr2=0x7f71f819a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:29:42.193 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.196+0000 7f71f7e00700 1 --2- 172.21.15.183:0/3471620699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71f8074b90 0x7f71f819a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:29:42.193 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.196+0000 7f71f7e00700 1 -- 172.21.15.183:0/3471620699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71f8076160 msgr2=0x7f71f819aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:29:42.193 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.196+0000 7f71f7e00700 1 --2- 172.21.15.183:0/3471620699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71f8076160 0x7f71f819aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:29:42.193 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.196+0000 7f71f7e00700 1 -- 172.21.15.183:0/3471620699 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f81a2270 con 0x7f71f810a0a0 2024-03-26T08:29:42.193 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.197+0000 7f71f7e00700 1 --2- 172.21.15.183:0/3471620699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f810a0a0 0x7f71f81a1bd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f71f000b9a0 tx=0x7f71f000bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:29:42.193 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.197+0000 7f71efe00700 1 -- 172.21.15.183:0/3471620699 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0019070 con 0x7f71f810a0a0 2024-03-26T08:29:42.193 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.197+0000 7f71efe00700 1 -- 172.21.15.183:0/3471620699 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f71f0014440 con 0x7f71f810a0a0 2024-03-26T08:29:42.193 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.197+0000 7f71efe00700 1 -- 172.21.15.183:0/3471620699 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f000ce60 con 0x7f71f810a0a0 2024-03-26T08:29:42.193 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.197+0000 7f71fe200700 1 -- 172.21.15.183:0/3471620699 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f81a2560 con 0x7f71f810a0a0 2024-03-26T08:29:42.193 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.197+0000 7f71fe200700 1 -- 172.21.15.183:0/3471620699 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f81a2ad0 con 0x7f71f810a0a0 2024-03-26T08:29:42.194 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.198+0000 7f71fe200700 1 -- 172.21.15.183:0/3471620699 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f8068e50 con 0x7f71f810a0a0 2024-03-26T08:29:42.194 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.200+0000 7f71efe00700 1 -- 172.21.15.183:0/3471620699 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f71f001c070 con 0x7f71f810a0a0 2024-03-26T08:29:42.195 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.202+0000 7f71efe00700 1 --2- 172.21.15.183:0/3471620699 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f71dc0a2890 0x7f71dc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:29:42.195 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.202+0000 7f71efe00700 1 -- 172.21.15.183:0/3471620699 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f71f00cd890 con 0x7f71f810a0a0 2024-03-26T08:29:42.196 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.202+0000 7f71f7400700 1 --2- 172.21.15.183:0/3471620699 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f71dc0a2890 0x7f71dc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:29:42.197 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.203+0000 7f71f7400700 1 --2- 172.21.15.183:0/3471620699 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f71dc0a2890 0x7f71dc0a4d50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f71e8006fd0 tx=0x7f71e8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:29:42.199 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.206+0000 7f71efe00700 1 -- 172.21.15.183:0/3471620699 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f71f009af70 con 0x7f71f810a0a0 2024-03-26T08:29:42.431 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:29:42.437+0000 7f71fe200700 1 -- 172.21.15.183:0/3471620699 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f71f80625d0 con 0x7f71dc0a2890 2024-03-26T08:29:43.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:43 smithi043 ceph-mon[31929]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T08:29:43.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:43 smithi043 ceph-mon[31929]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:29:43.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:43 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:29:43.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:43 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:29:43.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:43 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:29:43.677 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:43 smithi183 ceph-mon[35818]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T08:29:43.677 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:43 smithi183 ceph-mon[35818]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:29:43.677 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:43 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:29:43.677 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:43 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:29:43.678 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:43 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:29:43.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:43 smithi052 ceph-mon[35071]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T08:29:43.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:43 smithi052 ceph-mon[35071]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:29:43.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:43 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:29:43.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:43 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:29:43.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:43 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:29:45.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:45 smithi183 ceph-mon[35818]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-26T08:29:45.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:45 smithi043 ceph-mon[31929]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-26T08:29:45.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:45 smithi052 ceph-mon[35071]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-26T08:29:47.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:47 smithi043 ceph-mon[31929]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-26T08:29:47.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:47 smithi043 ceph-mon[31929]: from='client.? 172.21.15.183:0/1217603279' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a40282a0-0c64-49ee-b64e-6f3f8119e827"}]: dispatch 2024-03-26T08:29:47.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:47 smithi043 ceph-mon[31929]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a40282a0-0c64-49ee-b64e-6f3f8119e827"}]: dispatch 2024-03-26T08:29:47.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:47 smithi043 ceph-mon[31929]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a40282a0-0c64-49ee-b64e-6f3f8119e827"}]': finished 2024-03-26T08:29:47.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:47 smithi043 ceph-mon[31929]: osdmap e78: 12 total, 11 up, 12 in 2024-03-26T08:29:47.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:47 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:29:47.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:47 smithi043 ceph-mon[31929]: from='client.? 172.21.15.183:0/3343410977' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:29:47.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:47 smithi183 ceph-mon[35818]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-26T08:29:47.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:47 smithi183 ceph-mon[35818]: from='client.? 172.21.15.183:0/1217603279' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a40282a0-0c64-49ee-b64e-6f3f8119e827"}]: dispatch 2024-03-26T08:29:47.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:47 smithi183 ceph-mon[35818]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a40282a0-0c64-49ee-b64e-6f3f8119e827"}]: dispatch 2024-03-26T08:29:47.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:47 smithi183 ceph-mon[35818]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a40282a0-0c64-49ee-b64e-6f3f8119e827"}]': finished 2024-03-26T08:29:47.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:47 smithi183 ceph-mon[35818]: osdmap e78: 12 total, 11 up, 12 in 2024-03-26T08:29:47.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:47 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:29:47.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:47 smithi183 ceph-mon[35818]: from='client.? 172.21.15.183:0/3343410977' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:29:47.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:47 smithi052 ceph-mon[35071]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-26T08:29:47.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:47 smithi052 ceph-mon[35071]: from='client.? 172.21.15.183:0/1217603279' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a40282a0-0c64-49ee-b64e-6f3f8119e827"}]: dispatch 2024-03-26T08:29:47.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:47 smithi052 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a40282a0-0c64-49ee-b64e-6f3f8119e827"}]: dispatch 2024-03-26T08:29:47.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:47 smithi052 ceph-mon[35071]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a40282a0-0c64-49ee-b64e-6f3f8119e827"}]': finished 2024-03-26T08:29:47.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:47 smithi052 ceph-mon[35071]: osdmap e78: 12 total, 11 up, 12 in 2024-03-26T08:29:47.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:47 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:29:47.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:47 smithi052 ceph-mon[35071]: from='client.? 172.21.15.183:0/3343410977' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:29:49.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:49 smithi043 ceph-mon[31929]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-26T08:29:49.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:49 smithi183 ceph-mon[35818]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-26T08:29:49.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:49 smithi052 ceph-mon[35071]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-26T08:29:51.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:51 smithi043 ceph-mon[31929]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:29:51.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:51 smithi183 ceph-mon[35818]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:29:51.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:51 smithi052 ceph-mon[35071]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:29:53.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:53 smithi043 ceph-mon[31929]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:29:53.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:53 smithi183 ceph-mon[35818]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:29:53.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:53 smithi052 ceph-mon[35071]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:29:55.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:55 smithi043 ceph-mon[31929]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:29:55.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:55 smithi183 ceph-mon[35818]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:29:55.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:55 smithi052 ceph-mon[35071]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:29:56.697 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:56 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-26T08:29:56.697 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:56 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:29:56.698 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:56 smithi183 ceph-mon[35818]: Deploying daemon osd.11 on smithi183 2024-03-26T08:29:56.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:56 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-26T08:29:56.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:56 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:29:56.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:56 smithi052 ceph-mon[35071]: Deploying daemon osd.11 on smithi183 2024-03-26T08:29:56.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:56 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-26T08:29:56.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:56 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:29:56.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:56 smithi043 ceph-mon[31929]: Deploying daemon osd.11 on smithi183 2024-03-26T08:29:57.721 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:57 smithi183 ceph-mon[35818]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:29:57.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:57 smithi052 ceph-mon[35071]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:29:57.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:57 smithi043 ceph-mon[31929]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:29:59.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:29:59 smithi183 ceph-mon[35818]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:29:59.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:29:59 smithi052 ceph-mon[35071]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:29:59.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:29:59 smithi043 ceph-mon[31929]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:30:00.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:00 smithi183 ceph-mon[35818]: overall HEALTH_OK 2024-03-26T08:30:00.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:00 smithi052 ceph-mon[35071]: overall HEALTH_OK 2024-03-26T08:30:00.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:00 smithi043 ceph-mon[31929]: overall HEALTH_OK 2024-03-26T08:30:01.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:01 smithi183 ceph-mon[35818]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:30:01.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:01 smithi052 ceph-mon[35071]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:30:01.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:01 smithi043 ceph-mon[31929]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:30:03.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:03 smithi183 ceph-mon[35818]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:30:03.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:03 smithi052 ceph-mon[35071]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:30:03.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:03 smithi043 ceph-mon[31929]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:30:05.014 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:04 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:05.015 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:04 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:30:05.015 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:04 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:05.037 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:04 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:05.038 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:04 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:30:05.038 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:04 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:05.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:04 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:05.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:04 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:30:05.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:04 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:06.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:05 smithi183 ceph-mon[35818]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:30:06.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:05 smithi043 ceph-mon[31929]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:30:06.248 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:05 smithi052 ceph-mon[35071]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:30:08.010 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:07 smithi183 ceph-mon[35818]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:30:08.010 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:07 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:08.010 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:07 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:08.010 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:07 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:08.010 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:07 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:30:08.010 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:07 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:08.010 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:07 smithi183 ceph-mon[35818]: from='osd.11 [v2:172.21.15.183:6824/1373086433,v1:172.21.15.183:6825/1373086433]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T08:30:08.010 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:07 smithi183 ceph-mon[35818]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T08:30:08.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:07 smithi043 ceph-mon[31929]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:30:08.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:07 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:08.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:07 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:08.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:07 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:08.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:07 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:30:08.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:07 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:08.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:07 smithi043 ceph-mon[31929]: from='osd.11 [v2:172.21.15.183:6824/1373086433,v1:172.21.15.183:6825/1373086433]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T08:30:08.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:07 smithi043 ceph-mon[31929]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T08:30:08.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:07 smithi052 ceph-mon[35071]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:30:08.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:07 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:08.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:07 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:08.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:07 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:08.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:07 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:30:08.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:07 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:08.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:07 smithi052 ceph-mon[35071]: from='osd.11 [v2:172.21.15.183:6824/1373086433,v1:172.21.15.183:6825/1373086433]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T08:30:08.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:07 smithi052 ceph-mon[35071]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T08:30:08.441 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:08.447+0000 7f71efe00700 1 -- 172.21.15.183:0/3471620699 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f71f80625d0 con 0x7f71dc0a2890 2024-03-26T08:30:08.442 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 11 on host 'smithi183' 2024-03-26T08:30:08.445 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:08.451+0000 7f71fe200700 1 -- 172.21.15.183:0/3471620699 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f71dc0a2890 msgr2=0x7f71dc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:08.445 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:08.451+0000 7f71fe200700 1 --2- 172.21.15.183:0/3471620699 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f71dc0a2890 0x7f71dc0a4d50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f71e8006fd0 tx=0x7f71e8008040 comp rx=0 tx=0).stop 2024-03-26T08:30:08.446 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:08.451+0000 7f71fe200700 1 -- 172.21.15.183:0/3471620699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f810a0a0 msgr2=0x7f71f81a1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:08.446 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:08.451+0000 7f71fe200700 1 --2- 172.21.15.183:0/3471620699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f810a0a0 0x7f71f81a1bd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f71f000b9a0 tx=0x7f71f000bd60 comp rx=0 tx=0).stop 2024-03-26T08:30:08.446 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:08.451+0000 7f71fe200700 1 -- 172.21.15.183:0/3471620699 shutdown_connections 2024-03-26T08:30:08.446 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:08.451+0000 7f71fe200700 1 --2- 172.21.15.183:0/3471620699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f810a0a0 0x7f71f81a1bd0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:08.446 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:08.451+0000 7f71fe200700 1 --2- 172.21.15.183:0/3471620699 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f71dc0a2890 0x7f71dc0a4d50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:08.446 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:08.451+0000 7f71fe200700 1 --2- 172.21.15.183:0/3471620699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71f8074b90 0x7f71f819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:08.446 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:08.451+0000 7f71fe200700 1 --2- 172.21.15.183:0/3471620699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71f8076160 0x7f71f819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:08.446 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:08.451+0000 7f71fe200700 1 -- 172.21.15.183:0/3471620699 >> 172.21.15.183:0/3471620699 conn(0x7f71f80fd7a0 msgr2=0x7f71f8105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:08.446 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:08.451+0000 7f71fe200700 1 -- 172.21.15.183:0/3471620699 shutdown_connections 2024-03-26T08:30:08.446 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:08.452+0000 7f71fe200700 1 -- 172.21.15.183:0/3471620699 wait complete. 2024-03-26T08:30:09.129 DEBUG:teuthology.orchestra.run.smithi183:osd.11> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.11.service 2024-03-26T08:30:09.134 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-26T08:30:09.134 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph osd stat -f json 2024-03-26T08:30:09.601 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:30:09.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:09 smithi043 ceph-mon[31929]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:30:09.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:09 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:09.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:09 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:09.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:09 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:30:09.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:09 smithi043 ceph-mon[31929]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-26T08:30:09.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:09 smithi043 ceph-mon[31929]: from='osd.11 [v2:172.21.15.183:6824/1373086433,v1:172.21.15.183:6825/1373086433]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-26T08:30:09.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:09 smithi043 ceph-mon[31929]: osdmap e79: 12 total, 11 up, 12 in 2024-03-26T08:30:09.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:09 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:30:09.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:09 smithi043 ceph-mon[31929]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-26T08:30:09.680 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:09 smithi183 ceph-mon[35818]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:30:09.681 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:09 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:09.681 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:09 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:09.681 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:09 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:30:09.681 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:09 smithi183 ceph-mon[35818]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-26T08:30:09.681 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:09 smithi183 ceph-mon[35818]: from='osd.11 [v2:172.21.15.183:6824/1373086433,v1:172.21.15.183:6825/1373086433]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-26T08:30:09.681 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:09 smithi183 ceph-mon[35818]: osdmap e79: 12 total, 11 up, 12 in 2024-03-26T08:30:09.681 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:09 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:30:09.681 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:09 smithi183 ceph-mon[35818]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-26T08:30:09.681 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:30:09 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:30:09.543+0000 7f7d34000700 -1 osd.11 0 waiting for initial osdmap 2024-03-26T08:30:09.682 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:30:09 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:30:09.572+0000 7f7d2ee00700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T08:30:09.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:09 smithi052 ceph-mon[35071]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:30:09.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:09 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:09.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:09 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:09.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:09 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:30:09.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:09 smithi052 ceph-mon[35071]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-26T08:30:09.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:09 smithi052 ceph-mon[35071]: from='osd.11 [v2:172.21.15.183:6824/1373086433,v1:172.21.15.183:6825/1373086433]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-26T08:30:09.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:09 smithi052 ceph-mon[35071]: osdmap e79: 12 total, 11 up, 12 in 2024-03-26T08:30:09.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:09 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:30:09.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:09 smithi052 ceph-mon[35071]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-03-26T08:30:10.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:10 smithi043 ceph-mon[31929]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-03-26T08:30:10.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:10 smithi043 ceph-mon[31929]: osdmap e80: 12 total, 11 up, 12 in 2024-03-26T08:30:10.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:10 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:30:10.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:10 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:30:10.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:10 smithi183 ceph-mon[35818]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-03-26T08:30:10.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:10 smithi183 ceph-mon[35818]: osdmap e80: 12 total, 11 up, 12 in 2024-03-26T08:30:10.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:10 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:30:10.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:10 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:30:10.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:10 smithi052 ceph-mon[35071]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-03-26T08:30:10.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:10 smithi052 ceph-mon[35071]: osdmap e80: 12 total, 11 up, 12 in 2024-03-26T08:30:10.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:10 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:30:10.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:10 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:30:11.776 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:11 smithi043 ceph-mon[31929]: purged_snaps scrub starts 2024-03-26T08:30:11.776 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:11 smithi043 ceph-mon[31929]: purged_snaps scrub ok 2024-03-26T08:30:11.777 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:11 smithi043 ceph-mon[31929]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:30:11.777 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:11 smithi043 ceph-mon[31929]: osd.11 [v2:172.21.15.183:6824/1373086433,v1:172.21.15.183:6825/1373086433] boot 2024-03-26T08:30:11.777 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:11 smithi043 ceph-mon[31929]: osdmap e81: 12 total, 12 up, 12 in 2024-03-26T08:30:11.777 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:11 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:30:11.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:11 smithi183 ceph-mon[35818]: purged_snaps scrub starts 2024-03-26T08:30:11.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:11 smithi183 ceph-mon[35818]: purged_snaps scrub ok 2024-03-26T08:30:11.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:11 smithi183 ceph-mon[35818]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:30:11.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:11 smithi183 ceph-mon[35818]: osd.11 [v2:172.21.15.183:6824/1373086433,v1:172.21.15.183:6825/1373086433] boot 2024-03-26T08:30:11.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:11 smithi183 ceph-mon[35818]: osdmap e81: 12 total, 12 up, 12 in 2024-03-26T08:30:11.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:11 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:30:11.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:11 smithi052 ceph-mon[35071]: purged_snaps scrub starts 2024-03-26T08:30:11.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:11 smithi052 ceph-mon[35071]: purged_snaps scrub ok 2024-03-26T08:30:11.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:11 smithi052 ceph-mon[35071]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:30:11.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:11 smithi052 ceph-mon[35071]: osd.11 [v2:172.21.15.183:6824/1373086433,v1:172.21.15.183:6825/1373086433] boot 2024-03-26T08:30:11.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:11 smithi052 ceph-mon[35071]: osdmap e81: 12 total, 12 up, 12 in 2024-03-26T08:30:11.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:11 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:30:12.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.242+0000 7f089fa00700 1 -- 172.21.15.43:0/1504026772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0898109a10 msgr2=0x7f0898109e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:12.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.242+0000 7f089fa00700 1 --2- 172.21.15.43:0/1504026772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0898109a10 0x7f0898109e70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f08800099c0 tx=0x7f0880009cd0 comp rx=0 tx=0).stop 2024-03-26T08:30:12.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.242+0000 7f089fa00700 1 -- 172.21.15.43:0/1504026772 shutdown_connections 2024-03-26T08:30:12.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.242+0000 7f089fa00700 1 --2- 172.21.15.43:0/1504026772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f089810a3b0 0x7f0898110c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:12.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.242+0000 7f089fa00700 1 --2- 172.21.15.43:0/1504026772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0898108810 0x7f0898108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:12.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.242+0000 7f089fa00700 1 --2- 172.21.15.43:0/1504026772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0898109a10 0x7f0898109e70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:12.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.242+0000 7f089fa00700 1 -- 172.21.15.43:0/1504026772 >> 172.21.15.43:0/1504026772 conn(0x7f08980777e0 msgr2=0x7f0898079c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:12.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.242+0000 7f089fa00700 1 -- 172.21.15.43:0/1504026772 shutdown_connections 2024-03-26T08:30:12.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.242+0000 7f089fa00700 1 -- 172.21.15.43:0/1504026772 wait complete. 2024-03-26T08:30:12.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.244+0000 7f089fa00700 1 Processor -- start 2024-03-26T08:30:12.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.244+0000 7f089fa00700 1 -- start start 2024-03-26T08:30:12.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.244+0000 7f089fa00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0898108810 0x7f089819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:12.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.244+0000 7f089fa00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0898109a10 0x7f089819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:12.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.244+0000 7f089fa00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f089810a3b0 0x7f08981a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:12.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.244+0000 7f089fa00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0898114350 con 0x7f089810a3b0 2024-03-26T08:30:12.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.244+0000 7f089fa00700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08981141d0 con 0x7f0898108810 2024-03-26T08:30:12.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.244+0000 7f089fa00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08981144d0 con 0x7f0898109a10 2024-03-26T08:30:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.245+0000 7f089d000700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0898108810 0x7f089819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.245+0000 7f089da00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f089810a3b0 0x7f08981a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.245+0000 7f088fe00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0898109a10 0x7f089819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.245+0000 7f089d000700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0898108810 0x7f089819e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.43:53996/0 (socket says 172.21.15.43:53996) 2024-03-26T08:30:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.245+0000 7f089d000700 1 -- 172.21.15.43:0/2819508476 learned_addr learned my addr 172.21.15.43:0/2819508476 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:30:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.245+0000 7f089da00700 1 -- 172.21.15.43:0/2819508476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0898109a10 msgr2=0x7f089819ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.246+0000 7f089da00700 1 --2- 172.21.15.43:0/2819508476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0898109a10 0x7f089819ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.246+0000 7f089da00700 1 -- 172.21.15.43:0/2819508476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0898108810 msgr2=0x7f089819e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.246+0000 7f089da00700 1 --2- 172.21.15.43:0/2819508476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0898108810 0x7f089819e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.246+0000 7f089da00700 1 -- 172.21.15.43:0/2819508476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08981a6530 con 0x7f089810a3b0 2024-03-26T08:30:12.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.246+0000 7f089da00700 1 --2- 172.21.15.43:0/2819508476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f089810a3b0 0x7f08981a5e90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f089000be50 tx=0x7f0890009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:12.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.246+0000 7f0896000700 1 -- 172.21.15.43:0/2819508476 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0890019070 con 0x7f089810a3b0 2024-03-26T08:30:12.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.246+0000 7f0896000700 1 -- 172.21.15.43:0/2819508476 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f089000cb90 con 0x7f089810a3b0 2024-03-26T08:30:12.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.246+0000 7f0896000700 1 -- 172.21.15.43:0/2819508476 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08900125a0 con 0x7f089810a3b0 2024-03-26T08:30:12.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.246+0000 7f089fa00700 1 -- 172.21.15.43:0/2819508476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08981a6820 con 0x7f089810a3b0 2024-03-26T08:30:12.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.247+0000 7f089fa00700 1 -- 172.21.15.43:0/2819508476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08981a6d90 con 0x7f089810a3b0 2024-03-26T08:30:12.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.248+0000 7f089fa00700 1 -- 172.21.15.43:0/2819508476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0898068e50 con 0x7f089810a3b0 2024-03-26T08:30:12.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.250+0000 7f0896000700 1 -- 172.21.15.43:0/2819508476 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f089001c070 con 0x7f089810a3b0 2024-03-26T08:30:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.251+0000 7f0896000700 1 --2- 172.21.15.43:0/2819508476 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f087c0a2890 0x7f087c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.251+0000 7f0896000700 1 -- 172.21.15.43:0/2819508476 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f08900cdd80 con 0x7f089810a3b0 2024-03-26T08:30:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.251+0000 7f089d000700 1 --2- 172.21.15.43:0/2819508476 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f087c0a2890 0x7f087c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:12.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.252+0000 7f089d000700 1 --2- 172.21.15.43:0/2819508476 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f087c0a2890 0x7f087c0a4d50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f088800ace0 tx=0x7f088800a430 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:12.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.256+0000 7f0896000700 1 -- 172.21.15.43:0/2819508476 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f089009b220 con 0x7f089810a3b0 2024-03-26T08:30:12.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.492+0000 7f089fa00700 1 -- 172.21.15.43:0/2819508476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f0898062da0 con 0x7f089810a3b0 2024-03-26T08:30:12.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.492+0000 7f0896000700 1 -- 172.21.15.43:0/2819508476 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) v1 ==== 74+0+133 (secure 0 0 0) 0x7f089009ab20 con 0x7f089810a3b0 2024-03-26T08:30:12.486 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:30:12.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.495+0000 7f089fa00700 1 -- 172.21.15.43:0/2819508476 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f087c0a2890 msgr2=0x7f087c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:12.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.495+0000 7f089fa00700 1 --2- 172.21.15.43:0/2819508476 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f087c0a2890 0x7f087c0a4d50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f088800ace0 tx=0x7f088800a430 comp rx=0 tx=0).stop 2024-03-26T08:30:12.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.495+0000 7f089fa00700 1 -- 172.21.15.43:0/2819508476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f089810a3b0 msgr2=0x7f08981a5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:12.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.495+0000 7f089fa00700 1 --2- 172.21.15.43:0/2819508476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f089810a3b0 0x7f08981a5e90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f089000be50 tx=0x7f0890009ed0 comp rx=0 tx=0).stop 2024-03-26T08:30:12.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.495+0000 7f089fa00700 1 -- 172.21.15.43:0/2819508476 shutdown_connections 2024-03-26T08:30:12.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.495+0000 7f089fa00700 1 --2- 172.21.15.43:0/2819508476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f089810a3b0 0x7f08981a5e90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:12.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.495+0000 7f089fa00700 1 --2- 172.21.15.43:0/2819508476 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f087c0a2890 0x7f087c0a4d50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:12.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.495+0000 7f089fa00700 1 --2- 172.21.15.43:0/2819508476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0898108810 0x7f089819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:12.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.495+0000 7f089fa00700 1 --2- 172.21.15.43:0/2819508476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0898109a10 0x7f089819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:12.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.495+0000 7f089fa00700 1 -- 172.21.15.43:0/2819508476 >> 172.21.15.43:0/2819508476 conn(0x7f08980777e0 msgr2=0x7f089810ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:12.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.496+0000 7f089fa00700 1 -- 172.21.15.43:0/2819508476 shutdown_connections 2024-03-26T08:30:12.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:12.496+0000 7f089fa00700 1 -- 172.21.15.43:0/2819508476 wait complete. 2024-03-26T08:30:12.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:12 smithi052 ceph-mon[35071]: osdmap e82: 12 total, 12 up, 12 in 2024-03-26T08:30:12.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:12 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2819508476' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T08:30:12.831 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:12 smithi183 ceph-mon[35818]: osdmap e82: 12 total, 12 up, 12 in 2024-03-26T08:30:12.831 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:12 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2819508476' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T08:30:12.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:12 smithi043 ceph-mon[31929]: osdmap e82: 12 total, 12 up, 12 in 2024-03-26T08:30:12.898 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:12 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2819508476' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T08:30:13.419 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1711441810,"num_in_osds":12,"osd_in_since":1711441786,"num_remapped_pgs":0} 2024-03-26T08:30:13.419 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-26T08:30:13.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:13 smithi043 ceph-mon[31929]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:13.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:13 smithi183 ceph-mon[35818]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:13.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:13 smithi052 ceph-mon[35071]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:13.888 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:30:14.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:14 smithi043 ceph-mon[31929]: Detected new or changed devices on smithi183 2024-03-26T08:30:14.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:14 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:14.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:14 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:14.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:14 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:30:14.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:14 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:30:14.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:14 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:30:14.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:14 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:30:14.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:14 smithi043 ceph-mon[31929]: Adjusting osd_memory_target on smithi183 to 4272M 2024-03-26T08:30:14.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:14 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:14.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:14 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:14.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:14 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:30:14.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:14 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:15.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:14 smithi183 ceph-mon[35818]: Detected new or changed devices on smithi183 2024-03-26T08:30:15.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:14 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:15.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:14 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:15.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:14 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:30:15.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:14 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:30:15.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:14 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:30:15.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:14 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:30:15.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:14 smithi183 ceph-mon[35818]: Adjusting osd_memory_target on smithi183 to 4272M 2024-03-26T08:30:15.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:14 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:15.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:14 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:15.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:14 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:30:15.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:14 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:15.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:14 smithi052 ceph-mon[35071]: Detected new or changed devices on smithi183 2024-03-26T08:30:15.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:14 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:15.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:14 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:15.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:14 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:30:15.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:14 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:30:15.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:14 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:30:15.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:14 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:30:15.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:14 smithi052 ceph-mon[35071]: Adjusting osd_memory_target on smithi183 to 4272M 2024-03-26T08:30:15.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:14 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:15.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:14 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:15.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:14 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:30:15.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:14 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:15.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:15 smithi043 ceph-mon[31929]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:16.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:15 smithi183 ceph-mon[35818]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:16.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:15 smithi052 ceph-mon[35071]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:16.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.258+0000 7f5a19600700 1 -- 172.21.15.43:0/874276051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a1410eb90 msgr2=0x7f5a14110f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:16.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.258+0000 7f5a19600700 1 --2- 172.21.15.43:0/874276051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a1410eb90 0x7f5a14110f80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00b0a0 tx=0x7f5a0c00b3b0 comp rx=0 tx=0).stop 2024-03-26T08:30:16.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.258+0000 7f5a19600700 1 -- 172.21.15.43:0/874276051 shutdown_connections 2024-03-26T08:30:16.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.258+0000 7f5a19600700 1 --2- 172.21.15.43:0/874276051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a1410eb90 0x7f5a14110f80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:16.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.258+0000 7f5a19600700 1 --2- 172.21.15.43:0/874276051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a140743c0 0x7f5a140747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:16.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.258+0000 7f5a19600700 1 --2- 172.21.15.43:0/874276051 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a14074ce0 0x7f5a1410e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:16.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.258+0000 7f5a19600700 1 -- 172.21.15.43:0/874276051 >> 172.21.15.43:0/874276051 conn(0x7f5a140792f0 msgr2=0x7f5a14079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:16.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.259+0000 7f5a19600700 1 -- 172.21.15.43:0/874276051 shutdown_connections 2024-03-26T08:30:16.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.259+0000 7f5a19600700 1 -- 172.21.15.43:0/874276051 wait complete. 2024-03-26T08:30:16.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.260+0000 7f5a19600700 1 Processor -- start 2024-03-26T08:30:16.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.260+0000 7f5a19600700 1 -- start start 2024-03-26T08:30:16.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.260+0000 7f5a19600700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a140743c0 0x7f5a141a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:16.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.260+0000 7f5a19600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a14074ce0 0x7f5a141a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:16.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.260+0000 7f5a19600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a1410eb90 0x7f5a1419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:16.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.261+0000 7f5a19600700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a141145d0 con 0x7f5a14074ce0 2024-03-26T08:30:16.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.261+0000 7f5a19600700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a14114450 con 0x7f5a140743c0 2024-03-26T08:30:16.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.261+0000 7f5a19600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a14114750 con 0x7f5a1410eb90 2024-03-26T08:30:16.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.261+0000 7f5a12000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a14074ce0 0x7f5a141a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:16.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.261+0000 7f5a12000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a14074ce0 0x7f5a141a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52602/0 (socket says 172.21.15.43:52602) 2024-03-26T08:30:16.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.261+0000 7f5a12000700 1 -- 172.21.15.43:0/3633640386 learned_addr learned my addr 172.21.15.43:0/3633640386 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:30:16.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.261+0000 7f5a12a00700 1 --2- 172.21.15.43:0/3633640386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a140743c0 0x7f5a141a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:16.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.261+0000 7f5a13400700 1 --2- 172.21.15.43:0/3633640386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a1410eb90 0x7f5a1419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:16.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.262+0000 7f5a12000700 1 -- 172.21.15.43:0/3633640386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a1410eb90 msgr2=0x7f5a1419a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:16.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.262+0000 7f5a12000700 1 --2- 172.21.15.43:0/3633640386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a1410eb90 0x7f5a1419a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:16.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.262+0000 7f5a12000700 1 -- 172.21.15.43:0/3633640386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a140743c0 msgr2=0x7f5a141a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:16.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.262+0000 7f5a12000700 1 --2- 172.21.15.43:0/3633640386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a140743c0 0x7f5a141a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:16.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.262+0000 7f5a12000700 1 -- 172.21.15.43:0/3633640386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1419af90 con 0x7f5a14074ce0 2024-03-26T08:30:16.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.262+0000 7f5a12000700 1 --2- 172.21.15.43:0/3633640386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a14074ce0 0x7f5a141a0b70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000c920 tx=0x7f5a0000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:16.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.262+0000 7f5a0b400700 1 -- 172.21.15.43:0/3633640386 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a00007b00 con 0x7f5a14074ce0 2024-03-26T08:30:16.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.262+0000 7f5a0b400700 1 -- 172.21.15.43:0/3633640386 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5a00004ce0 con 0x7f5a14074ce0 2024-03-26T08:30:16.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.262+0000 7f5a19600700 1 -- 172.21.15.43:0/3633640386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a1419b280 con 0x7f5a14074ce0 2024-03-26T08:30:16.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.262+0000 7f5a0b400700 1 -- 172.21.15.43:0/3633640386 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a00005550 con 0x7f5a14074ce0 2024-03-26T08:30:16.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.262+0000 7f5a19600700 1 -- 172.21.15.43:0/3633640386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a141a7490 con 0x7f5a14074ce0 2024-03-26T08:30:16.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.264+0000 7f5a19600700 1 -- 172.21.15.43:0/3633640386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a14075af0 con 0x7f5a14074ce0 2024-03-26T08:30:16.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.266+0000 7f5a0b400700 1 -- 172.21.15.43:0/3633640386 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5a00004750 con 0x7f5a14074ce0 2024-03-26T08:30:16.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.267+0000 7f5a0b400700 1 --2- 172.21.15.43:0/3633640386 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f59f80a2910 0x7f59f80a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:16.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.267+0000 7f5a0b400700 1 -- 172.21.15.43:0/3633640386 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5a000ce4f0 con 0x7f5a14074ce0 2024-03-26T08:30:16.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.267+0000 7f5a12a00700 1 --2- 172.21.15.43:0/3633640386 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f59f80a2910 0x7f59f80a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:16.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.268+0000 7f5a12a00700 1 --2- 172.21.15.43:0/3633640386 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f59f80a2910 0x7f59f80a4dd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5a04005ea0 tx=0x7f5a04005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:16.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.272+0000 7f5a0b400700 1 -- 172.21.15.43:0/3633640386 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5a00014030 con 0x7f5a14074ce0 2024-03-26T08:30:16.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.500+0000 7f5a19600700 1 -- 172.21.15.43:0/3633640386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a141a7750 con 0x7f5a14074ce0 2024-03-26T08:30:16.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.503+0000 7f5a0b400700 1 -- 172.21.15.43:0/3633640386 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18408 (secure 0 0 0) 0x7f5a0009b990 con 0x7f5a14074ce0 2024-03-26T08:30:16.497 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:30:16.497 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":82,"fsid":"88759420-eb49-11ee-95d0-87774f69a715","created":"2024-03-26T08:20:27.817107+0000","modified":"2024-03-26T08:30:11.537628+0000","last_up_change":"2024-03-26T08:30:10.520976+0000","last_in_change":"2024-03-26T08:29:46.394360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:24:54.852427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"28670673-9193-4fa3-9cc0-c1a5a6b27284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6803","nonce":936601146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6805","nonce":936601146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6809","nonce":936601146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6807","nonce":936601146}]},"public_addr":"172.21.15.43:6803/936601146","cluster_addr":"172.21.15.43:6805/936601146","heartbeat_back_addr":"172.21.15.43:6809/936601146","heartbeat_front_addr":"172.21.15.43:6807/936601146","state":["exists","up"]},{"osd":1,"uuid":"fa8ad6b6-277a-42b8-b0f8-b36fc219de2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6801","nonce":3268412328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6803","nonce":3268412328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6807","nonce":3268412328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6805","nonce":3268412328}]},"public_addr":"172.21.15.52:6801/3268412328","cluster_addr":"172.21.15.52:6803/3268412328","heartbeat_back_addr":"172.21.15.52:6807/3268412328","heartbeat_front_addr":"172.21.15.52:6805/3268412328","state":["exists","up"]},{"osd":2,"uuid":"ec0039c3-f9bd-419a-81de-fa4418359808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6801","nonce":2103180908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6803","nonce":2103180908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6807","nonce":2103180908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6805","nonce":2103180908}]},"public_addr":"172.21.15.183:6801/2103180908","cluster_addr":"172.21.15.183:6803/2103180908","heartbeat_back_addr":"172.21.15.183:6807/2103180908","heartbeat_front_addr":"172.21.15.183:6805/2103180908","state":["exists","up"]},{"osd":3,"uuid":"e7dcbf00-6586-4664-a69e-3cb0df8dfd30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6811","nonce":3722846033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6813","nonce":3722846033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6817","nonce":3722846033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6815","nonce":3722846033}]},"public_addr":"172.21.15.43:6811/3722846033","cluster_addr":"172.21.15.43:6813/3722846033","heartbeat_back_addr":"172.21.15.43:6817/3722846033","heartbeat_front_addr":"172.21.15.43:6815/3722846033","state":["exists","up"]},{"osd":4,"uuid":"db089a49-5e98-432d-9687-cdc044114c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6809","nonce":4028436523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6811","nonce":4028436523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6815","nonce":4028436523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6813","nonce":4028436523}]},"public_addr":"172.21.15.52:6809/4028436523","cluster_addr":"172.21.15.52:6811/4028436523","heartbeat_back_addr":"172.21.15.52:6815/4028436523","heartbeat_front_addr":"172.21.15.52:6813/4028436523","state":["exists","up"]},{"osd":5,"uuid":"556ac804-02ea-423b-a599-55baf5be4ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6809","nonce":787132252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6811","nonce":787132252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6815","nonce":787132252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6813","nonce":787132252}]},"public_addr":"172.21.15.183:6809/787132252","cluster_addr":"172.21.15.183:6811/787132252","heartbeat_back_addr":"172.21.15.183:6815/787132252","heartbeat_front_addr":"172.21.15.183:6813/787132252","state":["exists","up"]},{"osd":6,"uuid":"3de2a49d-6cef-4580-9d8a-f90cdfcecd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6819","nonce":4206861158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6821","nonce":4206861158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6825","nonce":4206861158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6823","nonce":4206861158}]},"public_addr":"172.21.15.43:6819/4206861158","cluster_addr":"172.21.15.43:6821/4206861158","heartbeat_back_addr":"172.21.15.43:6825/4206861158","heartbeat_front_addr":"172.21.15.43:6823/4206861158","state":["exists","up"]},{"osd":7,"uuid":"a3b832f7-7d7d-4a62-b679-db6da98275d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6817","nonce":3452064146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6819","nonce":3452064146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6823","nonce":3452064146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6821","nonce":3452064146}]},"public_addr":"172.21.15.52:6817/3452064146","cluster_addr":"172.21.15.52:6819/3452064146","heartbeat_back_addr":"172.21.15.52:6823/3452064146","heartbeat_front_addr":"172.21.15.52:6821/3452064146","state":["exists","up"]},{"osd":8,"uuid":"ddc6b07f-d703-414d-a59a-02316376b13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6817","nonce":1023131725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6819","nonce":1023131725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6823","nonce":1023131725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6821","nonce":1023131725}]},"public_addr":"172.21.15.183:6817/1023131725","cluster_addr":"172.21.15.183:6819/1023131725","heartbeat_back_addr":"172.21.15.183:6823/1023131725","heartbeat_front_addr":"172.21.15.183:6821/1023131725","state":["exists","up"]},{"osd":9,"uuid":"ac16bb0e-6232-45c8-b16a-4d878874c588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6827","nonce":3935195657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6829","nonce":3935195657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6833","nonce":3935195657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6831","nonce":3935195657}]},"public_addr":"172.21.15.43:6827/3935195657","cluster_addr":"172.21.15.43:6829/3935195657","heartbeat_back_addr":"172.21.15.43:6833/3935195657","heartbeat_front_addr":"172.21.15.43:6831/3935195657","state":["exists","up"]},{"osd":10,"uuid":"fd839188-f398-4cff-aa7b-36522ff1a8a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6825","nonce":377628320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6827","nonce":377628320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6831","nonce":377628320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6829","nonce":377628320}]},"public_addr":"172.21.15.52:6825/377628320","cluster_addr":"172.21.15.52:6827/377628320","heartbeat_back_addr":"172.21.15.52:6831/377628320","heartbeat_front_addr":"172.21.15.52:6829/377628320","state":["exists","up"]},{"osd":11,"uuid":"a40282a0-0c64-49ee-b64e-6f3f8119e827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6825","nonce":1373086433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6827","nonce":1373086433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6831","nonce":1373086433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6829","nonce":1373086433}]},"public_addr":"172.21.15.183:6825/1373086433","cluster_addr":"172.21.15.183:6827/1373086433","heartbeat_back_addr":"172.21.15.183:6831/1373086433","heartbeat_front_addr":"172.21.15.183:6829/1373086433","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:23:43.478905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:17.573532+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:50.286610+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:25:27.407663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:03.081294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:36.057416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:12.062013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:47.032271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:21.023201+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:57.364739+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:29:34.039995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:30:08.827577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6801/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/4240775295":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/510203140":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/4251957687":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:6801/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1885244665":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/298320867":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1301092625":"2024-03-27T08:21:36.149092+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:30:16.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.506+0000 7f5a19600700 1 -- 172.21.15.43:0/3633640386 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f59f80a2910 msgr2=0x7f59f80a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:16.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.506+0000 7f5a19600700 1 --2- 172.21.15.43:0/3633640386 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f59f80a2910 0x7f59f80a4dd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5a04005ea0 tx=0x7f5a04005e30 comp rx=0 tx=0).stop 2024-03-26T08:30:16.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.506+0000 7f5a19600700 1 -- 172.21.15.43:0/3633640386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a14074ce0 msgr2=0x7f5a141a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:16.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.506+0000 7f5a19600700 1 --2- 172.21.15.43:0/3633640386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a14074ce0 0x7f5a141a0b70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000c920 tx=0x7f5a0000cc30 comp rx=0 tx=0).stop 2024-03-26T08:30:16.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.506+0000 7f5a19600700 1 -- 172.21.15.43:0/3633640386 shutdown_connections 2024-03-26T08:30:16.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.506+0000 7f5a19600700 1 --2- 172.21.15.43:0/3633640386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a1410eb90 0x7f5a1419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:16.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.506+0000 7f5a19600700 1 --2- 172.21.15.43:0/3633640386 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f59f80a2910 0x7f59f80a4dd0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:16.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.506+0000 7f5a19600700 1 --2- 172.21.15.43:0/3633640386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a140743c0 0x7f5a141a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:16.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.506+0000 7f5a19600700 1 --2- 172.21.15.43:0/3633640386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a14074ce0 0x7f5a141a0b70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:16.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.506+0000 7f5a19600700 1 -- 172.21.15.43:0/3633640386 >> 172.21.15.43:0/3633640386 conn(0x7f5a140792f0 msgr2=0x7f5a140ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:16.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.507+0000 7f5a19600700 1 -- 172.21.15.43:0/3633640386 shutdown_connections 2024-03-26T08:30:16.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:16.507+0000 7f5a19600700 1 -- 172.21.15.43:0/3633640386 wait complete. 2024-03-26T08:30:16.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:16 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3633640386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:30:17.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:16 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3633640386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:30:17.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:16 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3633640386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:30:17.571 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T08:24:54.852427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T08:30:17.572 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-26T08:30:17.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:17 smithi043 ceph-mon[31929]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:18.036 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:30:18.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:17 smithi183 ceph-mon[35818]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:18.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:17 smithi052 ceph-mon[35071]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:20.033 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:19 smithi043 ceph-mon[31929]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:20.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:19 smithi183 ceph-mon[35818]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:20.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:19 smithi052 ceph-mon[35071]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:20.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.216+0000 7f49f4400700 1 -- 172.21.15.43:0/4226040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec1081e0 msgr2=0x7f49ec1085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:20.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.216+0000 7f49f4400700 1 --2- 172.21.15.43:0/4226040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec1081e0 0x7f49ec1085c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f49e00099c0 tx=0x7f49e0009cd0 comp rx=0 tx=0).stop 2024-03-26T08:30:20.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.216+0000 7f49f4400700 1 -- 172.21.15.43:0/4226040 shutdown_connections 2024-03-26T08:30:20.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.216+0000 7f49f4400700 1 --2- 172.21.15.43:0/4226040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49ec06a7e0 0x7f49ec10f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:20.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.216+0000 7f49f4400700 1 --2- 172.21.15.43:0/4226040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec1081e0 0x7f49ec1085c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:20.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.216+0000 7f49f4400700 1 --2- 172.21.15.43:0/4226040 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f49ec069e40 0x7f49ec06a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:20.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.216+0000 7f49f4400700 1 -- 172.21.15.43:0/4226040 >> 172.21.15.43:0/4226040 conn(0x7f49ec077fc0 msgr2=0x7f49ec0783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:20.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.216+0000 7f49f4400700 1 -- 172.21.15.43:0/4226040 shutdown_connections 2024-03-26T08:30:20.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.216+0000 7f49f4400700 1 -- 172.21.15.43:0/4226040 wait complete. 2024-03-26T08:30:20.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.217+0000 7f49f4400700 1 Processor -- start 2024-03-26T08:30:20.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.217+0000 7f49f4400700 1 -- start start 2024-03-26T08:30:20.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.218+0000 7f49f4400700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f49ec069e40 0x7f49ec1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:20.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.218+0000 7f49f4400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49ec06a7e0 0x7f49ec1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:20.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.218+0000 7f49f4400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec1081e0 0x7f49ec19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:20.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.218+0000 7f49f4400700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec112350 con 0x7f49ec1081e0 2024-03-26T08:30:20.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.218+0000 7f49f4400700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec1121d0 con 0x7f49ec069e40 2024-03-26T08:30:20.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.218+0000 7f49f4400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec1124d0 con 0x7f49ec06a7e0 2024-03-26T08:30:20.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.218+0000 7f49f2400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec1081e0 0x7f49ec19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:20.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.218+0000 7f49f2400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec1081e0 0x7f49ec19a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52612/0 (socket says 172.21.15.43:52612) 2024-03-26T08:30:20.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.218+0000 7f49f2400700 1 -- 172.21.15.43:0/705063877 learned_addr learned my addr 172.21.15.43:0/705063877 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:30:20.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.218+0000 7f49f1a00700 1 --2- 172.21.15.43:0/705063877 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f49ec069e40 0x7f49ec1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:20.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.219+0000 7f49f1000700 1 --2- 172.21.15.43:0/705063877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49ec06a7e0 0x7f49ec1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:20.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.219+0000 7f49f2400700 1 -- 172.21.15.43:0/705063877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49ec06a7e0 msgr2=0x7f49ec1a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:20.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.219+0000 7f49f2400700 1 --2- 172.21.15.43:0/705063877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49ec06a7e0 0x7f49ec1a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:20.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.219+0000 7f49f2400700 1 -- 172.21.15.43:0/705063877 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f49ec069e40 msgr2=0x7f49ec1a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:20.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.219+0000 7f49f2400700 1 --2- 172.21.15.43:0/705063877 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f49ec069e40 0x7f49ec1a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:20.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.219+0000 7f49f2400700 1 -- 172.21.15.43:0/705063877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ec19afc0 con 0x7f49ec1081e0 2024-03-26T08:30:20.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.219+0000 7f49f2400700 1 --2- 172.21.15.43:0/705063877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec1081e0 0x7f49ec19a860 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f49e400b970 tx=0x7f49e400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:20.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.219+0000 7f49ea000700 1 -- 172.21.15.43:0/705063877 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e4019070 con 0x7f49ec1081e0 2024-03-26T08:30:20.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.220+0000 7f49f4400700 1 -- 172.21.15.43:0/705063877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49ec19b2b0 con 0x7f49ec1081e0 2024-03-26T08:30:20.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.220+0000 7f49f4400700 1 -- 172.21.15.43:0/705063877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49ec1a7570 con 0x7f49ec1081e0 2024-03-26T08:30:20.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.220+0000 7f49ea000700 1 -- 172.21.15.43:0/705063877 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f49e400cde0 con 0x7f49ec1081e0 2024-03-26T08:30:20.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.220+0000 7f49ea000700 1 -- 172.21.15.43:0/705063877 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e401d3b0 con 0x7f49ec1081e0 2024-03-26T08:30:20.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.221+0000 7f49f4400700 1 -- 172.21.15.43:0/705063877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49cc005330 con 0x7f49ec1081e0 2024-03-26T08:30:20.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.223+0000 7f49ea000700 1 -- 172.21.15.43:0/705063877 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f49e401c070 con 0x7f49ec1081e0 2024-03-26T08:30:20.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.224+0000 7f49ea000700 1 --2- 172.21.15.43:0/705063877 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f49d40a2910 0x7f49d40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:20.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.224+0000 7f49ea000700 1 -- 172.21.15.43:0/705063877 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f49e40cd8a0 con 0x7f49ec1081e0 2024-03-26T08:30:20.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.224+0000 7f49f1a00700 1 --2- 172.21.15.43:0/705063877 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f49d40a2910 0x7f49d40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:20.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.225+0000 7f49f1a00700 1 --2- 172.21.15.43:0/705063877 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f49d40a2910 0x7f49d40a4dd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f49e0000c00 tx=0x7f49e0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:20.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.231+0000 7f49ea000700 1 -- 172.21.15.43:0/705063877 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f49e409ad40 con 0x7f49ec1081e0 2024-03-26T08:30:20.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.457+0000 7f49f4400700 1 -- 172.21.15.43:0/705063877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f49cc005e30 con 0x7f49ec1081e0 2024-03-26T08:30:20.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.458+0000 7f49ea000700 1 -- 172.21.15.43:0/705063877 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f49e4096030 con 0x7f49ec1081e0 2024-03-26T08:30:20.452 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-26T08:30:20.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.461+0000 7f49f4400700 1 -- 172.21.15.43:0/705063877 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f49d40a2910 msgr2=0x7f49d40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:20.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.461+0000 7f49f4400700 1 --2- 172.21.15.43:0/705063877 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f49d40a2910 0x7f49d40a4dd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f49e0000c00 tx=0x7f49e0005fb0 comp rx=0 tx=0).stop 2024-03-26T08:30:20.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.461+0000 7f49f4400700 1 -- 172.21.15.43:0/705063877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec1081e0 msgr2=0x7f49ec19a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:20.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.461+0000 7f49f4400700 1 --2- 172.21.15.43:0/705063877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec1081e0 0x7f49ec19a860 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f49e400b970 tx=0x7f49e400bc80 comp rx=0 tx=0).stop 2024-03-26T08:30:20.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.461+0000 7f49f4400700 1 -- 172.21.15.43:0/705063877 shutdown_connections 2024-03-26T08:30:20.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.461+0000 7f49f4400700 1 --2- 172.21.15.43:0/705063877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec1081e0 0x7f49ec19a860 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:20.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.461+0000 7f49f4400700 1 --2- 172.21.15.43:0/705063877 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f49d40a2910 0x7f49d40a4dd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:20.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.461+0000 7f49f4400700 1 --2- 172.21.15.43:0/705063877 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f49ec069e40 0x7f49ec1a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:20.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.461+0000 7f49f4400700 1 --2- 172.21.15.43:0/705063877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49ec06a7e0 0x7f49ec1a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:20.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.461+0000 7f49f4400700 1 -- 172.21.15.43:0/705063877 >> 172.21.15.43:0/705063877 conn(0x7f49ec077fc0 msgr2=0x7f49ec0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:20.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.462+0000 7f49f4400700 1 -- 172.21.15.43:0/705063877 shutdown_connections 2024-03-26T08:30:20.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:20.462+0000 7f49f4400700 1 -- 172.21.15.43:0/705063877 wait complete. 2024-03-26T08:30:21.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:20 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/705063877' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:30:21.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:20 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/705063877' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:30:21.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:20 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/705063877' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:30:21.406 INFO:tasks.cephadm:Adding mds.a on smithi043 2024-03-26T08:30:21.406 INFO:tasks.cephadm:Adding mds.d on smithi043 2024-03-26T08:30:21.407 INFO:tasks.cephadm:Adding mds.g on smithi043 2024-03-26T08:30:21.407 INFO:tasks.cephadm:Adding mds.j on smithi043 2024-03-26T08:30:21.407 INFO:tasks.cephadm:Adding mds.b on smithi052 2024-03-26T08:30:21.407 INFO:tasks.cephadm:Adding mds.e on smithi052 2024-03-26T08:30:21.407 INFO:tasks.cephadm:Adding mds.h on smithi052 2024-03-26T08:30:21.407 INFO:tasks.cephadm:Adding mds.k on smithi052 2024-03-26T08:30:21.407 INFO:tasks.cephadm:Adding mds.c on smithi183 2024-03-26T08:30:21.407 INFO:tasks.cephadm:Adding mds.f on smithi183 2024-03-26T08:30:21.407 INFO:tasks.cephadm:Adding mds.i on smithi183 2024-03-26T08:30:21.407 INFO:tasks.cephadm:Adding mds.l on smithi183 2024-03-26T08:30:21.408 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph orch apply mds all '12;smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi052=b;smithi052=e;smithi052=h;smithi052=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l' 2024-03-26T08:30:21.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:21 smithi183 ceph-mon[35818]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:21.860 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:30:22.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:21 smithi052 ceph-mon[35071]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:22.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:21 smithi043 ceph-mon[31929]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:23.929 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.936+0000 7f8cfd600700 1 -- 172.21.15.183:0/2747281998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cf8101dc0 msgr2=0x7f8cf8102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:23.930 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.936+0000 7f8cfd600700 1 --2- 172.21.15.183:0/2747281998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cf8101dc0 0x7f8cf8102240 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0009990 tx=0x7f8cf0009ca0 comp rx=0 tx=0).stop 2024-03-26T08:30:23.930 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.936+0000 7f8cfd600700 1 -- 172.21.15.183:0/2747281998 shutdown_connections 2024-03-26T08:30:23.930 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.936+0000 7f8cfd600700 1 --2- 172.21.15.183:0/2747281998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cf8101dc0 0x7f8cf8102240 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:23.930 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.936+0000 7f8cfd600700 1 --2- 172.21.15.183:0/2747281998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cf8100aa0 0x7f8cf8100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:23.930 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.936+0000 7f8cfd600700 1 --2- 172.21.15.183:0/2747281998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf8101400 0x7f8cf8101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:23.930 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.936+0000 7f8cfd600700 1 -- 172.21.15.183:0/2747281998 >> 172.21.15.183:0/2747281998 conn(0x7f8cf80fc6e0 msgr2=0x7f8cf80feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:23.930 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.936+0000 7f8cfd600700 1 -- 172.21.15.183:0/2747281998 shutdown_connections 2024-03-26T08:30:23.930 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.936+0000 7f8cfd600700 1 -- 172.21.15.183:0/2747281998 wait complete. 2024-03-26T08:30:23.930 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.937+0000 7f8cfd600700 1 Processor -- start 2024-03-26T08:30:23.932 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.937+0000 7f8cfd600700 1 -- start start 2024-03-26T08:30:23.932 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.938+0000 7f8cfd600700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cf8100aa0 0x7f8cf819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:23.932 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.938+0000 7f8cfd600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf8101400 0x7f8cf819a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:23.932 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.938+0000 7f8cfd600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cf8101dc0 0x7f8cf81a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:23.932 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.938+0000 7f8cfd600700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf810fe60 con 0x7f8cf8101400 2024-03-26T08:30:23.932 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.938+0000 7f8cfd600700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf810fce0 con 0x7f8cf8100aa0 2024-03-26T08:30:23.932 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.938+0000 7f8cfd600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf810ffe0 con 0x7f8cf8101dc0 2024-03-26T08:30:23.932 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.938+0000 7f8cf7400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cf8101dc0 0x7f8cf81a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:23.932 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.938+0000 7f8cf7400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cf8101dc0 0x7f8cf81a19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:34018/0 (socket says 172.21.15.183:34018) 2024-03-26T08:30:23.932 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.938+0000 7f8cf7400700 1 -- 172.21.15.183:0/3754217999 learned_addr learned my addr 172.21.15.183:0/3754217999 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-03-26T08:30:23.933 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.938+0000 7f8cee000700 1 --2- 172.21.15.183:0/3754217999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf8101400 0x7f8cf819a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:23.933 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.938+0000 7f8cf6a00700 1 --2- 172.21.15.183:0/3754217999 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cf8100aa0 0x7f8cf819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:23.933 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.939+0000 7f8cf7400700 1 -- 172.21.15.183:0/3754217999 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cf8100aa0 msgr2=0x7f8cf819a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:23.933 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.939+0000 7f8cf7400700 1 --2- 172.21.15.183:0/3754217999 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cf8100aa0 0x7f8cf819a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:23.933 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.939+0000 7f8cf7400700 1 -- 172.21.15.183:0/3754217999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf8101400 msgr2=0x7f8cf819a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:23.933 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.939+0000 7f8cf7400700 1 --2- 172.21.15.183:0/3754217999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf8101400 0x7f8cf819a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:23.933 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.939+0000 7f8cf7400700 1 -- 172.21.15.183:0/3754217999 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf81a2040 con 0x7f8cf8101dc0 2024-03-26T08:30:23.933 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.939+0000 7f8cf7400700 1 --2- 172.21.15.183:0/3754217999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cf8101dc0 0x7f8cf81a19a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0005910 tx=0x7f8cf0005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:23.933 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.939+0000 7f8cefe00700 1 -- 172.21.15.183:0/3754217999 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0027070 con 0x7f8cf8101dc0 2024-03-26T08:30:23.933 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.939+0000 7f8cefe00700 1 -- 172.21.15.183:0/3754217999 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8cf00041d0 con 0x7f8cf8101dc0 2024-03-26T08:30:23.933 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.939+0000 7f8cefe00700 1 -- 172.21.15.183:0/3754217999 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf000f460 con 0x7f8cf8101dc0 2024-03-26T08:30:23.933 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.939+0000 7f8cfd600700 1 -- 172.21.15.183:0/3754217999 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cf81a22d0 con 0x7f8cf8101dc0 2024-03-26T08:30:23.933 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.940+0000 7f8cfd600700 1 -- 172.21.15.183:0/3754217999 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf81a26d0 con 0x7f8cf8101dc0 2024-03-26T08:30:23.934 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.941+0000 7f8cfd600700 1 -- 172.21.15.183:0/3754217999 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cf8068e50 con 0x7f8cf8101dc0 2024-03-26T08:30:23.935 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.943+0000 7f8cefe00700 1 -- 172.21.15.183:0/3754217999 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8cf002a070 con 0x7f8cf8101dc0 2024-03-26T08:30:23.936 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.944+0000 7f8cefe00700 1 --2- 172.21.15.183:0/3754217999 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8cdc0a2890 0x7f8cdc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:23.936 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.944+0000 7f8cefe00700 1 -- 172.21.15.183:0/3754217999 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8cf00cf5c0 con 0x7f8cf8101dc0 2024-03-26T08:30:23.937 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.945+0000 7f8cf6a00700 1 --2- 172.21.15.183:0/3754217999 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8cdc0a2890 0x7f8cdc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:23.938 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.946+0000 7f8cf6a00700 1 --2- 172.21.15.183:0/3754217999 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8cdc0a2890 0x7f8cdc0a4d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8007fd0 tx=0x7f8ce8009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:23.941 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:23.949+0000 7f8cefe00700 1 -- 172.21.15.183:0/3754217999 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8cf005e740 con 0x7f8cf8101dc0 2024-03-26T08:30:24.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:23 smithi183 ceph-mon[35818]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:24.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:23 smithi052 ceph-mon[35071]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:24.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:23 smithi043 ceph-mon[31929]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:24.174 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:24.182+0000 7f8cfd600700 1 -- 172.21.15.183:0/3754217999 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi052=b;smithi052=e;smithi052=h;smithi052=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f8cf8062890 con 0x7f8cdc0a2890 2024-03-26T08:30:24.193 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:24.200+0000 7f8cefe00700 1 -- 172.21.15.183:0/3754217999 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f8cf8062890 con 0x7f8cdc0a2890 2024-03-26T08:30:24.193 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled mds.all update... 2024-03-26T08:30:24.196 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:24.203+0000 7f8cfd600700 1 -- 172.21.15.183:0/3754217999 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8cdc0a2890 msgr2=0x7f8cdc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:24.196 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:24.203+0000 7f8cfd600700 1 --2- 172.21.15.183:0/3754217999 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8cdc0a2890 0x7f8cdc0a4d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8007fd0 tx=0x7f8ce8009040 comp rx=0 tx=0).stop 2024-03-26T08:30:24.196 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:24.203+0000 7f8cfd600700 1 -- 172.21.15.183:0/3754217999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cf8101dc0 msgr2=0x7f8cf81a19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:24.196 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:24.203+0000 7f8cfd600700 1 --2- 172.21.15.183:0/3754217999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cf8101dc0 0x7f8cf81a19a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0005910 tx=0x7f8cf0005ef0 comp rx=0 tx=0).stop 2024-03-26T08:30:24.196 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:24.203+0000 7f8cfd600700 1 -- 172.21.15.183:0/3754217999 shutdown_connections 2024-03-26T08:30:24.196 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:24.203+0000 7f8cfd600700 1 --2- 172.21.15.183:0/3754217999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cf8101dc0 0x7f8cf81a19a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:24.196 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:24.203+0000 7f8cfd600700 1 --2- 172.21.15.183:0/3754217999 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8cdc0a2890 0x7f8cdc0a4d50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:24.196 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:24.203+0000 7f8cfd600700 1 --2- 172.21.15.183:0/3754217999 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cf8100aa0 0x7f8cf819a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:24.196 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:24.203+0000 7f8cfd600700 1 --2- 172.21.15.183:0/3754217999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cf8101400 0x7f8cf819a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:24.196 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:24.203+0000 7f8cfd600700 1 -- 172.21.15.183:0/3754217999 >> 172.21.15.183:0/3754217999 conn(0x7f8cf80fc6e0 msgr2=0x7f8cf80fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:24.196 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:24.203+0000 7f8cfd600700 1 -- 172.21.15.183:0/3754217999 shutdown_connections 2024-03-26T08:30:24.197 INFO:teuthology.orchestra.run.smithi183.stderr:2024-03-26T08:30:24.203+0000 7f8cfd600700 1 -- 172.21.15.183:0/3754217999 wait complete. 2024-03-26T08:30:24.933 DEBUG:teuthology.orchestra.run.smithi043:mds.a> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.a.service 2024-03-26T08:30:24.941 DEBUG:teuthology.orchestra.run.smithi043:mds.d> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.d.service 2024-03-26T08:30:24.987 DEBUG:teuthology.orchestra.run.smithi043:mds.g> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.g.service 2024-03-26T08:30:25.034 DEBUG:teuthology.orchestra.run.smithi043:mds.j> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.j.service 2024-03-26T08:30:25.080 DEBUG:teuthology.orchestra.run.smithi052:mds.b> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.b.service 2024-03-26T08:30:25.088 DEBUG:teuthology.orchestra.run.smithi052:mds.e> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.e.service 2024-03-26T08:30:25.134 DEBUG:teuthology.orchestra.run.smithi052:mds.h> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.h.service 2024-03-26T08:30:25.180 DEBUG:teuthology.orchestra.run.smithi052:mds.k> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.k.service 2024-03-26T08:30:25.225 DEBUG:teuthology.orchestra.run.smithi183:mds.c> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.c.service 2024-03-26T08:30:25.231 DEBUG:teuthology.orchestra.run.smithi183:mds.f> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.f.service 2024-03-26T08:30:25.276 DEBUG:teuthology.orchestra.run.smithi183:mds.i> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.i.service 2024-03-26T08:30:25.321 DEBUG:teuthology.orchestra.run.smithi183:mds.l> sudo journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.l.service 2024-03-26T08:30:25.367 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-26T08:30:25.367 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:30:25.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:25 smithi043 ceph-mon[31929]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi052=b;smithi052=e;smithi052=h;smithi052=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:30:25.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:25 smithi043 ceph-mon[31929]: Saving service mds.all spec with placement smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi052=b;smithi052=e;smithi052=h;smithi052=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2024-03-26T08:30:25.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:25 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:25.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:25 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:30:25.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:25 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:25.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:25 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:30:25.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:25 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:25.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:25 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:30:25.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:25 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:30:25.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:25 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:25.538 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:25 smithi183 ceph-mon[35818]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi052=b;smithi052=e;smithi052=h;smithi052=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:30:25.538 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:25 smithi183 ceph-mon[35818]: Saving service mds.all spec with placement smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi052=b;smithi052=e;smithi052=h;smithi052=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2024-03-26T08:30:25.539 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:25 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:25.539 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:25 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:30:25.539 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:25 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:25.539 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:25 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:30:25.539 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:25 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:25.539 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:25 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:30:25.539 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:25 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:30:25.539 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:25 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:25.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:25 smithi052 ceph-mon[35071]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi052=b;smithi052=e;smithi052=h;smithi052=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:30:25.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:25 smithi052 ceph-mon[35071]: Saving service mds.all spec with placement smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi052=b;smithi052=e;smithi052=h;smithi052=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2024-03-26T08:30:25.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:25 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:25.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:25 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:30:25.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:25 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:25.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:25 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:30:25.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:25 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:25.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:25 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:30:25.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:25 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:30:25.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:25 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:25.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.623+0000 7f9263e00640 1 -- 172.21.15.43:0/1035132017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9264109390 msgr2=0x7f926410b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:25.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.623+0000 7f9263e00640 1 --2- 172.21.15.43:0/1035132017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9264109390 0x7f926410b830 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f925c009900 tx=0x7f925c02bcd0 comp rx=0 tx=0).stop 2024-03-26T08:30:25.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.623+0000 7f9263e00640 1 -- 172.21.15.43:0/1035132017 shutdown_connections 2024-03-26T08:30:25.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.623+0000 7f9263e00640 1 --2- 172.21.15.43:0/1035132017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9264109390 0x7f926410b830 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:25.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.623+0000 7f9263e00640 1 --2- 172.21.15.43:0/1035132017 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92640740a0 0x7f9264074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:25.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.623+0000 7f9263e00640 1 --2- 172.21.15.43:0/1035132017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9264075680 0x7f9264073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:25.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.623+0000 7f9263e00640 1 -- 172.21.15.43:0/1035132017 >> 172.21.15.43:0/1035132017 conn(0x7f92640fca70 msgr2=0x7f92640feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:25.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.623+0000 7f9263e00640 1 -- 172.21.15.43:0/1035132017 shutdown_connections 2024-03-26T08:30:25.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.624+0000 7f9263e00640 1 -- 172.21.15.43:0/1035132017 wait complete. 2024-03-26T08:30:25.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.624+0000 7f9263e00640 1 Processor -- start 2024-03-26T08:30:25.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.625+0000 7f9263e00640 1 -- start start 2024-03-26T08:30:25.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.625+0000 7f9263e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92640740a0 0x7f92641998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:25.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.625+0000 7f9263e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9264075680 0x7f9264199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:25.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.625+0000 7f9263e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9264109390 0x7f92641a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:25.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.625+0000 7f9263e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926410f280 con 0x7f92640740a0 2024-03-26T08:30:25.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.625+0000 7f9263e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926410f100 con 0x7f9264075680 2024-03-26T08:30:25.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.625+0000 7f9263e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926410f400 con 0x7f9264109390 2024-03-26T08:30:25.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.625+0000 7f9262a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92640740a0 0x7f92641998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:25.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.625+0000 7f9262a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92640740a0 0x7f92641998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33162/0 (socket says 172.21.15.43:33162) 2024-03-26T08:30:25.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.625+0000 7f9262a00640 1 -- 172.21.15.43:0/3668909173 learned_addr learned my addr 172.21.15.43:0/3668909173 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:30:25.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.625+0000 7f9263400640 1 --2- 172.21.15.43:0/3668909173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9264109390 0x7f92641a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:25.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.626+0000 7f9262a00640 1 -- 172.21.15.43:0/3668909173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9264109390 msgr2=0x7f92641a0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:25.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.626+0000 7f9262000640 1 --2- 172.21.15.43:0/3668909173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9264075680 0x7f9264199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:25.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.626+0000 7f9262a00640 1 --2- 172.21.15.43:0/3668909173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9264109390 0x7f92641a0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:25.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.626+0000 7f9262a00640 1 -- 172.21.15.43:0/3668909173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9264075680 msgr2=0x7f9264199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:25.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.626+0000 7f9262a00640 1 --2- 172.21.15.43:0/3668909173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9264075680 0x7f9264199e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:25.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.626+0000 7f9262a00640 1 -- 172.21.15.43:0/3668909173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92641a1500 con 0x7f92640740a0 2024-03-26T08:30:25.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.626+0000 7f9262a00640 1 --2- 172.21.15.43:0/3668909173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92640740a0 0x7f92641998e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f925400eb90 tx=0x7f925400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:25.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.626+0000 7f925b400640 1 -- 172.21.15.43:0/3668909173 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9254004d20 con 0x7f92640740a0 2024-03-26T08:30:25.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.626+0000 7f925b400640 1 -- 172.21.15.43:0/3668909173 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f925400ee90 con 0x7f92640740a0 2024-03-26T08:30:25.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.626+0000 7f925b400640 1 -- 172.21.15.43:0/3668909173 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9254010670 con 0x7f92640740a0 2024-03-26T08:30:25.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.626+0000 7f9263e00640 1 -- 172.21.15.43:0/3668909173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92641a17f0 con 0x7f92640740a0 2024-03-26T08:30:25.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.626+0000 7f9263e00640 1 -- 172.21.15.43:0/3668909173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92641a1cf0 con 0x7f92640740a0 2024-03-26T08:30:25.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.628+0000 7f9263e00640 1 -- 172.21.15.43:0/3668909173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9264108150 con 0x7f92640740a0 2024-03-26T08:30:25.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.630+0000 7f925b400640 1 -- 172.21.15.43:0/3668909173 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f9254005020 con 0x7f92640740a0 2024-03-26T08:30:25.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.631+0000 7f925b400640 1 --2- 172.21.15.43:0/3668909173 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f92480a2880 0x7f92480a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:25.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.631+0000 7f925b400640 1 -- 172.21.15.43:0/3668909173 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f92540cecd0 con 0x7f92640740a0 2024-03-26T08:30:25.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.631+0000 7f9262000640 1 --2- 172.21.15.43:0/3668909173 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f92480a2880 0x7f92480a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:25.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.632+0000 7f9262000640 1 --2- 172.21.15.43:0/3668909173 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f92480a2880 0x7f92480a4d40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9250005fd0 tx=0x7f9250005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:25.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.636+0000 7f925b400640 1 -- 172.21.15.43:0/3668909173 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f925409c170 con 0x7f92640740a0 2024-03-26T08:30:25.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.844+0000 7f9263e00640 1 -- 172.21.15.43:0/3668909173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f926402d2f0 con 0x7f92640740a0 2024-03-26T08:30:25.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.846+0000 7f925b400640 1 -- 172.21.15.43:0/3668909173 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18408 (secure 0 0 0) 0x7f925409ba70 con 0x7f92640740a0 2024-03-26T08:30:25.840 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:30:25.840 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":82,"fsid":"88759420-eb49-11ee-95d0-87774f69a715","created":"2024-03-26T08:20:27.817107+0000","modified":"2024-03-26T08:30:11.537628+0000","last_up_change":"2024-03-26T08:30:10.520976+0000","last_in_change":"2024-03-26T08:29:46.394360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:24:54.852427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"28670673-9193-4fa3-9cc0-c1a5a6b27284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6803","nonce":936601146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6805","nonce":936601146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6809","nonce":936601146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6807","nonce":936601146}]},"public_addr":"172.21.15.43:6803/936601146","cluster_addr":"172.21.15.43:6805/936601146","heartbeat_back_addr":"172.21.15.43:6809/936601146","heartbeat_front_addr":"172.21.15.43:6807/936601146","state":["exists","up"]},{"osd":1,"uuid":"fa8ad6b6-277a-42b8-b0f8-b36fc219de2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6801","nonce":3268412328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6803","nonce":3268412328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6807","nonce":3268412328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6805","nonce":3268412328}]},"public_addr":"172.21.15.52:6801/3268412328","cluster_addr":"172.21.15.52:6803/3268412328","heartbeat_back_addr":"172.21.15.52:6807/3268412328","heartbeat_front_addr":"172.21.15.52:6805/3268412328","state":["exists","up"]},{"osd":2,"uuid":"ec0039c3-f9bd-419a-81de-fa4418359808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6801","nonce":2103180908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6803","nonce":2103180908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6807","nonce":2103180908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6805","nonce":2103180908}]},"public_addr":"172.21.15.183:6801/2103180908","cluster_addr":"172.21.15.183:6803/2103180908","heartbeat_back_addr":"172.21.15.183:6807/2103180908","heartbeat_front_addr":"172.21.15.183:6805/2103180908","state":["exists","up"]},{"osd":3,"uuid":"e7dcbf00-6586-4664-a69e-3cb0df8dfd30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6811","nonce":3722846033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6813","nonce":3722846033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6817","nonce":3722846033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6815","nonce":3722846033}]},"public_addr":"172.21.15.43:6811/3722846033","cluster_addr":"172.21.15.43:6813/3722846033","heartbeat_back_addr":"172.21.15.43:6817/3722846033","heartbeat_front_addr":"172.21.15.43:6815/3722846033","state":["exists","up"]},{"osd":4,"uuid":"db089a49-5e98-432d-9687-cdc044114c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6809","nonce":4028436523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6811","nonce":4028436523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6815","nonce":4028436523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6813","nonce":4028436523}]},"public_addr":"172.21.15.52:6809/4028436523","cluster_addr":"172.21.15.52:6811/4028436523","heartbeat_back_addr":"172.21.15.52:6815/4028436523","heartbeat_front_addr":"172.21.15.52:6813/4028436523","state":["exists","up"]},{"osd":5,"uuid":"556ac804-02ea-423b-a599-55baf5be4ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6809","nonce":787132252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6811","nonce":787132252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6815","nonce":787132252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6813","nonce":787132252}]},"public_addr":"172.21.15.183:6809/787132252","cluster_addr":"172.21.15.183:6811/787132252","heartbeat_back_addr":"172.21.15.183:6815/787132252","heartbeat_front_addr":"172.21.15.183:6813/787132252","state":["exists","up"]},{"osd":6,"uuid":"3de2a49d-6cef-4580-9d8a-f90cdfcecd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6819","nonce":4206861158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6821","nonce":4206861158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6825","nonce":4206861158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6823","nonce":4206861158}]},"public_addr":"172.21.15.43:6819/4206861158","cluster_addr":"172.21.15.43:6821/4206861158","heartbeat_back_addr":"172.21.15.43:6825/4206861158","heartbeat_front_addr":"172.21.15.43:6823/4206861158","state":["exists","up"]},{"osd":7,"uuid":"a3b832f7-7d7d-4a62-b679-db6da98275d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6817","nonce":3452064146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6819","nonce":3452064146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6823","nonce":3452064146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6821","nonce":3452064146}]},"public_addr":"172.21.15.52:6817/3452064146","cluster_addr":"172.21.15.52:6819/3452064146","heartbeat_back_addr":"172.21.15.52:6823/3452064146","heartbeat_front_addr":"172.21.15.52:6821/3452064146","state":["exists","up"]},{"osd":8,"uuid":"ddc6b07f-d703-414d-a59a-02316376b13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6817","nonce":1023131725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6819","nonce":1023131725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6823","nonce":1023131725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6821","nonce":1023131725}]},"public_addr":"172.21.15.183:6817/1023131725","cluster_addr":"172.21.15.183:6819/1023131725","heartbeat_back_addr":"172.21.15.183:6823/1023131725","heartbeat_front_addr":"172.21.15.183:6821/1023131725","state":["exists","up"]},{"osd":9,"uuid":"ac16bb0e-6232-45c8-b16a-4d878874c588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6827","nonce":3935195657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6829","nonce":3935195657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6833","nonce":3935195657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6831","nonce":3935195657}]},"public_addr":"172.21.15.43:6827/3935195657","cluster_addr":"172.21.15.43:6829/3935195657","heartbeat_back_addr":"172.21.15.43:6833/3935195657","heartbeat_front_addr":"172.21.15.43:6831/3935195657","state":["exists","up"]},{"osd":10,"uuid":"fd839188-f398-4cff-aa7b-36522ff1a8a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6825","nonce":377628320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6827","nonce":377628320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6831","nonce":377628320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6829","nonce":377628320}]},"public_addr":"172.21.15.52:6825/377628320","cluster_addr":"172.21.15.52:6827/377628320","heartbeat_back_addr":"172.21.15.52:6831/377628320","heartbeat_front_addr":"172.21.15.52:6829/377628320","state":["exists","up"]},{"osd":11,"uuid":"a40282a0-0c64-49ee-b64e-6f3f8119e827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6825","nonce":1373086433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6827","nonce":1373086433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6831","nonce":1373086433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6829","nonce":1373086433}]},"public_addr":"172.21.15.183:6825/1373086433","cluster_addr":"172.21.15.183:6827/1373086433","heartbeat_back_addr":"172.21.15.183:6831/1373086433","heartbeat_front_addr":"172.21.15.183:6829/1373086433","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:23:43.478905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:17.573532+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:50.286610+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:25:27.407663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:03.081294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:36.057416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:12.062013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:47.032271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:21.023201+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:57.364739+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:29:34.039995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:30:08.827577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6801/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/4240775295":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/510203140":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/4251957687":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:6801/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1885244665":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/298320867":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1301092625":"2024-03-27T08:21:36.149092+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:30:25.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.849+0000 7f9263e00640 1 -- 172.21.15.43:0/3668909173 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f92480a2880 msgr2=0x7f92480a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:25.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.849+0000 7f9263e00640 1 --2- 172.21.15.43:0/3668909173 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f92480a2880 0x7f92480a4d40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9250005fd0 tx=0x7f9250005850 comp rx=0 tx=0).stop 2024-03-26T08:30:25.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.849+0000 7f9263e00640 1 -- 172.21.15.43:0/3668909173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92640740a0 msgr2=0x7f92641998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:25.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.849+0000 7f9263e00640 1 --2- 172.21.15.43:0/3668909173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92640740a0 0x7f92641998e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f925400eb90 tx=0x7f925400c6a0 comp rx=0 tx=0).stop 2024-03-26T08:30:25.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.849+0000 7f9263e00640 1 -- 172.21.15.43:0/3668909173 shutdown_connections 2024-03-26T08:30:25.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.849+0000 7f9263e00640 1 --2- 172.21.15.43:0/3668909173 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f92480a2880 0x7f92480a4d40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:25.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.849+0000 7f9263e00640 1 --2- 172.21.15.43:0/3668909173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9264109390 0x7f92641a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:25.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.849+0000 7f9263e00640 1 --2- 172.21.15.43:0/3668909173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9264075680 0x7f9264199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:25.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.849+0000 7f9263e00640 1 --2- 172.21.15.43:0/3668909173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92640740a0 0x7f92641998e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:25.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.849+0000 7f9263e00640 1 -- 172.21.15.43:0/3668909173 >> 172.21.15.43:0/3668909173 conn(0x7f92640fca70 msgr2=0x7f9264105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:25.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.849+0000 7f9263e00640 1 -- 172.21.15.43:0/3668909173 shutdown_connections 2024-03-26T08:30:25.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:25.849+0000 7f9263e00640 1 -- 172.21.15.43:0/3668909173 wait complete. 2024-03-26T08:30:25.862 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T08:24:54.852427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T08:30:25.862 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T08:30:26.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.144+0000 7fa79e200640 1 -- 172.21.15.43:0/4153338743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa798109770 msgr2=0x7fa798110000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:26.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.144+0000 7fa79e200640 1 --2- 172.21.15.43:0/4153338743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa798109770 0x7fa798110000 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa790009900 tx=0x7fa79002bc30 comp rx=0 tx=0).stop 2024-03-26T08:30:26.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.144+0000 7fa79e200640 1 -- 172.21.15.43:0/4153338743 shutdown_connections 2024-03-26T08:30:26.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.144+0000 7fa79e200640 1 --2- 172.21.15.43:0/4153338743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa798109770 0x7fa798110000 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:26.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.144+0000 7fa79e200640 1 --2- 172.21.15.43:0/4153338743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa798108db0 0x7fa798109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:26.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.144+0000 7fa79e200640 1 --2- 172.21.15.43:0/4153338743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa798107bc0 0x7fa798107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:26.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.144+0000 7fa79e200640 1 -- 172.21.15.43:0/4153338743 >> 172.21.15.43:0/4153338743 conn(0x7fa798076b70 msgr2=0x7fa798078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:26.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.144+0000 7fa79e200640 1 -- 172.21.15.43:0/4153338743 shutdown_connections 2024-03-26T08:30:26.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.144+0000 7fa79e200640 1 -- 172.21.15.43:0/4153338743 wait complete. 2024-03-26T08:30:26.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.145+0000 7fa79e200640 1 Processor -- start 2024-03-26T08:30:26.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.145+0000 7fa79e200640 1 -- start start 2024-03-26T08:30:26.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.145+0000 7fa79e200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa798107bc0 0x7fa79819dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:26.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.146+0000 7fa79e200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa798108db0 0x7fa79819e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:26.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.146+0000 7fa79e200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa798109770 0x7fa7981a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:26.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.146+0000 7fa79e200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa798113740 con 0x7fa798109770 2024-03-26T08:30:26.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.146+0000 7fa79e200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7981135c0 con 0x7fa798107bc0 2024-03-26T08:30:26.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.146+0000 7fa79e200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7981138c0 con 0x7fa798108db0 2024-03-26T08:30:26.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.146+0000 7fa79d800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa798109770 0x7fa7981a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:26.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.146+0000 7fa79d800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa798109770 0x7fa7981a5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33190/0 (socket says 172.21.15.43:33190) 2024-03-26T08:30:26.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.146+0000 7fa79d800640 1 -- 172.21.15.43:0/3818163144 learned_addr learned my addr 172.21.15.43:0/3818163144 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:30:26.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.146+0000 7fa79ce00640 1 --2- 172.21.15.43:0/3818163144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa798107bc0 0x7fa79819dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:26.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.146+0000 7fa78fe00640 1 --2- 172.21.15.43:0/3818163144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa798108db0 0x7fa79819e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:26.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.146+0000 7fa79d800640 1 -- 172.21.15.43:0/3818163144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa798108db0 msgr2=0x7fa79819e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:26.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.146+0000 7fa79d800640 1 --2- 172.21.15.43:0/3818163144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa798108db0 0x7fa79819e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:26.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.146+0000 7fa79d800640 1 -- 172.21.15.43:0/3818163144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa798107bc0 msgr2=0x7fa79819dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:26.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.146+0000 7fa79d800640 1 --2- 172.21.15.43:0/3818163144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa798107bc0 0x7fa79819dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:26.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.146+0000 7fa79d800640 1 -- 172.21.15.43:0/3818163144 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7981a5970 con 0x7fa798109770 2024-03-26T08:30:26.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.147+0000 7fa79d800640 1 --2- 172.21.15.43:0/3818163144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa798109770 0x7fa7981a5300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa790036040 tx=0x7fa790004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:26.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.147+0000 7fa796000640 1 -- 172.21.15.43:0/3818163144 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa790043020 con 0x7fa798109770 2024-03-26T08:30:26.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.147+0000 7fa79e200640 1 -- 172.21.15.43:0/3818163144 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7981a5c00 con 0x7fa798109770 2024-03-26T08:30:26.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.147+0000 7fa796000640 1 -- 172.21.15.43:0/3818163144 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa790004490 con 0x7fa798109770 2024-03-26T08:30:26.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.147+0000 7fa796000640 1 -- 172.21.15.43:0/3818163144 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa790003d10 con 0x7fa798109770 2024-03-26T08:30:26.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.147+0000 7fa79e200640 1 -- 172.21.15.43:0/3818163144 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7981a61a0 con 0x7fa798109770 2024-03-26T08:30:26.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.149+0000 7fa79e200640 1 -- 172.21.15.43:0/3818163144 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa79810eee0 con 0x7fa798109770 2024-03-26T08:30:26.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.151+0000 7fa796000640 1 -- 172.21.15.43:0/3818163144 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa790046020 con 0x7fa798109770 2024-03-26T08:30:26.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.152+0000 7fa796000640 1 --2- 172.21.15.43:0/3818163144 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa77c0a2800 0x7fa77c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:26.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.152+0000 7fa796000640 1 -- 172.21.15.43:0/3818163144 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa7900f0af0 con 0x7fa798109770 2024-03-26T08:30:26.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.152+0000 7fa79ce00640 1 --2- 172.21.15.43:0/3818163144 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa77c0a2800 0x7fa77c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:26.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.153+0000 7fa79ce00640 1 --2- 172.21.15.43:0/3818163144 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa77c0a2800 0x7fa77c0a4cc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa798108c10 tx=0x7fa788009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:26.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.156+0000 7fa796000640 1 -- 172.21.15.43:0/3818163144 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa79004d020 con 0x7fa798109770 2024-03-26T08:30:26.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.366+0000 7fa79e200640 1 -- 172.21.15.43:0/3818163144 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa79802d070 con 0x7fa798109770 2024-03-26T08:30:26.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.367+0000 7fa796000640 1 -- 172.21.15.43:0/3818163144 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa7900be0b0 con 0x7fa798109770 2024-03-26T08:30:26.361 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-26T08:30:26.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.371+0000 7fa79e200640 1 -- 172.21.15.43:0/3818163144 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa77c0a2800 msgr2=0x7fa77c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:26.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.371+0000 7fa79e200640 1 --2- 172.21.15.43:0/3818163144 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa77c0a2800 0x7fa77c0a4cc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa798108c10 tx=0x7fa788009870 comp rx=0 tx=0).stop 2024-03-26T08:30:26.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.371+0000 7fa79e200640 1 -- 172.21.15.43:0/3818163144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa798109770 msgr2=0x7fa7981a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:26.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.371+0000 7fa79e200640 1 --2- 172.21.15.43:0/3818163144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa798109770 0x7fa7981a5300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa790036040 tx=0x7fa790004270 comp rx=0 tx=0).stop 2024-03-26T08:30:26.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.371+0000 7fa79e200640 1 -- 172.21.15.43:0/3818163144 shutdown_connections 2024-03-26T08:30:26.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.371+0000 7fa79e200640 1 --2- 172.21.15.43:0/3818163144 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa77c0a2800 0x7fa77c0a4cc0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:26.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.371+0000 7fa79e200640 1 --2- 172.21.15.43:0/3818163144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa798109770 0x7fa7981a5300 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:26.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.371+0000 7fa79e200640 1 --2- 172.21.15.43:0/3818163144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa798108db0 0x7fa79819e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:26.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.371+0000 7fa79e200640 1 --2- 172.21.15.43:0/3818163144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa798107bc0 0x7fa79819dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:26.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.371+0000 7fa79e200640 1 -- 172.21.15.43:0/3818163144 >> 172.21.15.43:0/3818163144 conn(0x7fa798076b70 msgr2=0x7fa79810c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:26.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.371+0000 7fa79e200640 1 -- 172.21.15.43:0/3818163144 shutdown_connections 2024-03-26T08:30:26.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.371+0000 7fa79e200640 1 -- 172.21.15.43:0/3818163144 wait complete. 2024-03-26T08:30:26.386 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:30:26.441 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:26 smithi183 ceph-mon[35818]: Deploying daemon mds.l on smithi183 2024-03-26T08:30:26.442 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:26 smithi183 ceph-mon[35818]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:26.442 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:26 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3668909173' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:30:26.471 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:26 smithi043 ceph-mon[31929]: Deploying daemon mds.l on smithi183 2024-03-26T08:30:26.471 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:26 smithi043 ceph-mon[31929]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:26.471 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:26 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3668909173' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:30:26.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:26 smithi052 ceph-mon[35071]: Deploying daemon mds.l on smithi183 2024-03-26T08:30:26.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:26 smithi052 ceph-mon[35071]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:26.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:26 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3668909173' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:30:26.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.618+0000 7fceeb200640 1 -- 172.21.15.43:0/1842118381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceec074080 msgr2=0x7fceec0744e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:26.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.618+0000 7fceeb200640 1 --2- 172.21.15.43:0/1842118381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceec074080 0x7fceec0744e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fced4009900 tx=0x7fced402bcd0 comp rx=0 tx=0).stop 2024-03-26T08:30:26.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.618+0000 7fceeb200640 1 -- 172.21.15.43:0/1842118381 shutdown_connections 2024-03-26T08:30:26.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.618+0000 7fceeb200640 1 --2- 172.21.15.43:0/1842118381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fceec1093b0 0x7fceec10b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:26.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.618+0000 7fceeb200640 1 --2- 172.21.15.43:0/1842118381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceec074080 0x7fceec0744e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:26.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.618+0000 7fceeb200640 1 --2- 172.21.15.43:0/1842118381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fceec075660 0x7fceec073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:26.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.618+0000 7fceeb200640 1 -- 172.21.15.43:0/1842118381 >> 172.21.15.43:0/1842118381 conn(0x7fceec0fca70 msgr2=0x7fceec0feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:26.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.618+0000 7fceeb200640 1 -- 172.21.15.43:0/1842118381 shutdown_connections 2024-03-26T08:30:26.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.618+0000 7fceeb200640 1 -- 172.21.15.43:0/1842118381 wait complete. 2024-03-26T08:30:26.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.619+0000 7fceeb200640 1 Processor -- start 2024-03-26T08:30:26.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.619+0000 7fceeb200640 1 -- start start 2024-03-26T08:30:26.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.620+0000 7fceeb200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceec074080 0x7fceec199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:26.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.620+0000 7fceeb200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fceec075660 0x7fceec199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:26.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.620+0000 7fceeb200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fceec1093b0 0x7fceec1a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:26.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.620+0000 7fceeb200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceec10f290 con 0x7fceec074080 2024-03-26T08:30:26.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.620+0000 7fceeb200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceec10f110 con 0x7fceec075660 2024-03-26T08:30:26.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.620+0000 7fceeb200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceec10f410 con 0x7fceec1093b0 2024-03-26T08:30:26.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.620+0000 7fcee9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceec074080 0x7fceec199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:26.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.620+0000 7fcee9e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceec074080 0x7fceec199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33208/0 (socket says 172.21.15.43:33208) 2024-03-26T08:30:26.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.620+0000 7fcee9e00640 1 -- 172.21.15.43:0/3285826098 learned_addr learned my addr 172.21.15.43:0/3285826098 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:30:26.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.620+0000 7fcee9400640 1 --2- 172.21.15.43:0/3285826098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fceec075660 0x7fceec199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:26.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.620+0000 7fceea800640 1 --2- 172.21.15.43:0/3285826098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fceec1093b0 0x7fceec1a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:26.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.620+0000 7fcee9e00640 1 -- 172.21.15.43:0/3285826098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fceec1093b0 msgr2=0x7fceec1a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:26.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.620+0000 7fcee9e00640 1 --2- 172.21.15.43:0/3285826098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fceec1093b0 0x7fceec1a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:26.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.620+0000 7fcee9e00640 1 -- 172.21.15.43:0/3285826098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fceec075660 msgr2=0x7fceec199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:26.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.620+0000 7fcee9e00640 1 --2- 172.21.15.43:0/3285826098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fceec075660 0x7fceec199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:26.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.620+0000 7fcee9e00640 1 -- 172.21.15.43:0/3285826098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceec1a1550 con 0x7fceec074080 2024-03-26T08:30:26.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.620+0000 7fcee9400640 1 --2- 172.21.15.43:0/3285826098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fceec075660 0x7fceec199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:30:26.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.621+0000 7fcee9e00640 1 --2- 172.21.15.43:0/3285826098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceec074080 0x7fceec199980 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcedc00eab0 tx=0x7fcedc00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:26.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.621+0000 7fcee2a00640 1 -- 172.21.15.43:0/3285826098 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc00ce40 con 0x7fceec074080 2024-03-26T08:30:26.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.621+0000 7fcee2a00640 1 -- 172.21.15.43:0/3285826098 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcedc011920 con 0x7fceec074080 2024-03-26T08:30:26.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.621+0000 7fcee2a00640 1 -- 172.21.15.43:0/3285826098 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc010a00 con 0x7fceec074080 2024-03-26T08:30:26.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.621+0000 7fceeb200640 1 -- 172.21.15.43:0/3285826098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceec1a1840 con 0x7fceec074080 2024-03-26T08:30:26.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.621+0000 7fceeb200640 1 -- 172.21.15.43:0/3285826098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceec1a1d40 con 0x7fceec074080 2024-03-26T08:30:26.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.623+0000 7fceeb200640 1 -- 172.21.15.43:0/3285826098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceec108170 con 0x7fceec074080 2024-03-26T08:30:26.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.624+0000 7fcee2a00640 1 -- 172.21.15.43:0/3285826098 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fcedc011ad0 con 0x7fceec074080 2024-03-26T08:30:26.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.626+0000 7fcee2a00640 1 --2- 172.21.15.43:0/3285826098 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fced00a2880 0x7fced00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:26.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.626+0000 7fcee2a00640 1 -- 172.21.15.43:0/3285826098 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcedc0ced10 con 0x7fceec074080 2024-03-26T08:30:26.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.626+0000 7fcee9400640 1 --2- 172.21.15.43:0/3285826098 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fced00a2880 0x7fced00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:26.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.627+0000 7fcee9400640 1 --2- 172.21.15.43:0/3285826098 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fced00a2880 0x7fced00a4d40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fced4036040 tx=0x7fced4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:26.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.631+0000 7fcee2a00640 1 -- 172.21.15.43:0/3285826098 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcedc0d4020 con 0x7fceec074080 2024-03-26T08:30:26.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.873+0000 7fceeb200640 1 -- 172.21.15.43:0/3285826098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fceec02d070 con 0x7fceec074080 2024-03-26T08:30:26.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.874+0000 7fcee2a00640 1 -- 172.21.15.43:0/3285826098 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7fcedc016030 con 0x7fceec074080 2024-03-26T08:30:26.867 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:30:26.868 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-26T08:30:26.868 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 1 2024-03-26T08:30:26.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.877+0000 7fceeb200640 1 -- 172.21.15.43:0/3285826098 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fced00a2880 msgr2=0x7fced00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:26.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.877+0000 7fceeb200640 1 --2- 172.21.15.43:0/3285826098 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fced00a2880 0x7fced00a4d40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fced4036040 tx=0x7fced4005fd0 comp rx=0 tx=0).stop 2024-03-26T08:30:26.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.877+0000 7fceeb200640 1 -- 172.21.15.43:0/3285826098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceec074080 msgr2=0x7fceec199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:26.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.877+0000 7fceeb200640 1 --2- 172.21.15.43:0/3285826098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceec074080 0x7fceec199980 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcedc00eab0 tx=0x7fcedc00ef70 comp rx=0 tx=0).stop 2024-03-26T08:30:26.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.877+0000 7fceeb200640 1 -- 172.21.15.43:0/3285826098 shutdown_connections 2024-03-26T08:30:26.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.877+0000 7fceeb200640 1 --2- 172.21.15.43:0/3285826098 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fced00a2880 0x7fced00a4d40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:26.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.877+0000 7fceeb200640 1 --2- 172.21.15.43:0/3285826098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fceec1093b0 0x7fceec1a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:26.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.877+0000 7fceeb200640 1 --2- 172.21.15.43:0/3285826098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fceec075660 0x7fceec199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:26.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.877+0000 7fceeb200640 1 --2- 172.21.15.43:0/3285826098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceec074080 0x7fceec199980 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:26.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.877+0000 7fceeb200640 1 -- 172.21.15.43:0/3285826098 >> 172.21.15.43:0/3285826098 conn(0x7fceec0fca70 msgr2=0x7fceec105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:26.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.877+0000 7fceeb200640 1 -- 172.21.15.43:0/3285826098 shutdown_connections 2024-03-26T08:30:26.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:26.878+0000 7fceeb200640 1 -- 172.21.15.43:0/3285826098 wait complete. 2024-03-26T08:30:27.259 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:27 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3818163144' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:30:27.260 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:27 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3285826098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:27.548 INFO:journalctl@ceph.mds.l.smithi183.stdout:Mar 26 08:30:27 smithi183 systemd[1]: Starting Ceph mds.l for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:30:27.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:27 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3818163144' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:30:27.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:27 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3285826098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:27.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:27 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3818163144' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:30:27.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:27 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3285826098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:27.880 INFO:journalctl@ceph.mds.l.smithi183.stdout:Mar 26 08:30:27 smithi183 podman[65909]: 2024-03-26 08:30:27.616897939 +0000 UTC m=+0.127338129 container create 476ecfbdfdd3cf3bc17c0e85e0cf2d9c2ae93115d8b38302f2785d6a6608b57f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-l, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T08:30:27.881 INFO:journalctl@ceph.mds.l.smithi183.stdout:Mar 26 08:30:27 smithi183 podman[65909]: 2024-03-26 08:30:27.522621276 +0000 UTC m=+0.033061441 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:30:28.209 INFO:journalctl@ceph.mds.l.smithi183.stdout:Mar 26 08:30:27 smithi183 podman[65909]: 2024-03-26 08:30:27.88731017 +0000 UTC m=+0.397750393 container init 476ecfbdfdd3cf3bc17c0e85e0cf2d9c2ae93115d8b38302f2785d6a6608b57f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-l, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_BRANCH=HEAD) 2024-03-26T08:30:28.209 INFO:journalctl@ceph.mds.l.smithi183.stdout:Mar 26 08:30:27 smithi183 podman[65909]: 2024-03-26 08:30:27.897780839 +0000 UTC m=+0.408221017 container start 476ecfbdfdd3cf3bc17c0e85e0cf2d9c2ae93115d8b38302f2785d6a6608b57f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-l, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T08:30:28.209 INFO:journalctl@ceph.mds.l.smithi183.stdout:Mar 26 08:30:27 smithi183 ceph-mds[65928]: -- 172.21.15.183:0/3963093403 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bf05999a40 con 0x55bf0596d000 2024-03-26T08:30:28.209 INFO:journalctl@ceph.mds.l.smithi183.stdout:Mar 26 08:30:27 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-l[65924]: starting mds.l at 2024-03-26T08:30:28.209 INFO:journalctl@ceph.mds.l.smithi183.stdout:Mar 26 08:30:27 smithi183 bash[65909]: 476ecfbdfdd3cf3bc17c0e85e0cf2d9c2ae93115d8b38302f2785d6a6608b57f 2024-03-26T08:30:28.209 INFO:journalctl@ceph.mds.l.smithi183.stdout:Mar 26 08:30:28 smithi183 systemd[1]: Started Ceph mds.l for 88759420-eb49-11ee-95d0-87774f69a715. 2024-03-26T08:30:28.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:28 smithi183 ceph-mon[35818]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:28.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:28 smithi183 ceph-mon[35818]: mds.? [v2:172.21.15.183:6832/3522741573,v1:172.21.15.183:6833/3522741573] up:boot 2024-03-26T08:30:28.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:28 smithi183 ceph-mon[35818]: fsmap 1 up:standby 2024-03-26T08:30:28.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:28 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T08:30:28.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:28 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:28.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:28 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:28.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:28 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:28.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:28 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:30:28.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:28 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:30:28.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:28 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:28.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:28 smithi183 ceph-mon[35818]: Deploying daemon mds.k on smithi052 2024-03-26T08:30:28.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:28 smithi052 ceph-mon[35071]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:28.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:28 smithi052 ceph-mon[35071]: mds.? [v2:172.21.15.183:6832/3522741573,v1:172.21.15.183:6833/3522741573] up:boot 2024-03-26T08:30:28.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:28 smithi052 ceph-mon[35071]: fsmap 1 up:standby 2024-03-26T08:30:28.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:28 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T08:30:28.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:28 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:28.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:28 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:28.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:28 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:28.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:28 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:30:28.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:28 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:30:28.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:28 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:28.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:28 smithi052 ceph-mon[35071]: Deploying daemon mds.k on smithi052 2024-03-26T08:30:28.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:28 smithi043 ceph-mon[31929]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:28.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:28 smithi043 ceph-mon[31929]: mds.? [v2:172.21.15.183:6832/3522741573,v1:172.21.15.183:6833/3522741573] up:boot 2024-03-26T08:30:28.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:28 smithi043 ceph-mon[31929]: fsmap 1 up:standby 2024-03-26T08:30:28.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:28 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T08:30:28.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:28 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:28.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:28 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:28.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:28 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:28.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:28 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:30:28.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:28 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:30:28.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:28 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:28.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:28 smithi043 ceph-mon[31929]: Deploying daemon mds.k on smithi052 2024-03-26T08:30:28.891 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:30:29.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.130+0000 7f3317200640 1 -- 172.21.15.43:0/1335254875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33181036f0 msgr2=0x7f3318103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:29.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.130+0000 7f3317200640 1 --2- 172.21.15.43:0/1335254875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33181036f0 0x7f3318103b10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3308009ca0 tx=0x7f330802c100 comp rx=0 tx=0).stop 2024-03-26T08:30:29.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.130+0000 7f3317200640 1 -- 172.21.15.43:0/1335254875 shutdown_connections 2024-03-26T08:30:29.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.130+0000 7f3317200640 1 --2- 172.21.15.43:0/1335254875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33181052a0 0x7f331810bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:29.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.130+0000 7f3317200640 1 --2- 172.21.15.43:0/1335254875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33181048e0 0x7f3318104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:29.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.130+0000 7f3317200640 1 --2- 172.21.15.43:0/1335254875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33181036f0 0x7f3318103b10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:29.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.130+0000 7f3317200640 1 -- 172.21.15.43:0/1335254875 >> 172.21.15.43:0/1335254875 conn(0x7f33180fec80 msgr2=0x7f33181010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:29.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.131+0000 7f3317200640 1 -- 172.21.15.43:0/1335254875 shutdown_connections 2024-03-26T08:30:29.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.131+0000 7f3317200640 1 -- 172.21.15.43:0/1335254875 wait complete. 2024-03-26T08:30:29.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.132+0000 7f3317200640 1 Processor -- start 2024-03-26T08:30:29.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.132+0000 7f3317200640 1 -- start start 2024-03-26T08:30:29.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.132+0000 7f3317200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33181036f0 0x7f3318199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:29.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.132+0000 7f3317200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33181048e0 0x7f3318199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:29.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.132+0000 7f3317200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33181052a0 0x7f33181a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:29.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.132+0000 7f3317200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331810f1f0 con 0x7f33181052a0 2024-03-26T08:30:29.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.132+0000 7f3317200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331810f070 con 0x7f33181048e0 2024-03-26T08:30:29.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.132+0000 7f3317200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331810f370 con 0x7f33181036f0 2024-03-26T08:30:29.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.132+0000 7f3316800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33181052a0 0x7f33181a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:29.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.132+0000 7f3316800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33181052a0 0x7f33181a0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33226/0 (socket says 172.21.15.43:33226) 2024-03-26T08:30:29.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.132+0000 7f3316800640 1 -- 172.21.15.43:0/3656700768 learned_addr learned my addr 172.21.15.43:0/3656700768 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:30:29.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.133+0000 7f3315400640 1 --2- 172.21.15.43:0/3656700768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33181048e0 0x7f3318199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:29.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.133+0000 7f3315e00640 1 --2- 172.21.15.43:0/3656700768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33181036f0 0x7f3318199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:29.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.133+0000 7f3316800640 1 -- 172.21.15.43:0/3656700768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33181036f0 msgr2=0x7f3318199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:29.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.133+0000 7f3316800640 1 --2- 172.21.15.43:0/3656700768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33181036f0 0x7f3318199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:29.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.133+0000 7f3316800640 1 -- 172.21.15.43:0/3656700768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33181048e0 msgr2=0x7f3318199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:29.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.133+0000 7f3316800640 1 --2- 172.21.15.43:0/3656700768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33181048e0 0x7f3318199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:29.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.133+0000 7f3316800640 1 -- 172.21.15.43:0/3656700768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33181a1540 con 0x7f33181052a0 2024-03-26T08:30:29.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.133+0000 7f3315400640 1 --2- 172.21.15.43:0/3656700768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33181048e0 0x7f3318199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:30:29.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.133+0000 7f3316800640 1 --2- 172.21.15.43:0/3656700768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33181052a0 0x7f33181a0ed0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f331000d750 tx=0x7f331000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:29.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.133+0000 7f330ea00640 1 -- 172.21.15.43:0/3656700768 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3310017070 con 0x7f33181052a0 2024-03-26T08:30:29.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.133+0000 7f3317200640 1 -- 172.21.15.43:0/3656700768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33181a1830 con 0x7f33181052a0 2024-03-26T08:30:29.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.133+0000 7f330ea00640 1 -- 172.21.15.43:0/3656700768 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f33100054d0 con 0x7f33181052a0 2024-03-26T08:30:29.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.133+0000 7f330ea00640 1 -- 172.21.15.43:0/3656700768 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3310004c60 con 0x7f33181052a0 2024-03-26T08:30:29.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.134+0000 7f3317200640 1 -- 172.21.15.43:0/3656700768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33181a1d30 con 0x7f33181052a0 2024-03-26T08:30:29.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.135+0000 7f3317200640 1 -- 172.21.15.43:0/3656700768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f331810aae0 con 0x7f33181052a0 2024-03-26T08:30:29.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.137+0000 7f330ea00640 1 -- 172.21.15.43:0/3656700768 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f3310004e00 con 0x7f33181052a0 2024-03-26T08:30:29.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.138+0000 7f330ea00640 1 --2- 172.21.15.43:0/3656700768 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f32fc0a2800 0x7f32fc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:29.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.138+0000 7f330ea00640 1 -- 172.21.15.43:0/3656700768 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f33100ce0e0 con 0x7f33181052a0 2024-03-26T08:30:29.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.138+0000 7f3315e00640 1 --2- 172.21.15.43:0/3656700768 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f32fc0a2800 0x7f32fc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:29.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.139+0000 7f3315e00640 1 --2- 172.21.15.43:0/3656700768 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f32fc0a2800 0x7f32fc0a4cc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f330802c710 tx=0x7f3308000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:29.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.142+0000 7f330ea00640 1 -- 172.21.15.43:0/3656700768 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f331009b580 con 0x7f33181052a0 2024-03-26T08:30:29.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.386+0000 7f3317200640 1 -- 172.21.15.43:0/3656700768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33180622a0 con 0x7f33181052a0 2024-03-26T08:30:29.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.387+0000 7f330ea00640 1 -- 172.21.15.43:0/3656700768 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f331009ae80 con 0x7f33181052a0 2024-03-26T08:30:29.381 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:30:29.381 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-26T08:30:29.381 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 2 2024-03-26T08:30:29.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.390+0000 7f3317200640 1 -- 172.21.15.43:0/3656700768 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f32fc0a2800 msgr2=0x7f32fc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:29.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.390+0000 7f3317200640 1 --2- 172.21.15.43:0/3656700768 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f32fc0a2800 0x7f32fc0a4cc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f330802c710 tx=0x7f3308000f40 comp rx=0 tx=0).stop 2024-03-26T08:30:29.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.390+0000 7f3317200640 1 -- 172.21.15.43:0/3656700768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33181052a0 msgr2=0x7f33181a0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:29.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.390+0000 7f3317200640 1 --2- 172.21.15.43:0/3656700768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33181052a0 0x7f33181a0ed0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f331000d750 tx=0x7f331000dc10 comp rx=0 tx=0).stop 2024-03-26T08:30:29.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.391+0000 7f3317200640 1 -- 172.21.15.43:0/3656700768 shutdown_connections 2024-03-26T08:30:29.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.391+0000 7f3317200640 1 --2- 172.21.15.43:0/3656700768 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f32fc0a2800 0x7f32fc0a4cc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:29.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.391+0000 7f3317200640 1 --2- 172.21.15.43:0/3656700768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33181052a0 0x7f33181a0ed0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:29.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.391+0000 7f3317200640 1 --2- 172.21.15.43:0/3656700768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33181048e0 0x7f3318199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:29.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.391+0000 7f3317200640 1 --2- 172.21.15.43:0/3656700768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33181036f0 0x7f3318199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:29.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.391+0000 7f3317200640 1 -- 172.21.15.43:0/3656700768 >> 172.21.15.43:0/3656700768 conn(0x7f33180fec80 msgr2=0x7f3318107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:29.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.391+0000 7f3317200640 1 -- 172.21.15.43:0/3656700768 shutdown_connections 2024-03-26T08:30:29.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:29.391+0000 7f3317200640 1 -- 172.21.15.43:0/3656700768 wait complete. 2024-03-26T08:30:29.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:29 smithi043 ceph-mon[31929]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:29.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:29 smithi183 ceph-mon[35818]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:29.552 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:29 smithi052 ceph-mon[35071]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:30.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:30 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3656700768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:30.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:30 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3656700768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:30.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:30 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3656700768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:31.403 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:30:31.474 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:31 smithi052 ceph-mon[35071]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:31.500 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:31 smithi043 ceph-mon[31929]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:31.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:31 smithi183 ceph-mon[35818]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:31.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.649+0000 7f59e8400640 1 -- 172.21.15.43:0/3586592237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59e0104a70 msgr2=0x7f59e0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:31.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.649+0000 7f59e8400640 1 --2- 172.21.15.43:0/3586592237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59e0104a70 0x7f59e0106e60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009900 tx=0x7f59cc02bc40 comp rx=0 tx=0).stop 2024-03-26T08:30:31.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.649+0000 7f59e8400640 1 -- 172.21.15.43:0/3586592237 shutdown_connections 2024-03-26T08:30:31.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.649+0000 7f59e8400640 1 --2- 172.21.15.43:0/3586592237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59e01073a0 0x7f59e01097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:31.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.649+0000 7f59e8400640 1 --2- 172.21.15.43:0/3586592237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59e0104a70 0x7f59e0106e60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:31.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.649+0000 7f59e8400640 1 --2- 172.21.15.43:0/3586592237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59e0102140 0x7f59e0104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:31.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.649+0000 7f59e8400640 1 -- 172.21.15.43:0/3586592237 >> 172.21.15.43:0/3586592237 conn(0x7f59e00fbb30 msgr2=0x7f59e00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:31.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.649+0000 7f59e8400640 1 -- 172.21.15.43:0/3586592237 shutdown_connections 2024-03-26T08:30:31.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.649+0000 7f59e8400640 1 -- 172.21.15.43:0/3586592237 wait complete. 2024-03-26T08:30:31.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.650+0000 7f59e8400640 1 Processor -- start 2024-03-26T08:30:31.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.651+0000 7f59e8400640 1 -- start start 2024-03-26T08:30:31.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.651+0000 7f59e8400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59e0102140 0x7f59e0199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:31.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.651+0000 7f59e8400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59e0104a70 0x7f59e019a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:31.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.651+0000 7f59e8400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59e01073a0 0x7f59e01a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:31.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.651+0000 7f59e8400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e010d250 con 0x7f59e0102140 2024-03-26T08:30:31.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.651+0000 7f59e8400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e010d0d0 con 0x7f59e0104a70 2024-03-26T08:30:31.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.651+0000 7f59e8400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e010d3d0 con 0x7f59e01073a0 2024-03-26T08:30:31.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.651+0000 7f59e5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59e0102140 0x7f59e0199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:31.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.651+0000 7f59e5a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59e0102140 0x7f59e0199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33248/0 (socket says 172.21.15.43:33248) 2024-03-26T08:30:31.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.651+0000 7f59e5a00640 1 -- 172.21.15.43:0/616237703 learned_addr learned my addr 172.21.15.43:0/616237703 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:30:31.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.651+0000 7f59e5000640 1 --2- 172.21.15.43:0/616237703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59e0104a70 0x7f59e019a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:31.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.651+0000 7f59e6400640 1 --2- 172.21.15.43:0/616237703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59e01073a0 0x7f59e01a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:31.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.652+0000 7f59e5a00640 1 -- 172.21.15.43:0/616237703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59e01073a0 msgr2=0x7f59e01a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:31.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.652+0000 7f59e5a00640 1 --2- 172.21.15.43:0/616237703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59e01073a0 0x7f59e01a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:31.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.652+0000 7f59e5a00640 1 -- 172.21.15.43:0/616237703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59e0104a70 msgr2=0x7f59e019a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:31.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.652+0000 7f59e5a00640 1 --2- 172.21.15.43:0/616237703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59e0104a70 0x7f59e019a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:31.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.652+0000 7f59e5a00640 1 -- 172.21.15.43:0/616237703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e01a1790 con 0x7f59e0102140 2024-03-26T08:30:31.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.652+0000 7f59e6400640 1 --2- 172.21.15.43:0/616237703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59e01073a0 0x7f59e01a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:30:31.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.652+0000 7f59e5a00640 1 --2- 172.21.15.43:0/616237703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59e0102140 0x7f59e0199b70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f59d400d770 tx=0x7f59d400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:31.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.652+0000 7f59de000640 1 -- 172.21.15.43:0/616237703 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4017070 con 0x7f59e0102140 2024-03-26T08:30:31.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.652+0000 7f59de000640 1 -- 172.21.15.43:0/616237703 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f59d40054d0 con 0x7f59e0102140 2024-03-26T08:30:31.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.652+0000 7f59de000640 1 -- 172.21.15.43:0/616237703 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4004cf0 con 0x7f59e0102140 2024-03-26T08:30:31.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.652+0000 7f59e8400640 1 -- 172.21.15.43:0/616237703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e01a1a80 con 0x7f59e0102140 2024-03-26T08:30:31.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.652+0000 7f59e8400640 1 -- 172.21.15.43:0/616237703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e01a1fa0 con 0x7f59e0102140 2024-03-26T08:30:31.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.653+0000 7f59e8400640 1 -- 172.21.15.43:0/616237703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e0108ef0 con 0x7f59e0102140 2024-03-26T08:30:31.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.656+0000 7f59de000640 1 -- 172.21.15.43:0/616237703 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f59d4004e90 con 0x7f59e0102140 2024-03-26T08:30:31.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.657+0000 7f59de000640 1 --2- 172.21.15.43:0/616237703 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f59c80a2880 0x7f59c80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:31.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.657+0000 7f59de000640 1 -- 172.21.15.43:0/616237703 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f59d40cdfe0 con 0x7f59e0102140 2024-03-26T08:30:31.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.657+0000 7f59e5000640 1 --2- 172.21.15.43:0/616237703 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f59c80a2880 0x7f59c80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:31.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.658+0000 7f59e5000640 1 --2- 172.21.15.43:0/616237703 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f59c80a2880 0x7f59c80a4d40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f59cc02c250 tx=0x7f59cc02c7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:31.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.661+0000 7f59de000640 1 -- 172.21.15.43:0/616237703 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f59d409b4e0 con 0x7f59e0102140 2024-03-26T08:30:31.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.909+0000 7f59e8400640 1 -- 172.21.15.43:0/616237703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59e00622d0 con 0x7f59e0102140 2024-03-26T08:30:31.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.910+0000 7f59de000640 1 -- 172.21.15.43:0/616237703 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f59d409ade0 con 0x7f59e0102140 2024-03-26T08:30:31.903 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:30:31.903 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-26T08:30:31.904 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 2 2024-03-26T08:30:31.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.913+0000 7f59e8400640 1 -- 172.21.15.43:0/616237703 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f59c80a2880 msgr2=0x7f59c80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:31.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.913+0000 7f59e8400640 1 --2- 172.21.15.43:0/616237703 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f59c80a2880 0x7f59c80a4d40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f59cc02c250 tx=0x7f59cc02c7f0 comp rx=0 tx=0).stop 2024-03-26T08:30:31.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.913+0000 7f59e8400640 1 -- 172.21.15.43:0/616237703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59e0102140 msgr2=0x7f59e0199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:31.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.913+0000 7f59e8400640 1 --2- 172.21.15.43:0/616237703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59e0102140 0x7f59e0199b70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f59d400d770 tx=0x7f59d400dc30 comp rx=0 tx=0).stop 2024-03-26T08:30:31.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.913+0000 7f59e8400640 1 -- 172.21.15.43:0/616237703 shutdown_connections 2024-03-26T08:30:31.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.913+0000 7f59e8400640 1 --2- 172.21.15.43:0/616237703 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f59c80a2880 0x7f59c80a4d40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:31.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.913+0000 7f59e8400640 1 --2- 172.21.15.43:0/616237703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59e01073a0 0x7f59e01a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:31.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.913+0000 7f59e8400640 1 --2- 172.21.15.43:0/616237703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59e0104a70 0x7f59e019a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:31.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.913+0000 7f59e8400640 1 --2- 172.21.15.43:0/616237703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59e0102140 0x7f59e0199b70 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:31.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.913+0000 7f59e8400640 1 -- 172.21.15.43:0/616237703 >> 172.21.15.43:0/616237703 conn(0x7f59e00fbb30 msgr2=0x7f59e01019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:31.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.913+0000 7f59e8400640 1 -- 172.21.15.43:0/616237703 shutdown_connections 2024-03-26T08:30:31.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:31.913+0000 7f59e8400640 1 -- 172.21.15.43:0/616237703 wait complete. 2024-03-26T08:30:32.074 INFO:journalctl@ceph.mds.k.smithi052.stdout:Mar 26 08:30:31 smithi052 podman[64657]: 2024-03-26 08:30:31.859645441 +0000 UTC m=+0.141444951 container create b0d46f3ecabb0a95b540570e0d6117794c731f581c6676379e7b7ecc6a9bc050 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-k, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True) 2024-03-26T08:30:32.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:32 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/616237703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:32.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:32 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/616237703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:32.573 INFO:journalctl@ceph.mds.k.smithi052.stdout:Mar 26 08:30:32 smithi052 podman[64657]: 2024-03-26 08:30:32.305635009 +0000 UTC m=+0.587434593 container init b0d46f3ecabb0a95b540570e0d6117794c731f581c6676379e7b7ecc6a9bc050 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-26T08:30:32.573 INFO:journalctl@ceph.mds.k.smithi052.stdout:Mar 26 08:30:32 smithi052 podman[64657]: 2024-03-26 08:30:32.314616412 +0000 UTC m=+0.596415959 container start b0d46f3ecabb0a95b540570e0d6117794c731f581c6676379e7b7ecc6a9bc050 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-26T08:30:32.573 INFO:journalctl@ceph.mds.k.smithi052.stdout:Mar 26 08:30:32 smithi052 ceph-mds[64683]: -- 172.21.15.52:0/661311401 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5587e192c4e0 con 0x5587e1902c00 2024-03-26T08:30:32.573 INFO:journalctl@ceph.mds.k.smithi052.stdout:Mar 26 08:30:32 smithi052 ceph-mds[64683]: -- 172.21.15.52:0/661311401 <== mon.2 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5587e192fa40 con 0x5587e1902c00 2024-03-26T08:30:32.573 INFO:journalctl@ceph.mds.k.smithi052.stdout:Mar 26 08:30:32 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-k[64679]: starting mds.k at 2024-03-26T08:30:32.573 INFO:journalctl@ceph.mds.k.smithi052.stdout:Mar 26 08:30:32 smithi052 bash[64657]: b0d46f3ecabb0a95b540570e0d6117794c731f581c6676379e7b7ecc6a9bc050 2024-03-26T08:30:32.573 INFO:journalctl@ceph.mds.k.smithi052.stdout:Mar 26 08:30:32 smithi052 systemd[1]: Started Ceph mds.k for 88759420-eb49-11ee-95d0-87774f69a715. 2024-03-26T08:30:32.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:32 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/616237703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:33.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:33 smithi183 ceph-mon[35818]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:33.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:33 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:33.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:33 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:33.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:33 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:33.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:33 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:30:33.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:33 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:30:33.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:33 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:33.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:33 smithi183 ceph-mon[35818]: Deploying daemon mds.j on smithi043 2024-03-26T08:30:33.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:33 smithi183 ceph-mon[35818]: mds.? [v2:172.21.15.52:6832/1108327179,v1:172.21.15.52:6833/1108327179] up:boot 2024-03-26T08:30:33.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:33 smithi183 ceph-mon[35818]: fsmap 2 up:standby 2024-03-26T08:30:33.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:33 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T08:30:33.810 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:33 smithi043 ceph-mon[31929]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:33.810 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:33 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:33.810 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:33 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:33.810 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:33 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:33.810 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:33 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:30:33.810 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:33 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:30:33.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:33 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:33.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:33 smithi043 ceph-mon[31929]: Deploying daemon mds.j on smithi043 2024-03-26T08:30:33.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:33 smithi043 ceph-mon[31929]: mds.? [v2:172.21.15.52:6832/1108327179,v1:172.21.15.52:6833/1108327179] up:boot 2024-03-26T08:30:33.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:33 smithi043 ceph-mon[31929]: fsmap 2 up:standby 2024-03-26T08:30:33.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:33 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T08:30:33.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:33 smithi052 ceph-mon[35071]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:33.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:33 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:33.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:33 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:33.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:33 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:33.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:33 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:30:33.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:33 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:30:33.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:33 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:33.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:33 smithi052 ceph-mon[35071]: Deploying daemon mds.j on smithi043 2024-03-26T08:30:33.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:33 smithi052 ceph-mon[35071]: mds.? [v2:172.21.15.52:6832/1108327179,v1:172.21.15.52:6833/1108327179] up:boot 2024-03-26T08:30:33.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:33 smithi052 ceph-mon[35071]: fsmap 2 up:standby 2024-03-26T08:30:33.824 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:33 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T08:30:33.926 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:30:34.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.142+0000 7f1a4cc00640 1 -- 172.21.15.43:0/275476277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a48077c40 msgr2=0x7f1a480784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:34.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.142+0000 7f1a4cc00640 1 --2- 172.21.15.43:0/275476277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a48077c40 0x7f1a480784e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009ca0 tx=0x7f1a4002c100 comp rx=0 tx=0).stop 2024-03-26T08:30:34.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.142+0000 7f1a4cc00640 1 -- 172.21.15.43:0/275476277 shutdown_connections 2024-03-26T08:30:34.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.142+0000 7f1a4cc00640 1 --2- 172.21.15.43:0/275476277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a48077c40 0x7f1a480784e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:34.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.142+0000 7f1a4cc00640 1 --2- 172.21.15.43:0/275476277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a480772a0 0x7f1a48077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:34.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.142+0000 7f1a4cc00640 1 --2- 172.21.15.43:0/275476277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a48076060 0x7f1a48076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:34.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.142+0000 7f1a4cc00640 1 -- 172.21.15.43:0/275476277 >> 172.21.15.43:0/275476277 conn(0x7f1a480fee50 msgr2=0x7f1a48101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:34.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.142+0000 7f1a4cc00640 1 -- 172.21.15.43:0/275476277 shutdown_connections 2024-03-26T08:30:34.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.142+0000 7f1a4cc00640 1 -- 172.21.15.43:0/275476277 wait complete. 2024-03-26T08:30:34.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.143+0000 7f1a4cc00640 1 Processor -- start 2024-03-26T08:30:34.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.144+0000 7f1a4cc00640 1 -- start start 2024-03-26T08:30:34.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.144+0000 7f1a4cc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a48076060 0x7f1a4819de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:34.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.144+0000 7f1a4cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a480772a0 0x7f1a4819e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:34.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.144+0000 7f1a4cc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a48077c40 0x7f1a481a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:34.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.144+0000 7f1a4cc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a48113820 con 0x7f1a480772a0 2024-03-26T08:30:34.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.144+0000 7f1a4cc00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a481136a0 con 0x7f1a48077c40 2024-03-26T08:30:34.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.144+0000 7f1a4cc00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a481139a0 con 0x7f1a48076060 2024-03-26T08:30:34.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.144+0000 7f1a45400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a480772a0 0x7f1a4819e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:34.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.144+0000 7f1a45400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a480772a0 0x7f1a4819e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34804/0 (socket says 172.21.15.43:34804) 2024-03-26T08:30:34.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.144+0000 7f1a45400640 1 -- 172.21.15.43:0/1134864753 learned_addr learned my addr 172.21.15.43:0/1134864753 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:30:34.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.144+0000 7f1a45e00640 1 --2- 172.21.15.43:0/1134864753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a48076060 0x7f1a4819de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:34.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.144+0000 7f1a46800640 1 --2- 172.21.15.43:0/1134864753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a48077c40 0x7f1a481a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:34.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.145+0000 7f1a45400640 1 -- 172.21.15.43:0/1134864753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a48076060 msgr2=0x7f1a4819de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:34.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.145+0000 7f1a45400640 1 --2- 172.21.15.43:0/1134864753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a48076060 0x7f1a4819de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:34.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.145+0000 7f1a45400640 1 -- 172.21.15.43:0/1134864753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a48077c40 msgr2=0x7f1a481a53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:34.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.145+0000 7f1a45400640 1 --2- 172.21.15.43:0/1134864753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a48077c40 0x7f1a481a53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:34.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.145+0000 7f1a45400640 1 -- 172.21.15.43:0/1134864753 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a481a5a60 con 0x7f1a480772a0 2024-03-26T08:30:34.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.145+0000 7f1a45400640 1 --2- 172.21.15.43:0/1134864753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a480772a0 0x7f1a4819e380 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400d770 tx=0x7f1a3400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:34.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.145+0000 7f1a3ea00640 1 -- 172.21.15.43:0/1134864753 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a34017070 con 0x7f1a480772a0 2024-03-26T08:30:34.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.145+0000 7f1a3ea00640 1 -- 172.21.15.43:0/1134864753 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1a340054d0 con 0x7f1a480772a0 2024-03-26T08:30:34.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.145+0000 7f1a4cc00640 1 -- 172.21.15.43:0/1134864753 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a481a5d50 con 0x7f1a480772a0 2024-03-26T08:30:34.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.145+0000 7f1a3ea00640 1 -- 172.21.15.43:0/1134864753 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a34004cf0 con 0x7f1a480772a0 2024-03-26T08:30:34.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.145+0000 7f1a4cc00640 1 -- 172.21.15.43:0/1134864753 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a481a61f0 con 0x7f1a480772a0 2024-03-26T08:30:34.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.147+0000 7f1a4cc00640 1 -- 172.21.15.43:0/1134864753 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a481103c0 con 0x7f1a480772a0 2024-03-26T08:30:34.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.150+0000 7f1a3ea00640 1 -- 172.21.15.43:0/1134864753 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1a34004e90 con 0x7f1a480772a0 2024-03-26T08:30:34.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.151+0000 7f1a3ea00640 1 --2- 172.21.15.43:0/1134864753 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1a2c0a2800 0x7f1a2c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:34.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.151+0000 7f1a3ea00640 1 -- 172.21.15.43:0/1134864753 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1a340ce170 con 0x7f1a480772a0 2024-03-26T08:30:34.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.151+0000 7f1a45e00640 1 --2- 172.21.15.43:0/1134864753 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1a2c0a2800 0x7f1a2c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:34.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.152+0000 7f1a45e00640 1 --2- 172.21.15.43:0/1134864753 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1a2c0a2800 0x7f1a2c0a4cc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1a38005e10 tx=0x7f1a38005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:34.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.155+0000 7f1a3ea00640 1 -- 172.21.15.43:0/1134864753 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1a3409b610 con 0x7f1a480772a0 2024-03-26T08:30:34.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.403+0000 7f1a4cc00640 1 -- 172.21.15.43:0/1134864753 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a48062340 con 0x7f1a480772a0 2024-03-26T08:30:34.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.404+0000 7f1a3ea00640 1 -- 172.21.15.43:0/1134864753 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7f1a3409af10 con 0x7f1a480772a0 2024-03-26T08:30:34.397 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:30:34.397 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34448,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/1108327179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108327179},{"type":"v1","addr":"172.21.15.52:6833","nonce":1108327179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-26T08:30:34.397 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 3 2024-03-26T08:30:34.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.407+0000 7f1a4cc00640 1 -- 172.21.15.43:0/1134864753 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1a2c0a2800 msgr2=0x7f1a2c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:34.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.407+0000 7f1a4cc00640 1 --2- 172.21.15.43:0/1134864753 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1a2c0a2800 0x7f1a2c0a4cc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1a38005e10 tx=0x7f1a38005da0 comp rx=0 tx=0).stop 2024-03-26T08:30:34.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.407+0000 7f1a4cc00640 1 -- 172.21.15.43:0/1134864753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a480772a0 msgr2=0x7f1a4819e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:34.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.407+0000 7f1a4cc00640 1 --2- 172.21.15.43:0/1134864753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a480772a0 0x7f1a4819e380 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400d770 tx=0x7f1a3400dc30 comp rx=0 tx=0).stop 2024-03-26T08:30:34.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.407+0000 7f1a4cc00640 1 -- 172.21.15.43:0/1134864753 shutdown_connections 2024-03-26T08:30:34.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.407+0000 7f1a4cc00640 1 --2- 172.21.15.43:0/1134864753 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1a2c0a2800 0x7f1a2c0a4cc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:34.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.407+0000 7f1a4cc00640 1 --2- 172.21.15.43:0/1134864753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a48077c40 0x7f1a481a53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:34.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.407+0000 7f1a4cc00640 1 --2- 172.21.15.43:0/1134864753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a480772a0 0x7f1a4819e380 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:34.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.407+0000 7f1a4cc00640 1 --2- 172.21.15.43:0/1134864753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a48076060 0x7f1a4819de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:34.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.407+0000 7f1a4cc00640 1 -- 172.21.15.43:0/1134864753 >> 172.21.15.43:0/1134864753 conn(0x7f1a480fee50 msgr2=0x7f1a48100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:34.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.407+0000 7f1a4cc00640 1 -- 172.21.15.43:0/1134864753 shutdown_connections 2024-03-26T08:30:34.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:34.407+0000 7f1a4cc00640 1 -- 172.21.15.43:0/1134864753 wait complete. 2024-03-26T08:30:34.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:34 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1134864753' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:34.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:34 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1134864753' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:35.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:34 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1134864753' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:35.786 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:35 smithi043 ceph-mon[31929]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:35.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:35 smithi052 ceph-mon[35071]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:35.912 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:35 smithi183 ceph-mon[35818]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:36.153 INFO:journalctl@ceph.mds.j.smithi043.stdout:Mar 26 08:30:35 smithi043 systemd[1]: Starting Ceph mds.j for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:30:36.419 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:30:36.442 INFO:journalctl@ceph.mds.j.smithi043.stdout:Mar 26 08:30:36 smithi043 podman[69550]: 2024-03-26 08:30:36.07368957 +0000 UTC m=+0.035065563 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:30:36.442 INFO:journalctl@ceph.mds.j.smithi043.stdout:Mar 26 08:30:36 smithi043 podman[69550]: 2024-03-26 08:30:36.273252063 +0000 UTC m=+0.234628431 container create de9f0ba937827eb60dd950c659f1178a3515948f928b001a90c954863d24f031 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-j, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319) 2024-03-26T08:30:36.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.651+0000 7ff799600640 1 -- 172.21.15.43:0/2292075176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7941071d0 msgr2=0x7ff794109620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:36.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.651+0000 7ff799600640 1 --2- 172.21.15.43:0/2292075176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7941071d0 0x7ff794109620 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009ca0 tx=0x7ff78c02c0f0 comp rx=0 tx=0).stop 2024-03-26T08:30:36.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.651+0000 7ff799600640 1 -- 172.21.15.43:0/2292075176 shutdown_connections 2024-03-26T08:30:36.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.651+0000 7ff799600640 1 --2- 172.21.15.43:0/2292075176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7941071d0 0x7ff794109620 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:36.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.651+0000 7ff799600640 1 --2- 172.21.15.43:0/2292075176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7941048a0 0x7ff794106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:36.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.651+0000 7ff799600640 1 --2- 172.21.15.43:0/2292075176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff794101f70 0x7ff794104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:36.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.651+0000 7ff799600640 1 -- 172.21.15.43:0/2292075176 >> 172.21.15.43:0/2292075176 conn(0x7ff7940fb980 msgr2=0x7ff7940fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:36.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.651+0000 7ff799600640 1 -- 172.21.15.43:0/2292075176 shutdown_connections 2024-03-26T08:30:36.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.651+0000 7ff799600640 1 -- 172.21.15.43:0/2292075176 wait complete. 2024-03-26T08:30:36.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.652+0000 7ff799600640 1 Processor -- start 2024-03-26T08:30:36.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.652+0000 7ff799600640 1 -- start start 2024-03-26T08:30:36.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.652+0000 7ff799600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff794101f70 0x7ff7941018e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:36.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.652+0000 7ff799600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7941048a0 0x7ff7940fff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:36.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.652+0000 7ff799600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7941071d0 0x7ff794100460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:36.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.652+0000 7ff799600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79410d030 con 0x7ff7941071d0 2024-03-26T08:30:36.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.652+0000 7ff799600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79410ceb0 con 0x7ff7941048a0 2024-03-26T08:30:36.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.652+0000 7ff799600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79410d1b0 con 0x7ff794101f70 2024-03-26T08:30:36.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.653+0000 7ff798c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7941071d0 0x7ff794100460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:36.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.653+0000 7ff798c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7941071d0 0x7ff794100460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34818/0 (socket says 172.21.15.43:34818) 2024-03-26T08:30:36.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.653+0000 7ff798c00640 1 -- 172.21.15.43:0/1122355829 learned_addr learned my addr 172.21.15.43:0/1122355829 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:30:36.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.653+0000 7ff793e00640 1 --2- 172.21.15.43:0/1122355829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff794101f70 0x7ff7941018e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:36.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.653+0000 7ff78be00640 1 --2- 172.21.15.43:0/1122355829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7941048a0 0x7ff7940fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:36.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.653+0000 7ff798c00640 1 -- 172.21.15.43:0/1122355829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff794101f70 msgr2=0x7ff7941018e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:36.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.653+0000 7ff798c00640 1 --2- 172.21.15.43:0/1122355829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff794101f70 0x7ff7941018e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:36.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.653+0000 7ff798c00640 1 -- 172.21.15.43:0/1122355829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7941048a0 msgr2=0x7ff7940fff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:36.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.653+0000 7ff798c00640 1 --2- 172.21.15.43:0/1122355829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7941048a0 0x7ff7940fff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:36.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.653+0000 7ff798c00640 1 -- 172.21.15.43:0/1122355829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff794100cf0 con 0x7ff7941071d0 2024-03-26T08:30:36.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.654+0000 7ff798c00640 1 --2- 172.21.15.43:0/1122355829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7941071d0 0x7ff794100460 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff78c032380 tx=0x7ff78c032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:36.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.654+0000 7ff791600640 1 -- 172.21.15.43:0/1122355829 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c041020 con 0x7ff7941071d0 2024-03-26T08:30:36.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.654+0000 7ff791600640 1 -- 172.21.15.43:0/1122355829 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff78c003e20 con 0x7ff7941071d0 2024-03-26T08:30:36.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.654+0000 7ff791600640 1 -- 172.21.15.43:0/1122355829 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c039040 con 0x7ff7941071d0 2024-03-26T08:30:36.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.654+0000 7ff799600640 1 -- 172.21.15.43:0/1122355829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7941a1750 con 0x7ff7941071d0 2024-03-26T08:30:36.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.654+0000 7ff799600640 1 -- 172.21.15.43:0/1122355829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7941a1cf0 con 0x7ff7941071d0 2024-03-26T08:30:36.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.655+0000 7ff799600640 1 -- 172.21.15.43:0/1122355829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7940fd470 con 0x7ff7941071d0 2024-03-26T08:30:36.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.658+0000 7ff791600640 1 -- 172.21.15.43:0/1122355829 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff78c044020 con 0x7ff7941071d0 2024-03-26T08:30:36.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.659+0000 7ff791600640 1 --2- 172.21.15.43:0/1122355829 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff7780a2880 0x7ff7780a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:36.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.659+0000 7ff791600640 1 -- 172.21.15.43:0/1122355829 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff78c0f0c40 con 0x7ff7941071d0 2024-03-26T08:30:36.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.659+0000 7ff793e00640 1 --2- 172.21.15.43:0/1122355829 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff7780a2880 0x7ff7780a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:36.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.660+0000 7ff793e00640 1 --2- 172.21.15.43:0/1122355829 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff7780a2880 0x7ff7780a4d40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff784009b60 tx=0x7ff784005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.667+0000 7ff791600640 1 -- 172.21.15.43:0/1122355829 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff78c04b020 con 0x7ff7941071d0 2024-03-26T08:30:36.903 INFO:journalctl@ceph.mds.j.smithi043.stdout:Mar 26 08:30:36 smithi043 podman[69550]: 2024-03-26 08:30:36.61314219 +0000 UTC m=+0.574518211 container init de9f0ba937827eb60dd950c659f1178a3515948f928b001a90c954863d24f031 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-j, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T08:30:36.903 INFO:journalctl@ceph.mds.j.smithi043.stdout:Mar 26 08:30:36 smithi043 podman[69550]: 2024-03-26 08:30:36.626818338 +0000 UTC m=+0.588194387 container start de9f0ba937827eb60dd950c659f1178a3515948f928b001a90c954863d24f031 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-j, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-26T08:30:36.903 INFO:journalctl@ceph.mds.j.smithi043.stdout:Mar 26 08:30:36 smithi043 ceph-mds[69588]: -- 172.21.15.43:0/2536381142 <== mon.0 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e5c88eda40 con 0x55e5c88c1000 2024-03-26T08:30:36.903 INFO:journalctl@ceph.mds.j.smithi043.stdout:Mar 26 08:30:36 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-j[69564]: starting mds.j at 2024-03-26T08:30:36.903 INFO:journalctl@ceph.mds.j.smithi043.stdout:Mar 26 08:30:36 smithi043 bash[69550]: de9f0ba937827eb60dd950c659f1178a3515948f928b001a90c954863d24f031 2024-03-26T08:30:36.903 INFO:journalctl@ceph.mds.j.smithi043.stdout:Mar 26 08:30:36 smithi043 systemd[1]: Started Ceph mds.j for 88759420-eb49-11ee-95d0-87774f69a715. 2024-03-26T08:30:36.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.929+0000 7ff799600640 1 -- 172.21.15.43:0/1122355829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff79402d070 con 0x7ff7941071d0 2024-03-26T08:30:36.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.930+0000 7ff791600640 1 -- 172.21.15.43:0/1122355829 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7ff78c0be0e0 con 0x7ff7941071d0 2024-03-26T08:30:36.923 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:30:36.924 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34448,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/1108327179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108327179},{"type":"v1","addr":"172.21.15.52:6833","nonce":1108327179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-26T08:30:36.924 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 3 2024-03-26T08:30:36.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.933+0000 7ff799600640 1 -- 172.21.15.43:0/1122355829 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff7780a2880 msgr2=0x7ff7780a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:36.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.933+0000 7ff799600640 1 --2- 172.21.15.43:0/1122355829 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff7780a2880 0x7ff7780a4d40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff784009b60 tx=0x7ff784005f90 comp rx=0 tx=0).stop 2024-03-26T08:30:36.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.933+0000 7ff799600640 1 -- 172.21.15.43:0/1122355829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7941071d0 msgr2=0x7ff794100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:36.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.933+0000 7ff799600640 1 --2- 172.21.15.43:0/1122355829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7941071d0 0x7ff794100460 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff78c032380 tx=0x7ff78c032820 comp rx=0 tx=0).stop 2024-03-26T08:30:36.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.933+0000 7ff799600640 1 -- 172.21.15.43:0/1122355829 shutdown_connections 2024-03-26T08:30:36.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.933+0000 7ff799600640 1 --2- 172.21.15.43:0/1122355829 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff7780a2880 0x7ff7780a4d40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:36.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.933+0000 7ff799600640 1 --2- 172.21.15.43:0/1122355829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7941071d0 0x7ff794100460 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:36.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.933+0000 7ff799600640 1 --2- 172.21.15.43:0/1122355829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7941048a0 0x7ff7940fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:36.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.933+0000 7ff799600640 1 --2- 172.21.15.43:0/1122355829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff794101f70 0x7ff7941018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:36.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.933+0000 7ff799600640 1 -- 172.21.15.43:0/1122355829 >> 172.21.15.43:0/1122355829 conn(0x7ff7940fb980 msgr2=0x7ff7941028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:36.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.933+0000 7ff799600640 1 -- 172.21.15.43:0/1122355829 shutdown_connections 2024-03-26T08:30:36.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:36.933+0000 7ff799600640 1 -- 172.21.15.43:0/1122355829 wait complete. 2024-03-26T08:30:38.049 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:37 smithi043 ceph-mon[31929]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:38.049 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:37 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:38.049 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:37 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:38.049 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:37 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:38.049 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:37 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:30:38.049 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:37 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:30:38.049 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:37 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:38.049 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:37 smithi043 ceph-mon[31929]: Deploying daemon mds.d on smithi043 2024-03-26T08:30:38.049 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:37 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1122355829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:38.049 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:37 smithi043 ceph-mon[31929]: mds.? [v2:172.21.15.43:6834/1452206807,v1:172.21.15.43:6835/1452206807] up:boot 2024-03-26T08:30:38.050 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:37 smithi043 ceph-mon[31929]: fsmap 3 up:standby 2024-03-26T08:30:38.050 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:37 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-26T08:30:38.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:37 smithi183 ceph-mon[35818]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:38.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:37 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:38.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:37 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:38.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:37 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:38.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:37 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:30:38.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:37 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:30:38.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:37 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:38.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:37 smithi183 ceph-mon[35818]: Deploying daemon mds.d on smithi043 2024-03-26T08:30:38.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:37 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1122355829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:38.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:37 smithi183 ceph-mon[35818]: mds.? [v2:172.21.15.43:6834/1452206807,v1:172.21.15.43:6835/1452206807] up:boot 2024-03-26T08:30:38.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:37 smithi183 ceph-mon[35818]: fsmap 3 up:standby 2024-03-26T08:30:38.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:37 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-26T08:30:38.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:37 smithi052 ceph-mon[35071]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:38.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:37 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:38.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:37 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:38.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:37 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:38.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:37 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:30:38.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:37 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:30:38.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:37 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:38.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:37 smithi052 ceph-mon[35071]: Deploying daemon mds.d on smithi043 2024-03-26T08:30:38.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:37 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1122355829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:38.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:37 smithi052 ceph-mon[35071]: mds.? [v2:172.21.15.43:6834/1452206807,v1:172.21.15.43:6835/1452206807] up:boot 2024-03-26T08:30:38.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:37 smithi052 ceph-mon[35071]: fsmap 3 up:standby 2024-03-26T08:30:38.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:37 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-26T08:30:38.946 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:30:39.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.203+0000 7f7299200640 1 -- 172.21.15.43:0/1187807117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72941028b0 msgr2=0x7f7294102d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:39.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.203+0000 7f7299200640 1 --2- 172.21.15.43:0/1187807117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72941028b0 0x7f7294102d30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7280009900 tx=0x7f728002bc30 comp rx=0 tx=0).stop 2024-03-26T08:30:39.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.203+0000 7f7299200640 1 -- 172.21.15.43:0/1187807117 shutdown_connections 2024-03-26T08:30:39.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.203+0000 7f7299200640 1 --2- 172.21.15.43:0/1187807117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7294103270 0x7f7294109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:39.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.203+0000 7f7299200640 1 --2- 172.21.15.43:0/1187807117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72941028b0 0x7f7294102d30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:39.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.203+0000 7f7299200640 1 --2- 172.21.15.43:0/1187807117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72941016c0 0x7f7294101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:39.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.203+0000 7f7299200640 1 -- 172.21.15.43:0/1187807117 >> 172.21.15.43:0/1187807117 conn(0x7f72940fcc50 msgr2=0x7f72940ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:39.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.203+0000 7f7299200640 1 -- 172.21.15.43:0/1187807117 shutdown_connections 2024-03-26T08:30:39.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.203+0000 7f7299200640 1 -- 172.21.15.43:0/1187807117 wait complete. 2024-03-26T08:30:39.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.204+0000 7f7299200640 1 Processor -- start 2024-03-26T08:30:39.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.205+0000 7f7299200640 1 -- start start 2024-03-26T08:30:39.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.205+0000 7f7299200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72941016c0 0x7f7294075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:39.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.205+0000 7f7299200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72941028b0 0x7f7294073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:39.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.205+0000 7f7299200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7294103270 0x7f7294073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:39.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.205+0000 7f7299200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729410d240 con 0x7f72941028b0 2024-03-26T08:30:39.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.205+0000 7f7299200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729410d0c0 con 0x7f72941016c0 2024-03-26T08:30:39.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.205+0000 7f7299200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729410d3c0 con 0x7f7294103270 2024-03-26T08:30:39.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.205+0000 7f7291400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72941028b0 0x7f7294073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:39.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.205+0000 7f7291400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72941028b0 0x7f7294073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34858/0 (socket says 172.21.15.43:34858) 2024-03-26T08:30:39.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.205+0000 7f7291400640 1 -- 172.21.15.43:0/3319774862 learned_addr learned my addr 172.21.15.43:0/3319774862 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:30:39.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.206+0000 7f7291e00640 1 --2- 172.21.15.43:0/3319774862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72941016c0 0x7f7294075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:39.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.206+0000 7f7292800640 1 --2- 172.21.15.43:0/3319774862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7294103270 0x7f7294073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:39.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.206+0000 7f7291400640 1 -- 172.21.15.43:0/3319774862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7294103270 msgr2=0x7f7294073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:39.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.206+0000 7f7291400640 1 --2- 172.21.15.43:0/3319774862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7294103270 0x7f7294073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:39.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.206+0000 7f7291400640 1 -- 172.21.15.43:0/3319774862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72941016c0 msgr2=0x7f7294075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:39.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.206+0000 7f7291400640 1 --2- 172.21.15.43:0/3319774862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72941016c0 0x7f7294075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:39.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.206+0000 7f7291400640 1 -- 172.21.15.43:0/3319774862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7294074860 con 0x7f72941028b0 2024-03-26T08:30:39.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.206+0000 7f7291400640 1 --2- 172.21.15.43:0/3319774862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72941028b0 0x7f7294073a90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7280000c00 tx=0x7f72800040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:39.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.207+0000 7f728aa00640 1 -- 172.21.15.43:0/3319774862 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7280043020 con 0x7f72941028b0 2024-03-26T08:30:39.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.207+0000 7f728aa00640 1 -- 172.21.15.43:0/3319774862 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f72800044c0 con 0x7f72941028b0 2024-03-26T08:30:39.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.207+0000 7f7299200640 1 -- 172.21.15.43:0/3319774862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72941a1a10 con 0x7f72941028b0 2024-03-26T08:30:39.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.207+0000 7f728aa00640 1 -- 172.21.15.43:0/3319774862 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7280003c30 con 0x7f72941028b0 2024-03-26T08:30:39.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.207+0000 7f7299200640 1 -- 172.21.15.43:0/3319774862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72941a1f60 con 0x7f72941028b0 2024-03-26T08:30:39.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.208+0000 7f7299200640 1 -- 172.21.15.43:0/3319774862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72941089e0 con 0x7f72941028b0 2024-03-26T08:30:39.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.211+0000 7f728aa00640 1 -- 172.21.15.43:0/3319774862 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f7280046020 con 0x7f72941028b0 2024-03-26T08:30:39.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.212+0000 7f728aa00640 1 --2- 172.21.15.43:0/3319774862 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f72780a2800 0x7f72780a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:39.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.212+0000 7f728aa00640 1 -- 172.21.15.43:0/3319774862 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f72800f09a0 con 0x7f72941028b0 2024-03-26T08:30:39.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.212+0000 7f7291e00640 1 --2- 172.21.15.43:0/3319774862 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f72780a2800 0x7f72780a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:39.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.213+0000 7f7291e00640 1 --2- 172.21.15.43:0/3319774862 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f72780a2800 0x7f72780a4cc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7294102710 tx=0x7f7284005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:39.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.217+0000 7f728aa00640 1 -- 172.21.15.43:0/3319774862 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f728004d020 con 0x7f72941028b0 2024-03-26T08:30:39.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.485+0000 7f7299200640 1 -- 172.21.15.43:0/3319774862 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72940622a0 con 0x7f72941028b0 2024-03-26T08:30:39.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.485+0000 7f728aa00640 1 -- 172.21.15.43:0/3319774862 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2910 (secure 0 0 0) 0x7f72800bde40 con 0x7f72941028b0 2024-03-26T08:30:39.479 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:30:39.479 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1452206807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1452206807},{"type":"v1","addr":"172.21.15.43:6835","nonce":1452206807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34448,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/1108327179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108327179},{"type":"v1","addr":"172.21.15.52:6833","nonce":1108327179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-26T08:30:39.479 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 4 2024-03-26T08:30:39.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.488+0000 7f7299200640 1 -- 172.21.15.43:0/3319774862 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f72780a2800 msgr2=0x7f72780a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:39.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.488+0000 7f7299200640 1 --2- 172.21.15.43:0/3319774862 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f72780a2800 0x7f72780a4cc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7294102710 tx=0x7f7284005f90 comp rx=0 tx=0).stop 2024-03-26T08:30:39.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.488+0000 7f7299200640 1 -- 172.21.15.43:0/3319774862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72941028b0 msgr2=0x7f7294073a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:39.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.488+0000 7f7299200640 1 --2- 172.21.15.43:0/3319774862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72941028b0 0x7f7294073a90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7280000c00 tx=0x7f72800040c0 comp rx=0 tx=0).stop 2024-03-26T08:30:39.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.488+0000 7f7299200640 1 -- 172.21.15.43:0/3319774862 shutdown_connections 2024-03-26T08:30:39.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.488+0000 7f7299200640 1 --2- 172.21.15.43:0/3319774862 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f72780a2800 0x7f72780a4cc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:39.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.488+0000 7f7299200640 1 --2- 172.21.15.43:0/3319774862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7294103270 0x7f7294073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:39.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.488+0000 7f7299200640 1 --2- 172.21.15.43:0/3319774862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72941028b0 0x7f7294073a90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:39.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.488+0000 7f7299200640 1 --2- 172.21.15.43:0/3319774862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72941016c0 0x7f7294075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:39.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.488+0000 7f7299200640 1 -- 172.21.15.43:0/3319774862 >> 172.21.15.43:0/3319774862 conn(0x7f72940fcc50 msgr2=0x7f7294105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:39.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.488+0000 7f7299200640 1 -- 172.21.15.43:0/3319774862 shutdown_connections 2024-03-26T08:30:39.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:39.488+0000 7f7299200640 1 -- 172.21.15.43:0/3319774862 wait complete. 2024-03-26T08:30:40.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:39 smithi043 ceph-mon[31929]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:40.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:39 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3319774862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:40.152 INFO:journalctl@ceph.mds.d.smithi043.stdout:Mar 26 08:30:39 smithi043 systemd[1]: Starting Ceph mds.d for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:30:40.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:39 smithi183 ceph-mon[35818]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:40.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:39 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3319774862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:40.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:39 smithi052 ceph-mon[35071]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:40.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:39 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3319774862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:40.578 INFO:journalctl@ceph.mds.d.smithi043.stdout:Mar 26 08:30:40 smithi043 podman[69941]: 2024-03-26 08:30:40.196841012 +0000 UTC m=+0.037351203 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:30:40.578 INFO:journalctl@ceph.mds.d.smithi043.stdout:Mar 26 08:30:40 smithi043 podman[69941]: 2024-03-26 08:30:40.306336258 +0000 UTC m=+0.146846467 container create 68d36da8a4dcf745f6f321ffa1eea6292a5bfc8112a467d1848baa77de2ff385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-26T08:30:40.903 INFO:journalctl@ceph.mds.d.smithi043.stdout:Mar 26 08:30:40 smithi043 podman[69941]: 2024-03-26 08:30:40.585281312 +0000 UTC m=+0.425791544 container init 68d36da8a4dcf745f6f321ffa1eea6292a5bfc8112a467d1848baa77de2ff385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-26T08:30:40.903 INFO:journalctl@ceph.mds.d.smithi043.stdout:Mar 26 08:30:40 smithi043 podman[69941]: 2024-03-26 08:30:40.59647408 +0000 UTC m=+0.436984276 container start 68d36da8a4dcf745f6f321ffa1eea6292a5bfc8112a467d1848baa77de2ff385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-d, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-26T08:30:40.903 INFO:journalctl@ceph.mds.d.smithi043.stdout:Mar 26 08:30:40 smithi043 ceph-mds[69959]: -- 172.21.15.43:0/3380722124 <== mon.0 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56007b579a40 con 0x56007b54cc00 2024-03-26T08:30:40.903 INFO:journalctl@ceph.mds.d.smithi043.stdout:Mar 26 08:30:40 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-d[69955]: starting mds.d at 2024-03-26T08:30:40.903 INFO:journalctl@ceph.mds.d.smithi043.stdout:Mar 26 08:30:40 smithi043 bash[69941]: 68d36da8a4dcf745f6f321ffa1eea6292a5bfc8112a467d1848baa77de2ff385 2024-03-26T08:30:40.903 INFO:journalctl@ceph.mds.d.smithi043.stdout:Mar 26 08:30:40 smithi043 systemd[1]: Started Ceph mds.d for 88759420-eb49-11ee-95d0-87774f69a715. 2024-03-26T08:30:41.503 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:30:41.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.745+0000 7f4213e00640 1 -- 172.21.15.43:0/3125827846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42141096b0 msgr2=0x7f421410ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:41.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.745+0000 7f4213e00640 1 --2- 172.21.15.43:0/3125827846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42141096b0 0x7f421410ff40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009900 tx=0x7f41fc02bc30 comp rx=0 tx=0).stop 2024-03-26T08:30:41.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.745+0000 7f4213e00640 1 -- 172.21.15.43:0/3125827846 shutdown_connections 2024-03-26T08:30:41.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.745+0000 7f4213e00640 1 --2- 172.21.15.43:0/3125827846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42141096b0 0x7f421410ff40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:41.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.745+0000 7f4213e00640 1 --2- 172.21.15.43:0/3125827846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4214108cf0 0x7f4214109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:41.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.745+0000 7f4213e00640 1 --2- 172.21.15.43:0/3125827846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4214107b00 0x7f4214107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:41.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.745+0000 7f4213e00640 1 -- 172.21.15.43:0/3125827846 >> 172.21.15.43:0/3125827846 conn(0x7f4214076ab0 msgr2=0x7f4214078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:41.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.745+0000 7f4213e00640 1 -- 172.21.15.43:0/3125827846 shutdown_connections 2024-03-26T08:30:41.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.745+0000 7f4213e00640 1 -- 172.21.15.43:0/3125827846 wait complete. 2024-03-26T08:30:41.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.746+0000 7f4213e00640 1 Processor -- start 2024-03-26T08:30:41.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.747+0000 7f4213e00640 1 -- start start 2024-03-26T08:30:41.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.747+0000 7f4213e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4214107b00 0x7f421419dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:41.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.747+0000 7f4213e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4214108cf0 0x7f421419e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:41.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.747+0000 7f4213e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42141096b0 0x7f42141a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:41.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.747+0000 7f4213e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4214113680 con 0x7f4214108cf0 2024-03-26T08:30:41.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.747+0000 7f4213e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4214113500 con 0x7f42141096b0 2024-03-26T08:30:41.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.747+0000 7f4213e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4214113800 con 0x7f4214107b00 2024-03-26T08:30:41.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.747+0000 7f4212000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4214108cf0 0x7f421419e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:41.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.747+0000 7f4212000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4214108cf0 0x7f421419e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34886/0 (socket says 172.21.15.43:34886) 2024-03-26T08:30:41.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.747+0000 7f4212000640 1 -- 172.21.15.43:0/267495288 learned_addr learned my addr 172.21.15.43:0/267495288 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:30:41.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.747+0000 7f4213400640 1 --2- 172.21.15.43:0/267495288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42141096b0 0x7f42141a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:41.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.748+0000 7f4212a00640 1 --2- 172.21.15.43:0/267495288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4214107b00 0x7f421419dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:41.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.748+0000 7f4213400640 1 -- 172.21.15.43:0/267495288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4214107b00 msgr2=0x7f421419dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:41.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.748+0000 7f4213400640 1 --2- 172.21.15.43:0/267495288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4214107b00 0x7f421419dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:41.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.748+0000 7f4213400640 1 -- 172.21.15.43:0/267495288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4214108cf0 msgr2=0x7f421419e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:41.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.748+0000 7f4213400640 1 --2- 172.21.15.43:0/267495288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4214108cf0 0x7f421419e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:41.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.748+0000 7f4213400640 1 -- 172.21.15.43:0/267495288 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42141a58e0 con 0x7f42141096b0 2024-03-26T08:30:41.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.748+0000 7f4212a00640 1 --2- 172.21.15.43:0/267495288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4214107b00 0x7f421419dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:30:41.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.748+0000 7f4213400640 1 --2- 172.21.15.43:0/267495288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42141096b0 0x7f42141a5240 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f41fc036040 tx=0x7f41fc02cdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:41.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.749+0000 7f420b400640 1 -- 172.21.15.43:0/267495288 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc043020 con 0x7f42141096b0 2024-03-26T08:30:41.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.749+0000 7f4213e00640 1 -- 172.21.15.43:0/267495288 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42141a5b70 con 0x7f42141096b0 2024-03-26T08:30:41.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.749+0000 7f420b400640 1 -- 172.21.15.43:0/267495288 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f41fc004490 con 0x7f42141096b0 2024-03-26T08:30:41.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.749+0000 7f420b400640 1 -- 172.21.15.43:0/267495288 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc003bc0 con 0x7f42141096b0 2024-03-26T08:30:41.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.749+0000 7f4213e00640 1 -- 172.21.15.43:0/267495288 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42141a6110 con 0x7f42141096b0 2024-03-26T08:30:41.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.750+0000 7f4213e00640 1 -- 172.21.15.43:0/267495288 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f421410ee20 con 0x7f42141096b0 2024-03-26T08:30:41.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.752+0000 7f420b400640 1 -- 172.21.15.43:0/267495288 <== mon.2 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f41fc046020 con 0x7f42141096b0 2024-03-26T08:30:41.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.753+0000 7f420b400640 1 --2- 172.21.15.43:0/267495288 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f41f80a2800 0x7f41f80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:41.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.754+0000 7f420b400640 1 -- 172.21.15.43:0/267495288 <== mon.2 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f41fc0f0e90 con 0x7f42141096b0 2024-03-26T08:30:41.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.754+0000 7f4212a00640 1 --2- 172.21.15.43:0/267495288 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f41f80a2800 0x7f41f80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:41.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.754+0000 7f4212a00640 1 --2- 172.21.15.43:0/267495288 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f41f80a2800 0x7f41f80a4cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f421419eb70 tx=0x7f420c009520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:41.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:41.761+0000 7f420b400640 1 -- 172.21.15.43:0/267495288 <== mon.2 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f41fc04d020 con 0x7f42141096b0 2024-03-26T08:30:41.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:41 smithi043 ceph-mon[31929]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:41.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:41.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:41.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:41.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:30:41.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:30:41.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:41.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:41 smithi043 ceph-mon[31929]: mds.? [v2:172.21.15.43:6836/3151775467,v1:172.21.15.43:6837/3151775467] up:boot 2024-03-26T08:30:41.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:41 smithi043 ceph-mon[31929]: fsmap 4 up:standby 2024-03-26T08:30:41.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:41 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T08:30:41.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:41 smithi043 ceph-mon[31929]: Deploying daemon mds.e on smithi052 2024-03-26T08:30:42.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:42.008+0000 7f4213e00640 1 -- 172.21.15.43:0/267495288 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f421402d220 con 0x7f42141096b0 2024-03-26T08:30:42.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:42.010+0000 7f420b400640 1 -- 172.21.15.43:0/267495288 <== mon.2 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3701 (secure 0 0 0) 0x7f41fc0f0090 con 0x7f42141096b0 2024-03-26T08:30:42.003 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:30:42.003 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1452206807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1452206807},{"type":"v1","addr":"172.21.15.43:6835","nonce":1452206807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3151775467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3151775467},{"type":"v1","addr":"172.21.15.43:6837","nonce":3151775467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34448,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/1108327179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108327179},{"type":"v1","addr":"172.21.15.52:6833","nonce":1108327179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-26T08:30:42.003 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 5 2024-03-26T08:30:42.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:42.013+0000 7f4213e00640 1 -- 172.21.15.43:0/267495288 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f41f80a2800 msgr2=0x7f41f80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:42.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:42.013+0000 7f4213e00640 1 --2- 172.21.15.43:0/267495288 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f41f80a2800 0x7f41f80a4cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f421419eb70 tx=0x7f420c009520 comp rx=0 tx=0).stop 2024-03-26T08:30:42.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:42.013+0000 7f4213e00640 1 -- 172.21.15.43:0/267495288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42141096b0 msgr2=0x7f42141a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:42.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:42.013+0000 7f4213e00640 1 --2- 172.21.15.43:0/267495288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42141096b0 0x7f42141a5240 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f41fc036040 tx=0x7f41fc02cdd0 comp rx=0 tx=0).stop 2024-03-26T08:30:42.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:42.013+0000 7f4213e00640 1 -- 172.21.15.43:0/267495288 shutdown_connections 2024-03-26T08:30:42.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:42.013+0000 7f4213e00640 1 --2- 172.21.15.43:0/267495288 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f41f80a2800 0x7f41f80a4cc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:42.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:42.013+0000 7f4213e00640 1 --2- 172.21.15.43:0/267495288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42141096b0 0x7f42141a5240 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:42.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:42.013+0000 7f4213e00640 1 --2- 172.21.15.43:0/267495288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4214108cf0 0x7f421419e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:42.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:42.013+0000 7f4213e00640 1 --2- 172.21.15.43:0/267495288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4214107b00 0x7f421419dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:42.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:42.013+0000 7f4213e00640 1 -- 172.21.15.43:0/267495288 >> 172.21.15.43:0/267495288 conn(0x7f4214076ab0 msgr2=0x7f421410bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:42.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:42.013+0000 7f4213e00640 1 -- 172.21.15.43:0/267495288 shutdown_connections 2024-03-26T08:30:42.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:42.013+0000 7f4213e00640 1 -- 172.21.15.43:0/267495288 wait complete. 2024-03-26T08:30:42.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:41 smithi052 ceph-mon[35071]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:42.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:42.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:42.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:42.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:30:42.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:30:42.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:42.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:41 smithi052 ceph-mon[35071]: mds.? [v2:172.21.15.43:6836/3151775467,v1:172.21.15.43:6837/3151775467] up:boot 2024-03-26T08:30:42.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:41 smithi052 ceph-mon[35071]: fsmap 4 up:standby 2024-03-26T08:30:42.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:41 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T08:30:42.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:41 smithi052 ceph-mon[35071]: Deploying daemon mds.e on smithi052 2024-03-26T08:30:42.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:41 smithi183 ceph-mon[35818]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:42.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:42.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:42.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:42.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:30:42.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:30:42.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:42.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:41 smithi183 ceph-mon[35818]: mds.? [v2:172.21.15.43:6836/3151775467,v1:172.21.15.43:6837/3151775467] up:boot 2024-03-26T08:30:42.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:41 smithi183 ceph-mon[35818]: fsmap 4 up:standby 2024-03-26T08:30:42.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:41 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T08:30:42.300 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:41 smithi183 ceph-mon[35818]: Deploying daemon mds.e on smithi052 2024-03-26T08:30:43.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:42 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/267495288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:43.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:42 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/267495288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:43.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:42 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/267495288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:44.029 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:30:44.124 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:43 smithi043 ceph-mon[31929]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:44.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.274+0000 7ff893400640 1 -- 172.21.15.43:0/522403381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff88c109890 msgr2=0x7ff88c110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:44.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.274+0000 7ff893400640 1 --2- 172.21.15.43:0/522403381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff88c109890 0x7ff88c110120 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff884009ca0 tx=0x7ff88402c0f0 comp rx=0 tx=0).stop 2024-03-26T08:30:44.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.274+0000 7ff893400640 1 -- 172.21.15.43:0/522403381 shutdown_connections 2024-03-26T08:30:44.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.274+0000 7ff893400640 1 --2- 172.21.15.43:0/522403381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff88c109890 0x7ff88c110120 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:44.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.274+0000 7ff893400640 1 --2- 172.21.15.43:0/522403381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff88c108ed0 0x7ff88c109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:44.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.274+0000 7ff893400640 1 --2- 172.21.15.43:0/522403381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff88c107ce0 0x7ff88c108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:44.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.274+0000 7ff893400640 1 -- 172.21.15.43:0/522403381 >> 172.21.15.43:0/522403381 conn(0x7ff88c076cb0 msgr2=0x7ff88c0790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:44.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.274+0000 7ff893400640 1 -- 172.21.15.43:0/522403381 shutdown_connections 2024-03-26T08:30:44.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.274+0000 7ff893400640 1 -- 172.21.15.43:0/522403381 wait complete. 2024-03-26T08:30:44.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.275+0000 7ff893400640 1 Processor -- start 2024-03-26T08:30:44.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.275+0000 7ff893400640 1 -- start start 2024-03-26T08:30:44.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.276+0000 7ff893400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff88c107ce0 0x7ff88c19de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:44.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.276+0000 7ff893400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff88c108ed0 0x7ff88c19e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:44.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.276+0000 7ff893400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff88c109890 0x7ff88c1a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:44.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.276+0000 7ff893400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c113810 con 0x7ff88c107ce0 2024-03-26T08:30:44.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.276+0000 7ff893400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c113690 con 0x7ff88c108ed0 2024-03-26T08:30:44.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.276+0000 7ff893400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c113990 con 0x7ff88c109890 2024-03-26T08:30:44.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.276+0000 7ff890a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff88c107ce0 0x7ff88c19de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:44.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.276+0000 7ff890a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff88c107ce0 0x7ff88c19de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44482/0 (socket says 172.21.15.43:44482) 2024-03-26T08:30:44.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.276+0000 7ff890a00640 1 -- 172.21.15.43:0/95804437 learned_addr learned my addr 172.21.15.43:0/95804437 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:30:44.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.276+0000 7ff88be00640 1 --2- 172.21.15.43:0/95804437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff88c108ed0 0x7ff88c19e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:44.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.276+0000 7ff891400640 1 --2- 172.21.15.43:0/95804437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff88c109890 0x7ff88c1a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:44.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.276+0000 7ff890a00640 1 -- 172.21.15.43:0/95804437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff88c109890 msgr2=0x7ff88c1a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:44.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.276+0000 7ff890a00640 1 --2- 172.21.15.43:0/95804437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff88c109890 0x7ff88c1a5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:44.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.276+0000 7ff890a00640 1 -- 172.21.15.43:0/95804437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff88c108ed0 msgr2=0x7ff88c19e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:44.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.276+0000 7ff890a00640 1 --2- 172.21.15.43:0/95804437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff88c108ed0 0x7ff88c19e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:44.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.276+0000 7ff890a00640 1 -- 172.21.15.43:0/95804437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88c1a5ab0 con 0x7ff88c107ce0 2024-03-26T08:30:44.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.276+0000 7ff891400640 1 --2- 172.21.15.43:0/95804437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff88c109890 0x7ff88c1a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:30:44.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.276+0000 7ff88be00640 1 --2- 172.21.15.43:0/95804437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff88c108ed0 0x7ff88c19e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:30:44.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.277+0000 7ff890a00640 1 --2- 172.21.15.43:0/95804437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff88c107ce0 0x7ff88c19de90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff88000e9c0 tx=0x7ff88000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:44.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.277+0000 7ff889600640 1 -- 172.21.15.43:0/95804437 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88000cd80 con 0x7ff88c107ce0 2024-03-26T08:30:44.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.277+0000 7ff889600640 1 -- 172.21.15.43:0/95804437 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff8800035f0 con 0x7ff88c107ce0 2024-03-26T08:30:44.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.277+0000 7ff893400640 1 -- 172.21.15.43:0/95804437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff88c1a5da0 con 0x7ff88c107ce0 2024-03-26T08:30:44.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.277+0000 7ff889600640 1 -- 172.21.15.43:0/95804437 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff880010690 con 0x7ff88c107ce0 2024-03-26T08:30:44.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.277+0000 7ff893400640 1 -- 172.21.15.43:0/95804437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff88c1a6210 con 0x7ff88c107ce0 2024-03-26T08:30:44.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.278+0000 7ff893400640 1 -- 172.21.15.43:0/95804437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff88c10f000 con 0x7ff88c107ce0 2024-03-26T08:30:44.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.280+0000 7ff889600640 1 -- 172.21.15.43:0/95804437 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff8800037a0 con 0x7ff88c107ce0 2024-03-26T08:30:44.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.282+0000 7ff889600640 1 --2- 172.21.15.43:0/95804437 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff8740a2800 0x7ff8740a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:44.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.282+0000 7ff889600640 1 -- 172.21.15.43:0/95804437 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff8800ceb30 con 0x7ff88c107ce0 2024-03-26T08:30:44.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.282+0000 7ff88be00640 1 --2- 172.21.15.43:0/95804437 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff8740a2800 0x7ff8740a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:44.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.283+0000 7ff88be00640 1 --2- 172.21.15.43:0/95804437 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff8740a2800 0x7ff8740a4cc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff8780058c0 tx=0x7ff878005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:44.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.289+0000 7ff889600640 1 -- 172.21.15.43:0/95804437 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff88009bfd0 con 0x7ff88c107ce0 2024-03-26T08:30:44.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:43 smithi183 ceph-mon[35818]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:44.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:43 smithi052 ceph-mon[35071]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:44.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.535+0000 7ff893400640 1 -- 172.21.15.43:0/95804437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff88c02d070 con 0x7ff88c107ce0 2024-03-26T08:30:44.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.536+0000 7ff889600640 1 -- 172.21.15.43:0/95804437 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3701 (secure 0 0 0) 0x7ff88c02d070 con 0x7ff88c107ce0 2024-03-26T08:30:44.529 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:30:44.530 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1452206807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1452206807},{"type":"v1","addr":"172.21.15.43:6835","nonce":1452206807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3151775467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3151775467},{"type":"v1","addr":"172.21.15.43:6837","nonce":3151775467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34448,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/1108327179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108327179},{"type":"v1","addr":"172.21.15.52:6833","nonce":1108327179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-26T08:30:44.530 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 5 2024-03-26T08:30:44.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.539+0000 7ff893400640 1 -- 172.21.15.43:0/95804437 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff8740a2800 msgr2=0x7ff8740a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:44.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.539+0000 7ff893400640 1 --2- 172.21.15.43:0/95804437 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff8740a2800 0x7ff8740a4cc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff8780058c0 tx=0x7ff878005850 comp rx=0 tx=0).stop 2024-03-26T08:30:44.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.539+0000 7ff893400640 1 -- 172.21.15.43:0/95804437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff88c107ce0 msgr2=0x7ff88c19de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:44.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.539+0000 7ff893400640 1 --2- 172.21.15.43:0/95804437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff88c107ce0 0x7ff88c19de90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff88000e9c0 tx=0x7ff88000ee80 comp rx=0 tx=0).stop 2024-03-26T08:30:44.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.539+0000 7ff893400640 1 -- 172.21.15.43:0/95804437 shutdown_connections 2024-03-26T08:30:44.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.539+0000 7ff893400640 1 --2- 172.21.15.43:0/95804437 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff8740a2800 0x7ff8740a4cc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:44.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.539+0000 7ff893400640 1 --2- 172.21.15.43:0/95804437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff88c109890 0x7ff88c1a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:44.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.539+0000 7ff893400640 1 --2- 172.21.15.43:0/95804437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff88c108ed0 0x7ff88c19e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:44.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.539+0000 7ff893400640 1 --2- 172.21.15.43:0/95804437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff88c107ce0 0x7ff88c19de90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:44.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.539+0000 7ff893400640 1 -- 172.21.15.43:0/95804437 >> 172.21.15.43:0/95804437 conn(0x7ff88c076cb0 msgr2=0x7ff88c10c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:44.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.540+0000 7ff893400640 1 -- 172.21.15.43:0/95804437 shutdown_connections 2024-03-26T08:30:44.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:44.540+0000 7ff893400640 1 -- 172.21.15.43:0/95804437 wait complete. 2024-03-26T08:30:44.724 INFO:journalctl@ceph.mds.e.smithi052.stdout:Mar 26 08:30:44 smithi052 systemd[1]: Starting Ceph mds.e for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:30:45.038 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:44 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/95804437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:45.039 INFO:journalctl@ceph.mds.e.smithi052.stdout:Mar 26 08:30:44 smithi052 podman[64969]: 2024-03-26 08:30:44.634406558 +0000 UTC m=+0.033121457 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:30:45.039 INFO:journalctl@ceph.mds.e.smithi052.stdout:Mar 26 08:30:44 smithi052 podman[64969]: 2024-03-26 08:30:44.749268709 +0000 UTC m=+0.147983632 container create 7f77c1bd618d1e95025056e983f252626862ee2251c8ecd2732175f0f0f9916d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T08:30:45.039 INFO:journalctl@ceph.mds.e.smithi052.stdout:Mar 26 08:30:45 smithi052 podman[64969]: 2024-03-26 08:30:45.045506069 +0000 UTC m=+0.444220999 container init 7f77c1bd618d1e95025056e983f252626862ee2251c8ecd2732175f0f0f9916d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-e, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-26T08:30:45.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:44 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/95804437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:45.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:44 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/95804437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:45.323 INFO:journalctl@ceph.mds.e.smithi052.stdout:Mar 26 08:30:45 smithi052 podman[64969]: 2024-03-26 08:30:45.058136418 +0000 UTC m=+0.456851314 container start 7f77c1bd618d1e95025056e983f252626862ee2251c8ecd2732175f0f0f9916d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-e, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-26T08:30:45.323 INFO:journalctl@ceph.mds.e.smithi052.stdout:Mar 26 08:30:45 smithi052 ceph-mds[64988]: -- 172.21.15.52:0/1808633074 <== mon.2 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56472907da40 con 0x564729051000 2024-03-26T08:30:45.323 INFO:journalctl@ceph.mds.e.smithi052.stdout:Mar 26 08:30:45 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-e[64984]: starting mds.e at 2024-03-26T08:30:45.323 INFO:journalctl@ceph.mds.e.smithi052.stdout:Mar 26 08:30:45 smithi052 bash[64969]: 7f77c1bd618d1e95025056e983f252626862ee2251c8ecd2732175f0f0f9916d 2024-03-26T08:30:45.323 INFO:journalctl@ceph.mds.e.smithi052.stdout:Mar 26 08:30:45 smithi052 systemd[1]: Started Ceph mds.e for 88759420-eb49-11ee-95d0-87774f69a715. 2024-03-26T08:30:46.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:45 smithi043 ceph-mon[31929]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:46.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:45 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:46.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:45 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:46.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:45 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:46.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:45 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:30:46.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:45 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:30:46.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:45 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:46.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:45 smithi183 ceph-mon[35818]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:46.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:45 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:46.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:45 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:46.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:45 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:46.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:45 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:30:46.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:45 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:30:46.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:45 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:46.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:45 smithi052 ceph-mon[35071]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:46.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:45 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:46.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:45 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:46.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:45 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:46.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:45 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:30:46.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:45 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:30:46.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:45 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:46.551 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:30:46.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.798+0000 7ff0b8c00640 1 -- 172.21.15.43:0/3728011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0b41073a0 msgr2=0x7ff0b41097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:46.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.798+0000 7ff0b8c00640 1 --2- 172.21.15.43:0/3728011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0b41073a0 0x7ff0b41097f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009900 tx=0x7ff0ac02bc30 comp rx=0 tx=0).stop 2024-03-26T08:30:46.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.798+0000 7ff0b8c00640 1 -- 172.21.15.43:0/3728011 shutdown_connections 2024-03-26T08:30:46.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.798+0000 7ff0b8c00640 1 --2- 172.21.15.43:0/3728011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0b41073a0 0x7ff0b41097f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:46.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.798+0000 7ff0b8c00640 1 --2- 172.21.15.43:0/3728011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0b4104a70 0x7ff0b4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:46.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.798+0000 7ff0b8c00640 1 --2- 172.21.15.43:0/3728011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0b4102140 0x7ff0b4104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:46.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.798+0000 7ff0b8c00640 1 -- 172.21.15.43:0/3728011 >> 172.21.15.43:0/3728011 conn(0x7ff0b40fbb50 msgr2=0x7ff0b40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:46.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.798+0000 7ff0b8c00640 1 -- 172.21.15.43:0/3728011 shutdown_connections 2024-03-26T08:30:46.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.798+0000 7ff0b8c00640 1 -- 172.21.15.43:0/3728011 wait complete. 2024-03-26T08:30:46.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.799+0000 7ff0b8c00640 1 Processor -- start 2024-03-26T08:30:46.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.799+0000 7ff0b8c00640 1 -- start start 2024-03-26T08:30:46.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.799+0000 7ff0b8c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0b4102140 0x7ff0b4101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:46.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.799+0000 7ff0b8c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0b4104a70 0x7ff0b41000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:46.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.799+0000 7ff0b8c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0b41073a0 0x7ff0b4100630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:46.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.799+0000 7ff0b8c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b410d260 con 0x7ff0b4102140 2024-03-26T08:30:46.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.799+0000 7ff0b8c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b410d0e0 con 0x7ff0b4104a70 2024-03-26T08:30:46.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.799+0000 7ff0b8c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b410d3e0 con 0x7ff0b41073a0 2024-03-26T08:30:46.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.800+0000 7ff0b1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0b4102140 0x7ff0b4101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:46.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.800+0000 7ff0b1e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0b4102140 0x7ff0b4101ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44510/0 (socket says 172.21.15.43:44510) 2024-03-26T08:30:46.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.800+0000 7ff0b1e00640 1 -- 172.21.15.43:0/375326504 learned_addr learned my addr 172.21.15.43:0/375326504 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:30:46.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.800+0000 7ff0b1400640 1 --2- 172.21.15.43:0/375326504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0b4104a70 0x7ff0b41000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:46.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.800+0000 7ff0b2800640 1 --2- 172.21.15.43:0/375326504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0b41073a0 0x7ff0b4100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:46.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.800+0000 7ff0b1e00640 1 -- 172.21.15.43:0/375326504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0b41073a0 msgr2=0x7ff0b4100630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:46.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.800+0000 7ff0b1e00640 1 --2- 172.21.15.43:0/375326504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0b41073a0 0x7ff0b4100630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:46.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.800+0000 7ff0b1e00640 1 -- 172.21.15.43:0/375326504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0b4104a70 msgr2=0x7ff0b41000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:46.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.800+0000 7ff0b1e00640 1 --2- 172.21.15.43:0/375326504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0b4104a70 0x7ff0b41000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:46.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.800+0000 7ff0b1e00640 1 -- 172.21.15.43:0/375326504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b4100ec0 con 0x7ff0b4102140 2024-03-26T08:30:46.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.800+0000 7ff0b1400640 1 --2- 172.21.15.43:0/375326504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0b4104a70 0x7ff0b41000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:30:46.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.800+0000 7ff0b2800640 1 --2- 172.21.15.43:0/375326504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0b41073a0 0x7ff0b4100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:30:46.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.800+0000 7ff0b1e00640 1 --2- 172.21.15.43:0/375326504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0b4102140 0x7ff0b4101ab0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400d770 tx=0x7ff0a400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:46.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.801+0000 7ff0aaa00640 1 -- 172.21.15.43:0/375326504 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4017070 con 0x7ff0b4102140 2024-03-26T08:30:46.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.801+0000 7ff0aaa00640 1 -- 172.21.15.43:0/375326504 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff0a40054d0 con 0x7ff0b4102140 2024-03-26T08:30:46.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.801+0000 7ff0b8c00640 1 -- 172.21.15.43:0/375326504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b41a1a00 con 0x7ff0b4102140 2024-03-26T08:30:46.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.801+0000 7ff0aaa00640 1 -- 172.21.15.43:0/375326504 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4004cf0 con 0x7ff0b4102140 2024-03-26T08:30:46.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.801+0000 7ff0b8c00640 1 -- 172.21.15.43:0/375326504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b41a1f00 con 0x7ff0b4102140 2024-03-26T08:30:46.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.802+0000 7ff0b8c00640 1 -- 172.21.15.43:0/375326504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b40fd640 con 0x7ff0b4102140 2024-03-26T08:30:46.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.804+0000 7ff0aaa00640 1 -- 172.21.15.43:0/375326504 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff0a4004e90 con 0x7ff0b4102140 2024-03-26T08:30:46.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.807+0000 7ff0aaa00640 1 --2- 172.21.15.43:0/375326504 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff0940a2800 0x7ff0940a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:46.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.807+0000 7ff0aaa00640 1 -- 172.21.15.43:0/375326504 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff0a40ce170 con 0x7ff0b4102140 2024-03-26T08:30:46.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.807+0000 7ff0b1400640 1 --2- 172.21.15.43:0/375326504 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff0940a2800 0x7ff0940a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:46.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.808+0000 7ff0b1400640 1 --2- 172.21.15.43:0/375326504 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff0940a2800 0x7ff0940a4cc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4101820 tx=0x7ff09c0082f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:46.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:46.812+0000 7ff0aaa00640 1 -- 172.21.15.43:0/375326504 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff0a409b610 con 0x7ff0b4102140 2024-03-26T08:30:46.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:46 smithi043 ceph-mon[31929]: Deploying daemon mds.f on smithi183 2024-03-26T08:30:46.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:46 smithi043 ceph-mon[31929]: mds.? [v2:172.21.15.52:6834/742057410,v1:172.21.15.52:6835/742057410] up:boot 2024-03-26T08:30:46.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:46 smithi043 ceph-mon[31929]: fsmap 5 up:standby 2024-03-26T08:30:46.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:46 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T08:30:47.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:47.056+0000 7ff0b8c00640 1 -- 172.21.15.43:0/375326504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0b402d070 con 0x7ff0b4102140 2024-03-26T08:30:47.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:46 smithi183 ceph-mon[35818]: Deploying daemon mds.f on smithi183 2024-03-26T08:30:47.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:46 smithi183 ceph-mon[35818]: mds.? [v2:172.21.15.52:6834/742057410,v1:172.21.15.52:6835/742057410] up:boot 2024-03-26T08:30:47.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:46 smithi183 ceph-mon[35818]: fsmap 5 up:standby 2024-03-26T08:30:47.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:46 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T08:30:47.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:47.057+0000 7ff0aaa00640 1 -- 172.21.15.43:0/375326504 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4489 (secure 0 0 0) 0x7ff0a409af10 con 0x7ff0b4102140 2024-03-26T08:30:47.050 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:30:47.050 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1452206807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1452206807},{"type":"v1","addr":"172.21.15.43:6835","nonce":1452206807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3151775467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3151775467},{"type":"v1","addr":"172.21.15.43:6837","nonce":3151775467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34448,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/1108327179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108327179},{"type":"v1","addr":"172.21.15.52:6833","nonce":1108327179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34487,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/742057410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":742057410},{"type":"v1","addr":"172.21.15.52:6835","nonce":742057410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-26T08:30:47.050 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 6 2024-03-26T08:30:47.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:47.060+0000 7ff0b8c00640 1 -- 172.21.15.43:0/375326504 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff0940a2800 msgr2=0x7ff0940a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:47.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:47.060+0000 7ff0b8c00640 1 --2- 172.21.15.43:0/375326504 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff0940a2800 0x7ff0940a4cc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4101820 tx=0x7ff09c0082f0 comp rx=0 tx=0).stop 2024-03-26T08:30:47.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:47.060+0000 7ff0b8c00640 1 -- 172.21.15.43:0/375326504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0b4102140 msgr2=0x7ff0b4101ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:47.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:47.060+0000 7ff0b8c00640 1 --2- 172.21.15.43:0/375326504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0b4102140 0x7ff0b4101ab0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400d770 tx=0x7ff0a400dc30 comp rx=0 tx=0).stop 2024-03-26T08:30:47.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:47.060+0000 7ff0b8c00640 1 -- 172.21.15.43:0/375326504 shutdown_connections 2024-03-26T08:30:47.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:47.060+0000 7ff0b8c00640 1 --2- 172.21.15.43:0/375326504 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff0940a2800 0x7ff0940a4cc0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:47.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:47.060+0000 7ff0b8c00640 1 --2- 172.21.15.43:0/375326504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0b41073a0 0x7ff0b4100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:47.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:47.060+0000 7ff0b8c00640 1 --2- 172.21.15.43:0/375326504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0b4104a70 0x7ff0b41000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:47.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:47.060+0000 7ff0b8c00640 1 --2- 172.21.15.43:0/375326504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0b4102140 0x7ff0b4101ab0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:47.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:47.060+0000 7ff0b8c00640 1 -- 172.21.15.43:0/375326504 >> 172.21.15.43:0/375326504 conn(0x7ff0b40fbb50 msgr2=0x7ff0b4102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:47.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:47.061+0000 7ff0b8c00640 1 -- 172.21.15.43:0/375326504 shutdown_connections 2024-03-26T08:30:47.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:47.061+0000 7ff0b8c00640 1 -- 172.21.15.43:0/375326504 wait complete. 2024-03-26T08:30:47.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:46 smithi052 ceph-mon[35071]: Deploying daemon mds.f on smithi183 2024-03-26T08:30:47.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:46 smithi052 ceph-mon[35071]: mds.? [v2:172.21.15.52:6834/742057410,v1:172.21.15.52:6835/742057410] up:boot 2024-03-26T08:30:47.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:46 smithi052 ceph-mon[35071]: fsmap 5 up:standby 2024-03-26T08:30:47.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:46 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T08:30:48.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:47 smithi043 ceph-mon[31929]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:48.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:47 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/375326504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:48.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:47 smithi183 ceph-mon[35818]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:48.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:47 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/375326504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:48.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:47 smithi052 ceph-mon[35071]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:48.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:47 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/375326504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:48.983 INFO:journalctl@ceph.mds.f.smithi183.stdout:Mar 26 08:30:48 smithi183 podman[66213]: 2024-03-26 08:30:48.837393284 +0000 UTC m=+0.165020939 container create 321fef0effb14b8de5d66fe5421c5e4391c01cc7acd8e34f859d70c02b2334ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-f, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-26T08:30:49.075 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:30:49.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.318+0000 7f66cb400640 1 -- 172.21.15.43:0/2824547413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c4104ad0 msgr2=0x7f66c4104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:49.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.318+0000 7f66cb400640 1 --2- 172.21.15.43:0/2824547413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c4104ad0 0x7f66c4104f50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009900 tx=0x7f66b002bc30 comp rx=0 tx=0).stop 2024-03-26T08:30:49.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.319+0000 7f66cb400640 1 -- 172.21.15.43:0/2824547413 shutdown_connections 2024-03-26T08:30:49.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.319+0000 7f66cb400640 1 --2- 172.21.15.43:0/2824547413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66c4105490 0x7f66c410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:49.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.319+0000 7f66cb400640 1 --2- 172.21.15.43:0/2824547413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c4104ad0 0x7f66c4104f50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:49.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.319+0000 7f66cb400640 1 --2- 172.21.15.43:0/2824547413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66c41038e0 0x7f66c4103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:49.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.319+0000 7f66cb400640 1 -- 172.21.15.43:0/2824547413 >> 172.21.15.43:0/2824547413 conn(0x7f66c40fee90 msgr2=0x7f66c41012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:49.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.319+0000 7f66cb400640 1 -- 172.21.15.43:0/2824547413 shutdown_connections 2024-03-26T08:30:49.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.319+0000 7f66cb400640 1 -- 172.21.15.43:0/2824547413 wait complete. 2024-03-26T08:30:49.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.320+0000 7f66cb400640 1 Processor -- start 2024-03-26T08:30:49.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.320+0000 7f66cb400640 1 -- start start 2024-03-26T08:30:49.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.320+0000 7f66cb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66c41038e0 0x7f66c4199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:49.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.320+0000 7f66cb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c4104ad0 0x7f66c419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:49.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.320+0000 7f66cb400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66c4105490 0x7f66c41a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:49.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.320+0000 7f66cb400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c410f460 con 0x7f66c4104ad0 2024-03-26T08:30:49.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.320+0000 7f66cb400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c410f2e0 con 0x7f66c4105490 2024-03-26T08:30:49.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.320+0000 7f66cb400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c410f5e0 con 0x7f66c41038e0 2024-03-26T08:30:49.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.321+0000 7f66c3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c4104ad0 0x7f66c419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:49.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.321+0000 7f66c3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c4104ad0 0x7f66c419a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44532/0 (socket says 172.21.15.43:44532) 2024-03-26T08:30:49.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.321+0000 7f66c3e00640 1 -- 172.21.15.43:0/4268657868 learned_addr learned my addr 172.21.15.43:0/4268657868 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:30:49.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.321+0000 7f66c9400640 1 --2- 172.21.15.43:0/4268657868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66c4105490 0x7f66c41a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:49.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.321+0000 7f66c3e00640 1 -- 172.21.15.43:0/4268657868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66c41038e0 msgr2=0x7f66c4199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:49.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.321+0000 7f66c3e00640 1 --2- 172.21.15.43:0/4268657868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66c41038e0 0x7f66c4199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:49.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.321+0000 7f66c3e00640 1 -- 172.21.15.43:0/4268657868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66c4105490 msgr2=0x7f66c41a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:49.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.321+0000 7f66c3e00640 1 --2- 172.21.15.43:0/4268657868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66c4105490 0x7f66c41a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:49.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.321+0000 7f66c3e00640 1 -- 172.21.15.43:0/4268657868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c41a1750 con 0x7f66c4104ad0 2024-03-26T08:30:49.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.322+0000 7f66c3e00640 1 --2- 172.21.15.43:0/4268657868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c4104ad0 0x7f66c419a070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f66b0000c00 tx=0x7f66b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:49.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.325+0000 7f66c1600640 1 -- 172.21.15.43:0/4268657868 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b0043020 con 0x7f66c4104ad0 2024-03-26T08:30:49.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.326+0000 7f66cb400640 1 -- 172.21.15.43:0/4268657868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c41a19e0 con 0x7f66c4104ad0 2024-03-26T08:30:49.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.326+0000 7f66c1600640 1 -- 172.21.15.43:0/4268657868 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f66b00054c0 con 0x7f66c4104ad0 2024-03-26T08:30:49.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.326+0000 7f66c1600640 1 -- 172.21.15.43:0/4268657868 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b0003d70 con 0x7f66c4104ad0 2024-03-26T08:30:49.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.326+0000 7f66cb400640 1 -- 172.21.15.43:0/4268657868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c41a1f80 con 0x7f66c4104ad0 2024-03-26T08:30:49.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.327+0000 7f66cb400640 1 -- 172.21.15.43:0/4268657868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c410ac50 con 0x7f66c4104ad0 2024-03-26T08:30:49.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.329+0000 7f66c1600640 1 -- 172.21.15.43:0/4268657868 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f66b0046020 con 0x7f66c4104ad0 2024-03-26T08:30:49.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.333+0000 7f66c1600640 1 --2- 172.21.15.43:0/4268657868 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f66ac0a2800 0x7f66ac0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:49.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.334+0000 7f66c1600640 1 -- 172.21.15.43:0/4268657868 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f66b00f0f20 con 0x7f66c4104ad0 2024-03-26T08:30:49.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.334+0000 7f66c8a00640 1 --2- 172.21.15.43:0/4268657868 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f66ac0a2800 0x7f66ac0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:49.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.334+0000 7f66c1600640 1 -- 172.21.15.43:0/4268657868 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f66b004d020 con 0x7f66c4104ad0 2024-03-26T08:30:49.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.335+0000 7f66c8a00640 1 --2- 172.21.15.43:0/4268657868 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f66ac0a2800 0x7f66ac0a4cc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f66b8005fd0 tx=0x7f66b800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:49.548 INFO:journalctl@ceph.mds.f.smithi183.stdout:Mar 26 08:30:49 smithi183 podman[66213]: 2024-03-26 08:30:49.10812421 +0000 UTC m=+0.435751872 container init 321fef0effb14b8de5d66fe5421c5e4391c01cc7acd8e34f859d70c02b2334ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T08:30:49.549 INFO:journalctl@ceph.mds.f.smithi183.stdout:Mar 26 08:30:49 smithi183 podman[66213]: 2024-03-26 08:30:49.119149709 +0000 UTC m=+0.446777332 container start 321fef0effb14b8de5d66fe5421c5e4391c01cc7acd8e34f859d70c02b2334ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-f, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T08:30:49.549 INFO:journalctl@ceph.mds.f.smithi183.stdout:Mar 26 08:30:49 smithi183 ceph-mds[66238]: -- 172.21.15.183:0/3984905738 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5557968dba40 con 0x5557968af000 2024-03-26T08:30:49.549 INFO:journalctl@ceph.mds.f.smithi183.stdout:Mar 26 08:30:49 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-f[66232]: starting mds.f at 2024-03-26T08:30:49.549 INFO:journalctl@ceph.mds.f.smithi183.stdout:Mar 26 08:30:49 smithi183 bash[66213]: 321fef0effb14b8de5d66fe5421c5e4391c01cc7acd8e34f859d70c02b2334ba 2024-03-26T08:30:49.549 INFO:journalctl@ceph.mds.f.smithi183.stdout:Mar 26 08:30:49 smithi183 systemd[1]: Started Ceph mds.f for 88759420-eb49-11ee-95d0-87774f69a715. 2024-03-26T08:30:49.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.585+0000 7f66cb400640 1 -- 172.21.15.43:0/4268657868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66c402d490 con 0x7f66c4104ad0 2024-03-26T08:30:49.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.586+0000 7f66c1600640 1 -- 172.21.15.43:0/4268657868 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4489 (secure 0 0 0) 0x7f66b00be4e0 con 0x7f66c4104ad0 2024-03-26T08:30:49.579 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:30:49.580 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1452206807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1452206807},{"type":"v1","addr":"172.21.15.43:6835","nonce":1452206807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3151775467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3151775467},{"type":"v1","addr":"172.21.15.43:6837","nonce":3151775467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34448,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/1108327179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108327179},{"type":"v1","addr":"172.21.15.52:6833","nonce":1108327179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34487,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/742057410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":742057410},{"type":"v1","addr":"172.21.15.52:6835","nonce":742057410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-26T08:30:49.580 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 6 2024-03-26T08:30:49.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.589+0000 7f66cb400640 1 -- 172.21.15.43:0/4268657868 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f66ac0a2800 msgr2=0x7f66ac0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:49.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.589+0000 7f66cb400640 1 --2- 172.21.15.43:0/4268657868 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f66ac0a2800 0x7f66ac0a4cc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f66b8005fd0 tx=0x7f66b800b410 comp rx=0 tx=0).stop 2024-03-26T08:30:49.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.589+0000 7f66cb400640 1 -- 172.21.15.43:0/4268657868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c4104ad0 msgr2=0x7f66c419a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:49.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.589+0000 7f66cb400640 1 --2- 172.21.15.43:0/4268657868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c4104ad0 0x7f66c419a070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f66b0000c00 tx=0x7f66b00042f0 comp rx=0 tx=0).stop 2024-03-26T08:30:49.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.590+0000 7f66cb400640 1 -- 172.21.15.43:0/4268657868 shutdown_connections 2024-03-26T08:30:49.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.590+0000 7f66cb400640 1 --2- 172.21.15.43:0/4268657868 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f66ac0a2800 0x7f66ac0a4cc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:49.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.590+0000 7f66cb400640 1 --2- 172.21.15.43:0/4268657868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66c4105490 0x7f66c41a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:49.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.590+0000 7f66cb400640 1 --2- 172.21.15.43:0/4268657868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c4104ad0 0x7f66c419a070 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:49.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.590+0000 7f66cb400640 1 --2- 172.21.15.43:0/4268657868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66c41038e0 0x7f66c4199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:49.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.590+0000 7f66cb400640 1 -- 172.21.15.43:0/4268657868 >> 172.21.15.43:0/4268657868 conn(0x7f66c40fee90 msgr2=0x7f66c4107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:49.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.590+0000 7f66cb400640 1 -- 172.21.15.43:0/4268657868 shutdown_connections 2024-03-26T08:30:49.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:49.590+0000 7f66cb400640 1 -- 172.21.15.43:0/4268657868 wait complete. 2024-03-26T08:30:50.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:49 smithi043 ceph-mon[31929]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:50.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:49 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:50.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:49 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:50.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:49 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:50.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:49 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:30:50.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:49 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:30:50.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:49 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:50.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:49 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/4268657868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:50.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:49 smithi183 ceph-mon[35818]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:50.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:49 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:50.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:49 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:50.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:49 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:50.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:49 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:30:50.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:49 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:30:50.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:49 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:50.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:49 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/4268657868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:50.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:49 smithi052 ceph-mon[35071]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:50.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:49 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:50.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:49 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:50.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:49 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:50.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:49 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:30:50.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:49 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:30:50.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:49 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:50.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:49 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/4268657868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:51.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:50 smithi183 ceph-mon[35818]: Deploying daemon mds.c on smithi183 2024-03-26T08:30:51.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:50 smithi183 ceph-mon[35818]: mds.? [v2:172.21.15.183:6834/1842295306,v1:172.21.15.183:6835/1842295306] up:boot 2024-03-26T08:30:51.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:50 smithi183 ceph-mon[35818]: fsmap 6 up:standby 2024-03-26T08:30:51.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:50 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T08:30:51.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:50 smithi043 ceph-mon[31929]: Deploying daemon mds.c on smithi183 2024-03-26T08:30:51.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:50 smithi043 ceph-mon[31929]: mds.? [v2:172.21.15.183:6834/1842295306,v1:172.21.15.183:6835/1842295306] up:boot 2024-03-26T08:30:51.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:50 smithi043 ceph-mon[31929]: fsmap 6 up:standby 2024-03-26T08:30:51.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:50 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T08:30:51.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:50 smithi052 ceph-mon[35071]: Deploying daemon mds.c on smithi183 2024-03-26T08:30:51.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:50 smithi052 ceph-mon[35071]: mds.? [v2:172.21.15.183:6834/1842295306,v1:172.21.15.183:6835/1842295306] up:boot 2024-03-26T08:30:51.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:50 smithi052 ceph-mon[35071]: fsmap 6 up:standby 2024-03-26T08:30:51.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:50 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T08:30:51.605 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:30:51.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.850+0000 7f09e2a00640 1 -- 172.21.15.43:0/416265618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09dc101500 msgr2=0x7f09dc101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:51.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.850+0000 7f09e2a00640 1 --2- 172.21.15.43:0/416265618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09dc101500 0x7f09dc101920 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009900 tx=0x7f09d002bc30 comp rx=0 tx=0).stop 2024-03-26T08:30:51.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.851+0000 7f09e2a00640 1 -- 172.21.15.43:0/416265618 shutdown_connections 2024-03-26T08:30:51.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.851+0000 7f09e2a00640 1 --2- 172.21.15.43:0/416265618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09dc1030b0 0x7f09dc109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:51.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.851+0000 7f09e2a00640 1 --2- 172.21.15.43:0/416265618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09dc1026f0 0x7f09dc102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:51.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.851+0000 7f09e2a00640 1 --2- 172.21.15.43:0/416265618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09dc101500 0x7f09dc101920 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:51.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.851+0000 7f09e2a00640 1 -- 172.21.15.43:0/416265618 >> 172.21.15.43:0/416265618 conn(0x7f09dc0fca70 msgr2=0x7f09dc0feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:51.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.851+0000 7f09e2a00640 1 -- 172.21.15.43:0/416265618 shutdown_connections 2024-03-26T08:30:51.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.851+0000 7f09e2a00640 1 -- 172.21.15.43:0/416265618 wait complete. 2024-03-26T08:30:51.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.852+0000 7f09e2a00640 1 Processor -- start 2024-03-26T08:30:51.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.852+0000 7f09e2a00640 1 -- start start 2024-03-26T08:30:51.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.852+0000 7f09e2a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09dc101500 0x7f09dc199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:51.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.852+0000 7f09e2a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09dc1026f0 0x7f09dc199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:51.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.852+0000 7f09e2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09dc1030b0 0x7f09dc1a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:51.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.852+0000 7f09e2a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc10d090 con 0x7f09dc1030b0 2024-03-26T08:30:51.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.852+0000 7f09e2a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc10cf10 con 0x7f09dc1026f0 2024-03-26T08:30:51.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.852+0000 7f09e2a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc10d210 con 0x7f09dc101500 2024-03-26T08:30:51.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.853+0000 7f09e2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09dc1030b0 0x7f09dc1a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:51.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.853+0000 7f09e1600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09dc101500 0x7f09dc199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:51.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.853+0000 7f09e2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09dc1030b0 0x7f09dc1a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44544/0 (socket says 172.21.15.43:44544) 2024-03-26T08:30:51.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.853+0000 7f09e2000640 1 -- 172.21.15.43:0/1400157846 learned_addr learned my addr 172.21.15.43:0/1400157846 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:30:51.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.853+0000 7f09e0c00640 1 --2- 172.21.15.43:0/1400157846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09dc1026f0 0x7f09dc199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:51.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.853+0000 7f09e2000640 1 -- 172.21.15.43:0/1400157846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09dc101500 msgr2=0x7f09dc199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:51.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.853+0000 7f09e2000640 1 --2- 172.21.15.43:0/1400157846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09dc101500 0x7f09dc199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:51.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.853+0000 7f09e2000640 1 -- 172.21.15.43:0/1400157846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09dc1026f0 msgr2=0x7f09dc199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:51.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.853+0000 7f09e2000640 1 --2- 172.21.15.43:0/1400157846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09dc1026f0 0x7f09dc199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:51.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.853+0000 7f09e2000640 1 -- 172.21.15.43:0/1400157846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09dc1a1550 con 0x7f09dc1030b0 2024-03-26T08:30:51.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.853+0000 7f09e2000640 1 --2- 172.21.15.43:0/1400157846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09dc1030b0 0x7f09dc1a0ee0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f09d400eb10 tx=0x7f09d400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:51.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.854+0000 7f09da000640 1 -- 172.21.15.43:0/1400157846 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d4004b20 con 0x7f09dc1030b0 2024-03-26T08:30:51.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.854+0000 7f09da000640 1 -- 172.21.15.43:0/1400157846 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f09d4004cc0 con 0x7f09dc1030b0 2024-03-26T08:30:51.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.854+0000 7f09da000640 1 -- 172.21.15.43:0/1400157846 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d4010690 con 0x7f09dc1030b0 2024-03-26T08:30:51.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.854+0000 7f09e2a00640 1 -- 172.21.15.43:0/1400157846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09dc1a1840 con 0x7f09dc1030b0 2024-03-26T08:30:51.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.854+0000 7f09e2a00640 1 -- 172.21.15.43:0/1400157846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09dc1a1d40 con 0x7f09dc1030b0 2024-03-26T08:30:51.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.855+0000 7f09e2a00640 1 -- 172.21.15.43:0/1400157846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09dc1a2000 con 0x7f09dc1030b0 2024-03-26T08:30:51.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.857+0000 7f09da000640 1 -- 172.21.15.43:0/1400157846 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f09d4005020 con 0x7f09dc1030b0 2024-03-26T08:30:51.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.858+0000 7f09da000640 1 --2- 172.21.15.43:0/1400157846 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f09c40a2880 0x7f09c40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:51.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.859+0000 7f09da000640 1 -- 172.21.15.43:0/1400157846 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f09d40ced50 con 0x7f09dc1030b0 2024-03-26T08:30:51.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.859+0000 7f09e1600640 1 --2- 172.21.15.43:0/1400157846 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f09c40a2880 0x7f09c40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:51.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.859+0000 7f09e1600640 1 --2- 172.21.15.43:0/1400157846 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f09c40a2880 0x7f09c40a4d40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f09d002c240 tx=0x7f09d002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:51.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:51.863+0000 7f09da000640 1 -- 172.21.15.43:0/1400157846 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f09d409c1f0 con 0x7f09dc1030b0 2024-03-26T08:30:52.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:52.107+0000 7f09e2a00640 1 -- 172.21.15.43:0/1400157846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09dc1a2270 con 0x7f09dc1030b0 2024-03-26T08:30:52.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:52.108+0000 7f09da000640 1 -- 172.21.15.43:0/1400157846 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5283 (secure 0 0 0) 0x7f09d409ce20 con 0x7f09dc1030b0 2024-03-26T08:30:52.101 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:30:52.101 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1452206807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1452206807},{"type":"v1","addr":"172.21.15.43:6835","nonce":1452206807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3151775467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3151775467},{"type":"v1","addr":"172.21.15.43:6837","nonce":3151775467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1842295306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1842295306},{"type":"v1","addr":"172.21.15.183:6835","nonce":1842295306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":34448,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/1108327179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108327179},{"type":"v1","addr":"172.21.15.52:6833","nonce":1108327179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34487,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/742057410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":742057410},{"type":"v1","addr":"172.21.15.52:6835","nonce":742057410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-26T08:30:52.101 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 7 2024-03-26T08:30:52.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:52.110+0000 7f09e2a00640 1 -- 172.21.15.43:0/1400157846 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f09c40a2880 msgr2=0x7f09c40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:52.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:52.110+0000 7f09e2a00640 1 --2- 172.21.15.43:0/1400157846 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f09c40a2880 0x7f09c40a4d40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f09d002c240 tx=0x7f09d002c7e0 comp rx=0 tx=0).stop 2024-03-26T08:30:52.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:52.110+0000 7f09e2a00640 1 -- 172.21.15.43:0/1400157846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09dc1030b0 msgr2=0x7f09dc1a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:52.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:52.110+0000 7f09e2a00640 1 --2- 172.21.15.43:0/1400157846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09dc1030b0 0x7f09dc1a0ee0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f09d400eb10 tx=0x7f09d400c2d0 comp rx=0 tx=0).stop 2024-03-26T08:30:52.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:52.110+0000 7f09e2a00640 1 -- 172.21.15.43:0/1400157846 shutdown_connections 2024-03-26T08:30:52.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:52.110+0000 7f09e2a00640 1 --2- 172.21.15.43:0/1400157846 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f09c40a2880 0x7f09c40a4d40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:52.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:52.110+0000 7f09e2a00640 1 --2- 172.21.15.43:0/1400157846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09dc1030b0 0x7f09dc1a0ee0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:52.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:52.110+0000 7f09e2a00640 1 --2- 172.21.15.43:0/1400157846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09dc1026f0 0x7f09dc199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:52.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:52.110+0000 7f09e2a00640 1 --2- 172.21.15.43:0/1400157846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09dc101500 0x7f09dc199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:52.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:52.110+0000 7f09e2a00640 1 -- 172.21.15.43:0/1400157846 >> 172.21.15.43:0/1400157846 conn(0x7f09dc0fca70 msgr2=0x7f09dc107ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:52.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:52.111+0000 7f09e2a00640 1 -- 172.21.15.43:0/1400157846 shutdown_connections 2024-03-26T08:30:52.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:52.111+0000 7f09e2a00640 1 -- 172.21.15.43:0/1400157846 wait complete. 2024-03-26T08:30:52.120 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:51 smithi043 ceph-mon[31929]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:52.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:51 smithi183 ceph-mon[35818]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:52.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:51 smithi052 ceph-mon[35071]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:52.669 INFO:journalctl@ceph.mds.c.smithi183.stdout:Mar 26 08:30:52 smithi183 systemd[1]: Starting Ceph mds.c for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:30:52.964 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:52 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1400157846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:52.965 INFO:journalctl@ceph.mds.c.smithi183.stdout:Mar 26 08:30:52 smithi183 podman[66520]: 2024-03-26 08:30:52.580437855 +0000 UTC m=+0.032067288 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:30:52.965 INFO:journalctl@ceph.mds.c.smithi183.stdout:Mar 26 08:30:52 smithi183 podman[66520]: 2024-03-26 08:30:52.701000564 +0000 UTC m=+0.152630054 container create acfb6a572373cc49196183f38311a5ae1fe3b239aa504e30574431be71ff2e82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T08:30:52.965 INFO:journalctl@ceph.mds.c.smithi183.stdout:Mar 26 08:30:52 smithi183 podman[66520]: 2024-03-26 08:30:52.97172254 +0000 UTC m=+0.423352016 container init acfb6a572373cc49196183f38311a5ae1fe3b239aa504e30574431be71ff2e82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-26T08:30:53.299 INFO:journalctl@ceph.mds.c.smithi183.stdout:Mar 26 08:30:52 smithi183 podman[66520]: 2024-03-26 08:30:52.983168021 +0000 UTC m=+0.434797466 container start acfb6a572373cc49196183f38311a5ae1fe3b239aa504e30574431be71ff2e82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319) 2024-03-26T08:30:53.299 INFO:journalctl@ceph.mds.c.smithi183.stdout:Mar 26 08:30:53 smithi183 ceph-mds[66539]: -- 172.21.15.183:0/2275578173 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561425efda40 con 0x561425ed1000 2024-03-26T08:30:53.299 INFO:journalctl@ceph.mds.c.smithi183.stdout:Mar 26 08:30:53 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-c[66535]: starting mds.c at 2024-03-26T08:30:53.299 INFO:journalctl@ceph.mds.c.smithi183.stdout:Mar 26 08:30:53 smithi183 bash[66520]: acfb6a572373cc49196183f38311a5ae1fe3b239aa504e30574431be71ff2e82 2024-03-26T08:30:53.299 INFO:journalctl@ceph.mds.c.smithi183.stdout:Mar 26 08:30:53 smithi183 systemd[1]: Started Ceph mds.c for 88759420-eb49-11ee-95d0-87774f69a715. 2024-03-26T08:30:53.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:52 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1400157846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:53.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:52 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1400157846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:54.124 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:30:54.219 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:53 smithi043 ceph-mon[31929]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:54.219 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:53 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:54.219 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:53 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:54.219 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:53 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:54.219 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:53 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:30:54.219 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:53 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:30:54.219 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:53 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:54.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:53 smithi183 ceph-mon[35818]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:54.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:53 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:54.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:53 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:54.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:53 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:54.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:53 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:30:54.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:53 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:30:54.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:53 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:54.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:53 smithi052 ceph-mon[35071]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:54.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:53 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:54.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:53 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:54.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:53 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:54.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:53 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:30:54.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:53 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:30:54.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:53 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:54.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.370+0000 7feec8400640 1 -- 172.21.15.43:0/3260218067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feec0107ce0 msgr2=0x7feec0108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:54.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.370+0000 7feec8400640 1 --2- 172.21.15.43:0/3260218067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feec0107ce0 0x7feec0108100 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009900 tx=0x7feeb402bc30 comp rx=0 tx=0).stop 2024-03-26T08:30:54.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.371+0000 7feec8400640 1 -- 172.21.15.43:0/3260218067 shutdown_connections 2024-03-26T08:30:54.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.371+0000 7feec8400640 1 --2- 172.21.15.43:0/3260218067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feec0109890 0x7feec0110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:54.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.371+0000 7feec8400640 1 --2- 172.21.15.43:0/3260218067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feec0108ed0 0x7feec0109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:54.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.371+0000 7feec8400640 1 --2- 172.21.15.43:0/3260218067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feec0107ce0 0x7feec0108100 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:54.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.371+0000 7feec8400640 1 -- 172.21.15.43:0/3260218067 >> 172.21.15.43:0/3260218067 conn(0x7feec0076cb0 msgr2=0x7feec00790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:54.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.371+0000 7feec8400640 1 -- 172.21.15.43:0/3260218067 shutdown_connections 2024-03-26T08:30:54.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.371+0000 7feec8400640 1 -- 172.21.15.43:0/3260218067 wait complete. 2024-03-26T08:30:54.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.372+0000 7feec8400640 1 Processor -- start 2024-03-26T08:30:54.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.372+0000 7feec8400640 1 -- start start 2024-03-26T08:30:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.372+0000 7feec8400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feec0107ce0 0x7feec019df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.372+0000 7feec8400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feec0108ed0 0x7feec019e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.372+0000 7feec8400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feec0109890 0x7feec01a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.372+0000 7feec8400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec0113830 con 0x7feec0109890 2024-03-26T08:30:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.372+0000 7feec8400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec01136b0 con 0x7feec0107ce0 2024-03-26T08:30:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.372+0000 7feec8400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec01139b0 con 0x7feec0108ed0 2024-03-26T08:30:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.373+0000 7feec6400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feec0109890 0x7feec01a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.373+0000 7feec6400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feec0109890 0x7feec01a5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55776/0 (socket says 172.21.15.43:55776) 2024-03-26T08:30:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.373+0000 7feec6400640 1 -- 172.21.15.43:0/1304646989 learned_addr learned my addr 172.21.15.43:0/1304646989 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:30:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.373+0000 7feec5a00640 1 --2- 172.21.15.43:0/1304646989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feec0107ce0 0x7feec019df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.373+0000 7feec5000640 1 --2- 172.21.15.43:0/1304646989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feec0108ed0 0x7feec019e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.373+0000 7feec6400640 1 -- 172.21.15.43:0/1304646989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feec0108ed0 msgr2=0x7feec019e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.373+0000 7feec6400640 1 --2- 172.21.15.43:0/1304646989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feec0108ed0 0x7feec019e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.373+0000 7feec6400640 1 -- 172.21.15.43:0/1304646989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feec0107ce0 msgr2=0x7feec019df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.373+0000 7feec6400640 1 --2- 172.21.15.43:0/1304646989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feec0107ce0 0x7feec019df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.373+0000 7feec6400640 1 -- 172.21.15.43:0/1304646989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec01a5b80 con 0x7feec0109890 2024-03-26T08:30:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.373+0000 7feec5a00640 1 --2- 172.21.15.43:0/1304646989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feec0107ce0 0x7feec019df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:30:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.373+0000 7feec6400640 1 --2- 172.21.15.43:0/1304646989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feec0109890 0x7feec01a5510 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feeb800cad0 tx=0x7feeb800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.374+0000 7feebe000640 1 -- 172.21.15.43:0/1304646989 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb800ae20 con 0x7feec0109890 2024-03-26T08:30:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.374+0000 7feebe000640 1 -- 172.21.15.43:0/1304646989 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7feeb8005020 con 0x7feec0109890 2024-03-26T08:30:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.374+0000 7feebe000640 1 -- 172.21.15.43:0/1304646989 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb800f6b0 con 0x7feec0109890 2024-03-26T08:30:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.374+0000 7feec8400640 1 -- 172.21.15.43:0/1304646989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec01a5e70 con 0x7feec0109890 2024-03-26T08:30:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.374+0000 7feec8400640 1 -- 172.21.15.43:0/1304646989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec01a6330 con 0x7feec0109890 2024-03-26T08:30:54.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.375+0000 7feec8400640 1 -- 172.21.15.43:0/1304646989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec010f080 con 0x7feec0109890 2024-03-26T08:30:54.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.377+0000 7feebe000640 1 -- 172.21.15.43:0/1304646989 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7feeb80051d0 con 0x7feec0109890 2024-03-26T08:30:54.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.378+0000 7feebe000640 1 --2- 172.21.15.43:0/1304646989 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7feea80a2880 0x7feea80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:54.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.379+0000 7feebe000640 1 -- 172.21.15.43:0/1304646989 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7feeb80ce710 con 0x7feec0109890 2024-03-26T08:30:54.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.379+0000 7feec5a00640 1 --2- 172.21.15.43:0/1304646989 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7feea80a2880 0x7feea80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:54.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.379+0000 7feec5a00640 1 --2- 172.21.15.43:0/1304646989 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7feea80a2880 0x7feea80a4d40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feeb402c240 tx=0x7feeb4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:54.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.383+0000 7feebe000640 1 -- 172.21.15.43:0/1304646989 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7feeb8014030 con 0x7feec0109890 2024-03-26T08:30:54.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.628+0000 7feec8400640 1 -- 172.21.15.43:0/1304646989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feec002d070 con 0x7feec0109890 2024-03-26T08:30:54.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.629+0000 7feebe000640 1 -- 172.21.15.43:0/1304646989 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6077 (secure 0 0 0) 0x7feeb809bbb0 con 0x7feec0109890 2024-03-26T08:30:54.623 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:30:54.623 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1452206807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1452206807},{"type":"v1","addr":"172.21.15.43:6835","nonce":1452206807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3151775467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3151775467},{"type":"v1","addr":"172.21.15.43:6837","nonce":3151775467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1842295306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1842295306},{"type":"v1","addr":"172.21.15.183:6835","nonce":1842295306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2267781666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2267781666},{"type":"v1","addr":"172.21.15.183:6837","nonce":2267781666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":34448,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/1108327179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108327179},{"type":"v1","addr":"172.21.15.52:6833","nonce":1108327179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34487,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/742057410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":742057410},{"type":"v1","addr":"172.21.15.52:6835","nonce":742057410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-26T08:30:54.623 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 8 2024-03-26T08:30:54.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.633+0000 7feec8400640 1 -- 172.21.15.43:0/1304646989 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7feea80a2880 msgr2=0x7feea80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:54.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.633+0000 7feec8400640 1 --2- 172.21.15.43:0/1304646989 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7feea80a2880 0x7feea80a4d40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feeb402c240 tx=0x7feeb4005fd0 comp rx=0 tx=0).stop 2024-03-26T08:30:54.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.633+0000 7feec8400640 1 -- 172.21.15.43:0/1304646989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feec0109890 msgr2=0x7feec01a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:54.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.633+0000 7feec8400640 1 --2- 172.21.15.43:0/1304646989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feec0109890 0x7feec01a5510 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feeb800cad0 tx=0x7feeb800cf90 comp rx=0 tx=0).stop 2024-03-26T08:30:54.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.633+0000 7feec8400640 1 -- 172.21.15.43:0/1304646989 shutdown_connections 2024-03-26T08:30:54.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.633+0000 7feec8400640 1 --2- 172.21.15.43:0/1304646989 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7feea80a2880 0x7feea80a4d40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:54.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.633+0000 7feec8400640 1 --2- 172.21.15.43:0/1304646989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feec0109890 0x7feec01a5510 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:54.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.633+0000 7feec8400640 1 --2- 172.21.15.43:0/1304646989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feec0108ed0 0x7feec019e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:54.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.633+0000 7feec8400640 1 --2- 172.21.15.43:0/1304646989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feec0107ce0 0x7feec019df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:54.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.633+0000 7feec8400640 1 -- 172.21.15.43:0/1304646989 >> 172.21.15.43:0/1304646989 conn(0x7feec0076cb0 msgr2=0x7feec010c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:54.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.633+0000 7feec8400640 1 -- 172.21.15.43:0/1304646989 shutdown_connections 2024-03-26T08:30:54.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:54.633+0000 7feec8400640 1 -- 172.21.15.43:0/1304646989 wait complete. 2024-03-26T08:30:55.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:54 smithi052 ceph-mon[35071]: Deploying daemon mds.h on smithi052 2024-03-26T08:30:55.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:54 smithi052 ceph-mon[35071]: mds.? [v2:172.21.15.183:6836/2267781666,v1:172.21.15.183:6837/2267781666] up:boot 2024-03-26T08:30:55.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:54 smithi052 ceph-mon[35071]: fsmap 7 up:standby 2024-03-26T08:30:55.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:54 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T08:30:55.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:54 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1304646989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:55.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:54 smithi183 ceph-mon[35818]: Deploying daemon mds.h on smithi052 2024-03-26T08:30:55.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:54 smithi183 ceph-mon[35818]: mds.? [v2:172.21.15.183:6836/2267781666,v1:172.21.15.183:6837/2267781666] up:boot 2024-03-26T08:30:55.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:54 smithi183 ceph-mon[35818]: fsmap 7 up:standby 2024-03-26T08:30:55.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:54 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T08:30:55.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:54 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1304646989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:55.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:54 smithi043 ceph-mon[31929]: Deploying daemon mds.h on smithi052 2024-03-26T08:30:55.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:54 smithi043 ceph-mon[31929]: mds.? [v2:172.21.15.183:6836/2267781666,v1:172.21.15.183:6837/2267781666] up:boot 2024-03-26T08:30:55.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:54 smithi043 ceph-mon[31929]: fsmap 7 up:standby 2024-03-26T08:30:55.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:54 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T08:30:55.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:54 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1304646989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:56.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:55 smithi183 ceph-mon[35818]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:56.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:55 smithi052 ceph-mon[35071]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:56.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:55 smithi043 ceph-mon[31929]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:56.647 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:30:56.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.886+0000 7f59c3c00640 1 -- 172.21.15.43:0/1000885546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59bc109890 msgr2=0x7f59bc110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:56.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.886+0000 7f59c3c00640 1 --2- 172.21.15.43:0/1000885546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59bc109890 0x7f59bc110120 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009ca0 tx=0x7f59b402c100 comp rx=0 tx=0).stop 2024-03-26T08:30:56.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.886+0000 7f59c3c00640 1 -- 172.21.15.43:0/1000885546 shutdown_connections 2024-03-26T08:30:56.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.886+0000 7f59c3c00640 1 --2- 172.21.15.43:0/1000885546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59bc109890 0x7f59bc110120 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:56.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.886+0000 7f59c3c00640 1 --2- 172.21.15.43:0/1000885546 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59bc108ed0 0x7f59bc109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:56.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.886+0000 7f59c3c00640 1 --2- 172.21.15.43:0/1000885546 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59bc107ce0 0x7f59bc108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:56.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.886+0000 7f59c3c00640 1 -- 172.21.15.43:0/1000885546 >> 172.21.15.43:0/1000885546 conn(0x7f59bc076cb0 msgr2=0x7f59bc0790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:56.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.886+0000 7f59c3c00640 1 -- 172.21.15.43:0/1000885546 shutdown_connections 2024-03-26T08:30:56.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.886+0000 7f59c3c00640 1 -- 172.21.15.43:0/1000885546 wait complete. 2024-03-26T08:30:56.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.887+0000 7f59c3c00640 1 Processor -- start 2024-03-26T08:30:56.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.888+0000 7f59c3c00640 1 -- start start 2024-03-26T08:30:56.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.888+0000 7f59c3c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59bc107ce0 0x7f59bc19df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:56.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.888+0000 7f59c3c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59bc108ed0 0x7f59bc19e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:56.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.888+0000 7f59c3c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59bc109890 0x7f59bc1a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:56.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.888+0000 7f59c3c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc113810 con 0x7f59bc109890 2024-03-26T08:30:56.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.888+0000 7f59c3c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc113690 con 0x7f59bc108ed0 2024-03-26T08:30:56.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.888+0000 7f59c3c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc113990 con 0x7f59bc107ce0 2024-03-26T08:30:56.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.888+0000 7f59c1c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59bc109890 0x7f59bc1a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:56.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.888+0000 7f59c1c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59bc109890 0x7f59bc1a5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55804/0 (socket says 172.21.15.43:55804) 2024-03-26T08:30:56.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.888+0000 7f59c1c00640 1 -- 172.21.15.43:0/4100896051 learned_addr learned my addr 172.21.15.43:0/4100896051 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:30:56.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.888+0000 7f59bbe00640 1 --2- 172.21.15.43:0/4100896051 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59bc108ed0 0x7f59bc19e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:56.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.889+0000 7f59c1200640 1 --2- 172.21.15.43:0/4100896051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59bc107ce0 0x7f59bc19df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:56.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.889+0000 7f59c1c00640 1 -- 172.21.15.43:0/4100896051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59bc107ce0 msgr2=0x7f59bc19df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:56.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.889+0000 7f59c1c00640 1 --2- 172.21.15.43:0/4100896051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59bc107ce0 0x7f59bc19df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:56.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.889+0000 7f59c1c00640 1 -- 172.21.15.43:0/4100896051 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59bc108ed0 msgr2=0x7f59bc19e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:56.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.889+0000 7f59c1c00640 1 --2- 172.21.15.43:0/4100896051 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59bc108ed0 0x7f59bc19e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:56.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.889+0000 7f59c1c00640 1 -- 172.21.15.43:0/4100896051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59bc1a5ad0 con 0x7f59bc109890 2024-03-26T08:30:56.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.889+0000 7f59c1c00640 1 --2- 172.21.15.43:0/4100896051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59bc109890 0x7f59bc1a5460 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f59b4032520 tx=0x7f59b4035ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:56.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.890+0000 7f59b9600640 1 -- 172.21.15.43:0/4100896051 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b4039040 con 0x7f59bc109890 2024-03-26T08:30:56.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.890+0000 7f59c3c00640 1 -- 172.21.15.43:0/4100896051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59bc1a5d60 con 0x7f59bc109890 2024-03-26T08:30:56.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.890+0000 7f59b9600640 1 -- 172.21.15.43:0/4100896051 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f59b4044070 con 0x7f59bc109890 2024-03-26T08:30:56.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.890+0000 7f59b9600640 1 -- 172.21.15.43:0/4100896051 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b40043c0 con 0x7f59bc109890 2024-03-26T08:30:56.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.890+0000 7f59c3c00640 1 -- 172.21.15.43:0/4100896051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59bc1a6240 con 0x7f59bc109890 2024-03-26T08:30:56.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.892+0000 7f59c3c00640 1 -- 172.21.15.43:0/4100896051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59bc10f000 con 0x7f59bc109890 2024-03-26T08:30:56.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.893+0000 7f59b9600640 1 -- 172.21.15.43:0/4100896051 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f59b4004560 con 0x7f59bc109890 2024-03-26T08:30:56.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.895+0000 7f59b9600640 1 --2- 172.21.15.43:0/4100896051 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f59a40a2800 0x7f59a40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:56.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.895+0000 7f59b9600640 1 -- 172.21.15.43:0/4100896051 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f59b40f0dc0 con 0x7f59bc109890 2024-03-26T08:30:56.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.895+0000 7f59c1200640 1 --2- 172.21.15.43:0/4100896051 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f59a40a2800 0x7f59a40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:56.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.896+0000 7f59c1200640 1 --2- 172.21.15.43:0/4100896051 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f59a40a2800 0x7f59a40a4cc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f59b0005fd0 tx=0x7f59b0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:56.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:56.900+0000 7f59b9600640 1 -- 172.21.15.43:0/4100896051 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f59b40be320 con 0x7f59bc109890 2024-03-26T08:30:57.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:57.146+0000 7f59c3c00640 1 -- 172.21.15.43:0/4100896051 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59bc02d070 con 0x7f59bc109890 2024-03-26T08:30:57.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:57.147+0000 7f59b9600640 1 -- 172.21.15.43:0/4100896051 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6077 (secure 0 0 0) 0x7f59b40a8090 con 0x7f59bc109890 2024-03-26T08:30:57.140 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:30:57.141 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1452206807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1452206807},{"type":"v1","addr":"172.21.15.43:6835","nonce":1452206807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3151775467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3151775467},{"type":"v1","addr":"172.21.15.43:6837","nonce":3151775467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1842295306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1842295306},{"type":"v1","addr":"172.21.15.183:6835","nonce":1842295306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2267781666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2267781666},{"type":"v1","addr":"172.21.15.183:6837","nonce":2267781666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":34448,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/1108327179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108327179},{"type":"v1","addr":"172.21.15.52:6833","nonce":1108327179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34487,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/742057410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":742057410},{"type":"v1","addr":"172.21.15.52:6835","nonce":742057410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-26T08:30:57.141 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 8 2024-03-26T08:30:57.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:57.150+0000 7f59c3c00640 1 -- 172.21.15.43:0/4100896051 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f59a40a2800 msgr2=0x7f59a40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:57.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:57.150+0000 7f59c3c00640 1 --2- 172.21.15.43:0/4100896051 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f59a40a2800 0x7f59a40a4cc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f59b0005fd0 tx=0x7f59b0005850 comp rx=0 tx=0).stop 2024-03-26T08:30:57.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:57.150+0000 7f59c3c00640 1 -- 172.21.15.43:0/4100896051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59bc109890 msgr2=0x7f59bc1a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:57.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:57.150+0000 7f59c3c00640 1 --2- 172.21.15.43:0/4100896051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59bc109890 0x7f59bc1a5460 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f59b4032520 tx=0x7f59b4035ce0 comp rx=0 tx=0).stop 2024-03-26T08:30:57.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:57.150+0000 7f59c3c00640 1 -- 172.21.15.43:0/4100896051 shutdown_connections 2024-03-26T08:30:57.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:57.150+0000 7f59c3c00640 1 --2- 172.21.15.43:0/4100896051 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f59a40a2800 0x7f59a40a4cc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:57.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:57.150+0000 7f59c3c00640 1 --2- 172.21.15.43:0/4100896051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59bc109890 0x7f59bc1a5460 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:57.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:57.150+0000 7f59c3c00640 1 --2- 172.21.15.43:0/4100896051 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59bc108ed0 0x7f59bc19e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:57.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:57.150+0000 7f59c3c00640 1 --2- 172.21.15.43:0/4100896051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59bc107ce0 0x7f59bc19df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:57.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:57.150+0000 7f59c3c00640 1 -- 172.21.15.43:0/4100896051 >> 172.21.15.43:0/4100896051 conn(0x7f59bc076cb0 msgr2=0x7f59bc10c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:57.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:57.151+0000 7f59c3c00640 1 -- 172.21.15.43:0/4100896051 shutdown_connections 2024-03-26T08:30:57.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:57.151+0000 7f59c3c00640 1 -- 172.21.15.43:0/4100896051 wait complete. 2024-03-26T08:30:57.219 INFO:journalctl@ceph.mds.h.smithi052.stdout:Mar 26 08:30:56 smithi052 systemd[1]: Starting Ceph mds.h for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:30:57.531 INFO:journalctl@ceph.mds.h.smithi052.stdout:Mar 26 08:30:57 smithi052 podman[65271]: 2024-03-26 08:30:57.135408573 +0000 UTC m=+0.040111950 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:30:57.531 INFO:journalctl@ceph.mds.h.smithi052.stdout:Mar 26 08:30:57 smithi052 podman[65271]: 2024-03-26 08:30:57.25574542 +0000 UTC m=+0.160448828 container create c09851e5d51e031a915391579f43c55527bf9a1ad5b82d615ecd060b5c74e5e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-26T08:30:57.531 INFO:journalctl@ceph.mds.h.smithi052.stdout:Mar 26 08:30:57 smithi052 podman[65271]: 2024-03-26 08:30:57.537067892 +0000 UTC m=+0.441771336 container init c09851e5d51e031a915391579f43c55527bf9a1ad5b82d615ecd060b5c74e5e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-h, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-26T08:30:57.823 INFO:journalctl@ceph.mds.h.smithi052.stdout:Mar 26 08:30:57 smithi052 podman[65271]: 2024-03-26 08:30:57.547739248 +0000 UTC m=+0.452442630 container start c09851e5d51e031a915391579f43c55527bf9a1ad5b82d615ecd060b5c74e5e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-h, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T08:30:57.823 INFO:journalctl@ceph.mds.h.smithi052.stdout:Mar 26 08:30:57 smithi052 ceph-mds[65290]: -- 172.21.15.52:0/620455218 <== mon.2 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55da34eafa40 con 0x55da34e82c00 2024-03-26T08:30:57.823 INFO:journalctl@ceph.mds.h.smithi052.stdout:Mar 26 08:30:57 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-h[65286]: starting mds.h at 2024-03-26T08:30:57.823 INFO:journalctl@ceph.mds.h.smithi052.stdout:Mar 26 08:30:57 smithi052 bash[65271]: c09851e5d51e031a915391579f43c55527bf9a1ad5b82d615ecd060b5c74e5e0 2024-03-26T08:30:57.823 INFO:journalctl@ceph.mds.h.smithi052.stdout:Mar 26 08:30:57 smithi052 systemd[1]: Started Ceph mds.h for 88759420-eb49-11ee-95d0-87774f69a715. 2024-03-26T08:30:58.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:57 smithi183 ceph-mon[35818]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:58.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:57 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/4100896051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:58.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:57 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:58.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:57 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:58.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:57 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:58.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:57 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:30:58.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:57 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:30:58.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:57 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:58.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:57 smithi052 ceph-mon[35071]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:58.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:57 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/4100896051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:58.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:57 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:58.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:57 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:58.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:57 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:58.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:57 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:30:58.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:57 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:30:58.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:57 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:58.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:57 smithi043 ceph-mon[31929]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:30:58.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:57 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/4100896051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:30:58.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:57 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:58.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:57 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:58.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:57 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:30:58.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:57 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:30:58.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:57 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:30:58.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:57 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:30:59.165 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:30:59.192 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:58 smithi183 ceph-mon[35818]: Deploying daemon mds.i on smithi183 2024-03-26T08:30:59.193 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:58 smithi183 ceph-mon[35818]: mds.? [v2:172.21.15.52:6836/748239918,v1:172.21.15.52:6837/748239918] up:boot 2024-03-26T08:30:59.193 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:58 smithi183 ceph-mon[35818]: fsmap 8 up:standby 2024-03-26T08:30:59.193 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:58 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T08:30:59.252 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:58 smithi043 ceph-mon[31929]: Deploying daemon mds.i on smithi183 2024-03-26T08:30:59.252 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:58 smithi043 ceph-mon[31929]: mds.? [v2:172.21.15.52:6836/748239918,v1:172.21.15.52:6837/748239918] up:boot 2024-03-26T08:30:59.252 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:58 smithi043 ceph-mon[31929]: fsmap 8 up:standby 2024-03-26T08:30:59.253 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:58 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T08:30:59.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:58 smithi052 ceph-mon[35071]: Deploying daemon mds.i on smithi183 2024-03-26T08:30:59.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:58 smithi052 ceph-mon[35071]: mds.? [v2:172.21.15.52:6836/748239918,v1:172.21.15.52:6837/748239918] up:boot 2024-03-26T08:30:59.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:58 smithi052 ceph-mon[35071]: fsmap 8 up:standby 2024-03-26T08:30:59.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:58 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T08:30:59.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.401+0000 7fe22b400640 1 -- 172.21.15.43:0/2867893143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2241038e0 msgr2=0x7fe224103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:59.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.401+0000 7fe22b400640 1 --2- 172.21.15.43:0/2867893143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2241038e0 0x7fe224103d00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe218009900 tx=0x7fe21802bc30 comp rx=0 tx=0).stop 2024-03-26T08:30:59.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.401+0000 7fe22b400640 1 -- 172.21.15.43:0/2867893143 shutdown_connections 2024-03-26T08:30:59.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.401+0000 7fe22b400640 1 --2- 172.21.15.43:0/2867893143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe224105490 0x7fe22410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:59.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.401+0000 7fe22b400640 1 --2- 172.21.15.43:0/2867893143 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe224104ad0 0x7fe224104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:59.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.401+0000 7fe22b400640 1 --2- 172.21.15.43:0/2867893143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2241038e0 0x7fe224103d00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:59.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.401+0000 7fe22b400640 1 -- 172.21.15.43:0/2867893143 >> 172.21.15.43:0/2867893143 conn(0x7fe2240fee90 msgr2=0x7fe2241012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:59.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.401+0000 7fe22b400640 1 -- 172.21.15.43:0/2867893143 shutdown_connections 2024-03-26T08:30:59.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.401+0000 7fe22b400640 1 -- 172.21.15.43:0/2867893143 wait complete. 2024-03-26T08:30:59.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.402+0000 7fe22b400640 1 Processor -- start 2024-03-26T08:30:59.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.403+0000 7fe22b400640 1 -- start start 2024-03-26T08:30:59.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.403+0000 7fe22b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2241038e0 0x7fe224199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:59.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.403+0000 7fe22b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe224104ad0 0x7fe22419a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:59.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.403+0000 7fe22b400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe224105490 0x7fe2241a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:59.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.403+0000 7fe22b400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22410f460 con 0x7fe2241038e0 2024-03-26T08:30:59.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.403+0000 7fe22b400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22410f2e0 con 0x7fe224105490 2024-03-26T08:30:59.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.403+0000 7fe22b400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22410f5e0 con 0x7fe224104ad0 2024-03-26T08:30:59.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.403+0000 7fe228a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2241038e0 0x7fe224199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:59.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.403+0000 7fe228a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2241038e0 0x7fe224199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55830/0 (socket says 172.21.15.43:55830) 2024-03-26T08:30:59.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.403+0000 7fe228a00640 1 -- 172.21.15.43:0/276968627 learned_addr learned my addr 172.21.15.43:0/276968627 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:30:59.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.403+0000 7fe229400640 1 --2- 172.21.15.43:0/276968627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe224105490 0x7fe2241a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:59.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.404+0000 7fe223e00640 1 --2- 172.21.15.43:0/276968627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe224104ad0 0x7fe22419a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:59.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.404+0000 7fe228a00640 1 -- 172.21.15.43:0/276968627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe224104ad0 msgr2=0x7fe22419a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:59.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.404+0000 7fe228a00640 1 --2- 172.21.15.43:0/276968627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe224104ad0 0x7fe22419a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:59.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.404+0000 7fe228a00640 1 -- 172.21.15.43:0/276968627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe224105490 msgr2=0x7fe2241a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:59.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.404+0000 7fe228a00640 1 --2- 172.21.15.43:0/276968627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe224105490 0x7fe2241a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:59.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.404+0000 7fe228a00640 1 -- 172.21.15.43:0/276968627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2241a1760 con 0x7fe2241038e0 2024-03-26T08:30:59.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.404+0000 7fe228a00640 1 --2- 172.21.15.43:0/276968627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2241038e0 0x7fe224199b40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe218000c00 tx=0x7fe218005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:59.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.404+0000 7fe221600640 1 -- 172.21.15.43:0/276968627 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe218043020 con 0x7fe2241038e0 2024-03-26T08:30:59.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.404+0000 7fe221600640 1 -- 172.21.15.43:0/276968627 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe2180043d0 con 0x7fe2241038e0 2024-03-26T08:30:59.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.404+0000 7fe221600640 1 -- 172.21.15.43:0/276968627 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe218003b40 con 0x7fe2241038e0 2024-03-26T08:30:59.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.404+0000 7fe22b400640 1 -- 172.21.15.43:0/276968627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2241a19f0 con 0x7fe2241038e0 2024-03-26T08:30:59.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.405+0000 7fe22b400640 1 -- 172.21.15.43:0/276968627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2241a1f40 con 0x7fe2241038e0 2024-03-26T08:30:59.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.406+0000 7fe22b400640 1 -- 172.21.15.43:0/276968627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe22410ac20 con 0x7fe2241038e0 2024-03-26T08:30:59.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.408+0000 7fe221600640 1 -- 172.21.15.43:0/276968627 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fe218046020 con 0x7fe2241038e0 2024-03-26T08:30:59.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.409+0000 7fe221600640 1 --2- 172.21.15.43:0/276968627 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fe20c0a2800 0x7fe20c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:30:59.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.409+0000 7fe221600640 1 -- 172.21.15.43:0/276968627 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe2180f0920 con 0x7fe2241038e0 2024-03-26T08:30:59.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.409+0000 7fe223e00640 1 --2- 172.21.15.43:0/276968627 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fe20c0a2800 0x7fe20c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:30:59.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.410+0000 7fe223e00640 1 --2- 172.21.15.43:0/276968627 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fe20c0a2800 0x7fe20c0a4cc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe22419b050 tx=0x7fe210005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:30:59.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.414+0000 7fe221600640 1 -- 172.21.15.43:0/276968627 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe21804d020 con 0x7fe2241038e0 2024-03-26T08:30:59.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.658+0000 7fe22b400640 1 -- 172.21.15.43:0/276968627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe22402d070 con 0x7fe2241038e0 2024-03-26T08:30:59.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.660+0000 7fe221600640 1 -- 172.21.15.43:0/276968627 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6865 (secure 0 0 0) 0x7fe21803c350 con 0x7fe2241038e0 2024-03-26T08:30:59.654 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:30:59.654 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1452206807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1452206807},{"type":"v1","addr":"172.21.15.43:6835","nonce":1452206807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3151775467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3151775467},{"type":"v1","addr":"172.21.15.43:6837","nonce":3151775467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1842295306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1842295306},{"type":"v1","addr":"172.21.15.183:6835","nonce":1842295306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2267781666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2267781666},{"type":"v1","addr":"172.21.15.183:6837","nonce":2267781666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":34448,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/1108327179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108327179},{"type":"v1","addr":"172.21.15.52:6833","nonce":1108327179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34487,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/742057410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":742057410},{"type":"v1","addr":"172.21.15.52:6835","nonce":742057410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34517,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/748239918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":748239918},{"type":"v1","addr":"172.21.15.52:6837","nonce":748239918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-26T08:30:59.654 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2024-03-26T08:30:59.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.662+0000 7fe22b400640 1 -- 172.21.15.43:0/276968627 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fe20c0a2800 msgr2=0x7fe20c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:59.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.662+0000 7fe22b400640 1 --2- 172.21.15.43:0/276968627 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fe20c0a2800 0x7fe20c0a4cc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe22419b050 tx=0x7fe210005f90 comp rx=0 tx=0).stop 2024-03-26T08:30:59.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.663+0000 7fe22b400640 1 -- 172.21.15.43:0/276968627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2241038e0 msgr2=0x7fe224199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:30:59.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.663+0000 7fe22b400640 1 --2- 172.21.15.43:0/276968627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2241038e0 0x7fe224199b40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe218000c00 tx=0x7fe218005e60 comp rx=0 tx=0).stop 2024-03-26T08:30:59.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.663+0000 7fe22b400640 1 -- 172.21.15.43:0/276968627 shutdown_connections 2024-03-26T08:30:59.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.663+0000 7fe22b400640 1 --2- 172.21.15.43:0/276968627 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fe20c0a2800 0x7fe20c0a4cc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:59.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.663+0000 7fe22b400640 1 --2- 172.21.15.43:0/276968627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe224105490 0x7fe2241a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:59.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.663+0000 7fe22b400640 1 --2- 172.21.15.43:0/276968627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe224104ad0 0x7fe22419a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:59.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.663+0000 7fe22b400640 1 --2- 172.21.15.43:0/276968627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2241038e0 0x7fe224199b40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:30:59.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.663+0000 7fe22b400640 1 -- 172.21.15.43:0/276968627 >> 172.21.15.43:0/276968627 conn(0x7fe2240fee90 msgr2=0x7fe224107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:30:59.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.663+0000 7fe22b400640 1 -- 172.21.15.43:0/276968627 shutdown_connections 2024-03-26T08:30:59.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:30:59.663+0000 7fe22b400640 1 -- 172.21.15.43:0/276968627 wait complete. 2024-03-26T08:31:00.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:59 smithi183 ceph-mon[35818]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:00.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:30:59 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/276968627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:00.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:59 smithi052 ceph-mon[35071]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:00.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:30:59 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/276968627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:00.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:59 smithi043 ceph-mon[31929]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:00.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:30:59 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/276968627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:01.265 INFO:journalctl@ceph.mds.i.smithi183.stdout:Mar 26 08:31:00 smithi183 systemd[1]: Starting Ceph mds.i for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:31:01.544 INFO:journalctl@ceph.mds.i.smithi183.stdout:Mar 26 08:31:01 smithi183 podman[66823]: 2024-03-26 08:31:01.176782443 +0000 UTC m=+0.032000625 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:31:01.544 INFO:journalctl@ceph.mds.i.smithi183.stdout:Mar 26 08:31:01 smithi183 podman[66823]: 2024-03-26 08:31:01.296258112 +0000 UTC m=+0.151476316 container create 756510d7e47eff92ba184919269eab3c7f05ad11143039e0224aa8106134b5ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-i, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-26T08:31:01.544 INFO:journalctl@ceph.mds.i.smithi183.stdout:Mar 26 08:31:01 smithi183 podman[66823]: 2024-03-26 08:31:01.550067557 +0000 UTC m=+0.405285835 container init 756510d7e47eff92ba184919269eab3c7f05ad11143039e0224aa8106134b5ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-i, org.label-schema.build-date=20240319, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-26T08:31:01.675 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:31:01.799 INFO:journalctl@ceph.mds.i.smithi183.stdout:Mar 26 08:31:01 smithi183 podman[66823]: 2024-03-26 08:31:01.558793762 +0000 UTC m=+0.414011953 container start 756510d7e47eff92ba184919269eab3c7f05ad11143039e0224aa8106134b5ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-i, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T08:31:01.799 INFO:journalctl@ceph.mds.i.smithi183.stdout:Mar 26 08:31:01 smithi183 ceph-mds[66842]: -- 172.21.15.183:0/999811612 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5611e0951a40 con 0x5611e0924c00 2024-03-26T08:31:01.799 INFO:journalctl@ceph.mds.i.smithi183.stdout:Mar 26 08:31:01 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-i[66838]: starting mds.i at 2024-03-26T08:31:01.799 INFO:journalctl@ceph.mds.i.smithi183.stdout:Mar 26 08:31:01 smithi183 bash[66823]: 756510d7e47eff92ba184919269eab3c7f05ad11143039e0224aa8106134b5ef 2024-03-26T08:31:01.799 INFO:journalctl@ceph.mds.i.smithi183.stdout:Mar 26 08:31:01 smithi183 systemd[1]: Started Ceph mds.i for 88759420-eb49-11ee-95d0-87774f69a715. 2024-03-26T08:31:01.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.918+0000 7f8639600640 1 -- 172.21.15.43:0/2057578444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8634105490 msgr2=0x7f863410bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:01.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.918+0000 7f8639600640 1 --2- 172.21.15.43:0/2057578444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8634105490 0x7f863410bd20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f862c009900 tx=0x7f862c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:31:01.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.919+0000 7f8639600640 1 -- 172.21.15.43:0/2057578444 shutdown_connections 2024-03-26T08:31:01.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.919+0000 7f8639600640 1 --2- 172.21.15.43:0/2057578444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8634105490 0x7f863410bd20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:01.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.919+0000 7f8639600640 1 --2- 172.21.15.43:0/2057578444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8634104ad0 0x7f8634104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:01.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.919+0000 7f8639600640 1 --2- 172.21.15.43:0/2057578444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86341038e0 0x7f8634103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:01.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.919+0000 7f8639600640 1 -- 172.21.15.43:0/2057578444 >> 172.21.15.43:0/2057578444 conn(0x7f86340fee50 msgr2=0x7f8634101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:01.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.919+0000 7f8639600640 1 -- 172.21.15.43:0/2057578444 shutdown_connections 2024-03-26T08:31:01.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.919+0000 7f8639600640 1 -- 172.21.15.43:0/2057578444 wait complete. 2024-03-26T08:31:01.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.920+0000 7f8639600640 1 Processor -- start 2024-03-26T08:31:01.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.920+0000 7f8639600640 1 -- start start 2024-03-26T08:31:01.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.920+0000 7f8639600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86341038e0 0x7f8634199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:01.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.920+0000 7f8639600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8634104ad0 0x7f8634199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:01.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.920+0000 7f8639600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8634105490 0x7f86341a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:01.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.920+0000 7f8639600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863410f460 con 0x7f86341038e0 2024-03-26T08:31:01.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.920+0000 7f8639600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863410f2e0 con 0x7f8634104ad0 2024-03-26T08:31:01.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.920+0000 7f8639600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863410f5e0 con 0x7f8634105490 2024-03-26T08:31:01.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.920+0000 7f8632a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86341038e0 0x7f8634199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:01.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.920+0000 7f8632a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86341038e0 0x7f8634199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55858/0 (socket says 172.21.15.43:55858) 2024-03-26T08:31:01.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.920+0000 7f8632a00640 1 -- 172.21.15.43:0/3238985424 learned_addr learned my addr 172.21.15.43:0/3238985424 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:01.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.921+0000 7f8632a00640 1 -- 172.21.15.43:0/3238985424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8634105490 msgr2=0x7f86341a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:01.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.921+0000 7f862a000640 1 --2- 172.21.15.43:0/3238985424 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8634104ad0 0x7f8634199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:01.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.921+0000 7f8632a00640 1 --2- 172.21.15.43:0/3238985424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8634105490 0x7f86341a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:01.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.921+0000 7f8632a00640 1 -- 172.21.15.43:0/3238985424 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8634104ad0 msgr2=0x7f8634199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:01.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.921+0000 7f8632a00640 1 --2- 172.21.15.43:0/3238985424 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8634104ad0 0x7f8634199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:01.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.921+0000 7f8632a00640 1 -- 172.21.15.43:0/3238985424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86341a1640 con 0x7f86341038e0 2024-03-26T08:31:01.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.921+0000 7f8632a00640 1 --2- 172.21.15.43:0/3238985424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86341038e0 0x7f8634199a20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f862400eaa0 tx=0x7f862400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:01.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.922+0000 7f862be00640 1 -- 172.21.15.43:0/3238985424 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8624016070 con 0x7f86341038e0 2024-03-26T08:31:01.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.922+0000 7f862be00640 1 -- 172.21.15.43:0/3238985424 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8624003e20 con 0x7f86341038e0 2024-03-26T08:31:01.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.922+0000 7f862be00640 1 -- 172.21.15.43:0/3238985424 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86240116d0 con 0x7f86341038e0 2024-03-26T08:31:01.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.922+0000 7f8639600640 1 -- 172.21.15.43:0/3238985424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86341a1930 con 0x7f86341038e0 2024-03-26T08:31:01.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.922+0000 7f8639600640 1 -- 172.21.15.43:0/3238985424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86341a1e80 con 0x7f86341038e0 2024-03-26T08:31:01.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.923+0000 7f8639600640 1 -- 172.21.15.43:0/3238985424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f863410ac00 con 0x7f86341038e0 2024-03-26T08:31:01.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.925+0000 7f862be00640 1 -- 172.21.15.43:0/3238985424 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8624019070 con 0x7f86341038e0 2024-03-26T08:31:01.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.926+0000 7f862be00640 1 --2- 172.21.15.43:0/3238985424 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f86180a2880 0x7f86180a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:01.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.927+0000 7f862be00640 1 -- 172.21.15.43:0/3238985424 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f86240ce740 con 0x7f86341038e0 2024-03-26T08:31:01.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.927+0000 7f862a000640 1 --2- 172.21.15.43:0/3238985424 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f86180a2880 0x7f86180a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:01.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.928+0000 7f862a000640 1 --2- 172.21.15.43:0/3238985424 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f86180a2880 0x7f86180a4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f863419af30 tx=0x7f861c005cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:01.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:01.932+0000 7f862be00640 1 -- 172.21.15.43:0/3238985424 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f862409bbe0 con 0x7f86341038e0 2024-03-26T08:31:02.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:01 smithi043 ceph-mon[31929]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:02.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:01 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:02.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:01 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:02.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:01 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:02.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:01 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:31:02.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:01 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:31:02.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:01 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:31:02.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:02.180+0000 7f8639600640 1 -- 172.21.15.43:0/3238985424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f863402d070 con 0x7f86341038e0 2024-03-26T08:31:02.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:02.181+0000 7f862be00640 1 -- 172.21.15.43:0/3238985424 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7661 (secure 0 0 0) 0x7f863402d070 con 0x7f86341038e0 2024-03-26T08:31:02.174 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:31:02.175 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1452206807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1452206807},{"type":"v1","addr":"172.21.15.43:6835","nonce":1452206807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3151775467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3151775467},{"type":"v1","addr":"172.21.15.43:6837","nonce":3151775467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1842295306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1842295306},{"type":"v1","addr":"172.21.15.183:6835","nonce":1842295306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2267781666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2267781666},{"type":"v1","addr":"172.21.15.183:6837","nonce":2267781666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3572429948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3572429948},{"type":"v1","addr":"172.21.15.183:6839","nonce":3572429948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34448,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/1108327179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108327179},{"type":"v1","addr":"172.21.15.52:6833","nonce":1108327179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34487,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/742057410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":742057410},{"type":"v1","addr":"172.21.15.52:6835","nonce":742057410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34517,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/748239918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":748239918},{"type":"v1","addr":"172.21.15.52:6837","nonce":748239918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-26T08:31:02.175 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 10 2024-03-26T08:31:02.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:02.184+0000 7f8639600640 1 -- 172.21.15.43:0/3238985424 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f86180a2880 msgr2=0x7f86180a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:02.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:02.184+0000 7f8639600640 1 --2- 172.21.15.43:0/3238985424 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f86180a2880 0x7f86180a4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f863419af30 tx=0x7f861c005cc0 comp rx=0 tx=0).stop 2024-03-26T08:31:02.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:02.184+0000 7f8639600640 1 -- 172.21.15.43:0/3238985424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86341038e0 msgr2=0x7f8634199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:02.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:02.184+0000 7f8639600640 1 --2- 172.21.15.43:0/3238985424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86341038e0 0x7f8634199a20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f862400eaa0 tx=0x7f862400ef60 comp rx=0 tx=0).stop 2024-03-26T08:31:02.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:02.184+0000 7f8639600640 1 -- 172.21.15.43:0/3238985424 shutdown_connections 2024-03-26T08:31:02.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:02.184+0000 7f8639600640 1 --2- 172.21.15.43:0/3238985424 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f86180a2880 0x7f86180a4d40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:02.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:02.184+0000 7f8639600640 1 --2- 172.21.15.43:0/3238985424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8634105490 0x7f86341a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:02.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:02.184+0000 7f8639600640 1 --2- 172.21.15.43:0/3238985424 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8634104ad0 0x7f8634199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:02.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:02.184+0000 7f8639600640 1 --2- 172.21.15.43:0/3238985424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86341038e0 0x7f8634199a20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:02.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:02.184+0000 7f8639600640 1 -- 172.21.15.43:0/3238985424 >> 172.21.15.43:0/3238985424 conn(0x7f86340fee50 msgr2=0x7f8634107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:02.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:02.184+0000 7f8639600640 1 -- 172.21.15.43:0/3238985424 shutdown_connections 2024-03-26T08:31:02.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:02.184+0000 7f8639600640 1 -- 172.21.15.43:0/3238985424 wait complete. 2024-03-26T08:31:02.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:01 smithi183 ceph-mon[35818]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:02.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:01 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:02.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:01 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:02.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:01 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:02.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:01 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:31:02.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:01 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:31:02.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:01 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:31:02.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:01 smithi052 ceph-mon[35071]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:02.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:01 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:02.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:01 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:02.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:01 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:02.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:01 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:31:02.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:01 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:31:02.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:01 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:31:03.099 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:02 smithi043 ceph-mon[31929]: Deploying daemon mds.g on smithi043 2024-03-26T08:31:03.100 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:02 smithi043 ceph-mon[31929]: mds.? [v2:172.21.15.183:6838/3572429948,v1:172.21.15.183:6839/3572429948] up:boot 2024-03-26T08:31:03.100 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:02 smithi043 ceph-mon[31929]: fsmap 9 up:standby 2024-03-26T08:31:03.100 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:02 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T08:31:03.100 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:02 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3238985424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:03.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:02 smithi183 ceph-mon[35818]: Deploying daemon mds.g on smithi043 2024-03-26T08:31:03.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:02 smithi183 ceph-mon[35818]: mds.? [v2:172.21.15.183:6838/3572429948,v1:172.21.15.183:6839/3572429948] up:boot 2024-03-26T08:31:03.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:02 smithi183 ceph-mon[35818]: fsmap 9 up:standby 2024-03-26T08:31:03.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:02 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T08:31:03.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:02 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3238985424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:03.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:02 smithi052 ceph-mon[35071]: Deploying daemon mds.g on smithi043 2024-03-26T08:31:03.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:02 smithi052 ceph-mon[35071]: mds.? [v2:172.21.15.183:6838/3572429948,v1:172.21.15.183:6839/3572429948] up:boot 2024-03-26T08:31:03.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:02 smithi052 ceph-mon[35071]: fsmap 9 up:standby 2024-03-26T08:31:03.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:02 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T08:31:03.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:02 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3238985424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:04.197 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:31:04.209 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:03 smithi043 ceph-mon[31929]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:04.251 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:04 smithi052 ceph-mon[35071]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:04.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:04 smithi183 ceph-mon[35818]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:04.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.676+0000 7f4c22a00640 1 -- 172.21.15.43:0/4044494753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c1c104a70 msgr2=0x7f4c1c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:04.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.676+0000 7f4c22a00640 1 --2- 172.21.15.43:0/4044494753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c1c104a70 0x7f4c1c106e60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4c08009900 tx=0x7f4c0802bc30 comp rx=0 tx=0).stop 2024-03-26T08:31:04.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.676+0000 7f4c22a00640 1 -- 172.21.15.43:0/4044494753 shutdown_connections 2024-03-26T08:31:04.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.676+0000 7f4c22a00640 1 --2- 172.21.15.43:0/4044494753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c1c1073a0 0x7f4c1c1097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:04.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.676+0000 7f4c22a00640 1 --2- 172.21.15.43:0/4044494753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c1c104a70 0x7f4c1c106e60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:04.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.676+0000 7f4c22a00640 1 --2- 172.21.15.43:0/4044494753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c1c102140 0x7f4c1c104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:04.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.676+0000 7f4c22a00640 1 -- 172.21.15.43:0/4044494753 >> 172.21.15.43:0/4044494753 conn(0x7f4c1c0fbb50 msgr2=0x7f4c1c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:04.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.676+0000 7f4c22a00640 1 -- 172.21.15.43:0/4044494753 shutdown_connections 2024-03-26T08:31:04.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.677+0000 7f4c22a00640 1 -- 172.21.15.43:0/4044494753 wait complete. 2024-03-26T08:31:04.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.678+0000 7f4c22a00640 1 Processor -- start 2024-03-26T08:31:04.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.678+0000 7f4c22a00640 1 -- start start 2024-03-26T08:31:04.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.678+0000 7f4c22a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c1c102140 0x7f4c1c197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:04.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.678+0000 7f4c22a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c1c104a70 0x7f4c1c197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:04.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.679+0000 7f4c22a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c1c1073a0 0x7f4c1c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.679+0000 7f4c22a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c10d250 con 0x7f4c1c104a70 2024-03-26T08:31:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.679+0000 7f4c22a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c10d0d0 con 0x7f4c1c1073a0 2024-03-26T08:31:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.679+0000 7f4c22a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c10d3d0 con 0x7f4c1c102140 2024-03-26T08:31:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.679+0000 7f4c1b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c1c104a70 0x7f4c1c197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.679+0000 7f4c1b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c1c104a70 0x7f4c1c197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49990/0 (socket says 172.21.15.43:49990) 2024-03-26T08:31:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.679+0000 7f4c1b400640 1 -- 172.21.15.43:0/3484520920 learned_addr learned my addr 172.21.15.43:0/3484520920 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.679+0000 7f4c20a00640 1 --2- 172.21.15.43:0/3484520920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c1c1073a0 0x7f4c1c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.679+0000 7f4c1be00640 1 --2- 172.21.15.43:0/3484520920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c1c102140 0x7f4c1c197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.679+0000 7f4c1b400640 1 -- 172.21.15.43:0/3484520920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c1c102140 msgr2=0x7f4c1c197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.679+0000 7f4c1b400640 1 --2- 172.21.15.43:0/3484520920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c1c102140 0x7f4c1c197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.679+0000 7f4c1b400640 1 -- 172.21.15.43:0/3484520920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c1c1073a0 msgr2=0x7f4c1c19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.679+0000 7f4c1b400640 1 --2- 172.21.15.43:0/3484520920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c1c1073a0 0x7f4c1c19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.679+0000 7f4c1b400640 1 -- 172.21.15.43:0/3484520920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1c19f570 con 0x7f4c1c104a70 2024-03-26T08:31:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.679+0000 7f4c20a00640 1 --2- 172.21.15.43:0/3484520920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c1c1073a0 0x7f4c1c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:31:04.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.680+0000 7f4c1b400640 1 --2- 172.21.15.43:0/3484520920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c1c104a70 0x7f4c1c197e90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4c08005ab0 tx=0x7f4c080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:04.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.680+0000 7f4c18c00640 1 -- 172.21.15.43:0/3484520920 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c08043020 con 0x7f4c1c104a70 2024-03-26T08:31:04.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.680+0000 7f4c22a00640 1 -- 172.21.15.43:0/3484520920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c1c19f800 con 0x7f4c1c104a70 2024-03-26T08:31:04.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.680+0000 7f4c18c00640 1 -- 172.21.15.43:0/3484520920 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4c080054c0 con 0x7f4c1c104a70 2024-03-26T08:31:04.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.680+0000 7f4c18c00640 1 -- 172.21.15.43:0/3484520920 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c08003d10 con 0x7f4c1c104a70 2024-03-26T08:31:04.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.680+0000 7f4c22a00640 1 -- 172.21.15.43:0/3484520920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1c19fd50 con 0x7f4c1c104a70 2024-03-26T08:31:04.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.681+0000 7f4c22a00640 1 -- 172.21.15.43:0/3484520920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c1c0fd640 con 0x7f4c1c104a70 2024-03-26T08:31:04.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.684+0000 7f4c18c00640 1 -- 172.21.15.43:0/3484520920 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f4c08046020 con 0x7f4c1c104a70 2024-03-26T08:31:04.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.685+0000 7f4c18c00640 1 --2- 172.21.15.43:0/3484520920 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f4c000a2800 0x7f4c000a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:04.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.685+0000 7f4c18c00640 1 -- 172.21.15.43:0/3484520920 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4c080f0e70 con 0x7f4c1c104a70 2024-03-26T08:31:04.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.686+0000 7f4c1be00640 1 --2- 172.21.15.43:0/3484520920 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f4c000a2800 0x7f4c000a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:04.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.686+0000 7f4c1be00640 1 --2- 172.21.15.43:0/3484520920 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f4c000a2800 0x7f4c000a4cc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c005fd0 tx=0x7f4c0c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:04.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.692+0000 7f4c18c00640 1 -- 172.21.15.43:0/3484520920 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4c0804d020 con 0x7f4c1c104a70 2024-03-26T08:31:04.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.943+0000 7f4c22a00640 1 -- 172.21.15.43:0/3484520920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c1c02d2c0 con 0x7f4c1c104a70 2024-03-26T08:31:04.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.945+0000 7f4c18c00640 1 -- 172.21.15.43:0/3484520920 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7661 (secure 0 0 0) 0x7f4c080be310 con 0x7f4c1c104a70 2024-03-26T08:31:04.938 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:31:04.938 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1452206807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1452206807},{"type":"v1","addr":"172.21.15.43:6835","nonce":1452206807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3151775467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3151775467},{"type":"v1","addr":"172.21.15.43:6837","nonce":3151775467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1842295306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1842295306},{"type":"v1","addr":"172.21.15.183:6835","nonce":1842295306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2267781666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2267781666},{"type":"v1","addr":"172.21.15.183:6837","nonce":2267781666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3572429948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3572429948},{"type":"v1","addr":"172.21.15.183:6839","nonce":3572429948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34448,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/1108327179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108327179},{"type":"v1","addr":"172.21.15.52:6833","nonce":1108327179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34487,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/742057410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":742057410},{"type":"v1","addr":"172.21.15.52:6835","nonce":742057410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34517,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/748239918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":748239918},{"type":"v1","addr":"172.21.15.52:6837","nonce":748239918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-26T08:31:04.938 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 10 2024-03-26T08:31:04.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.947+0000 7f4c22a00640 1 -- 172.21.15.43:0/3484520920 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f4c000a2800 msgr2=0x7f4c000a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:04.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.947+0000 7f4c22a00640 1 --2- 172.21.15.43:0/3484520920 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f4c000a2800 0x7f4c000a4cc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c005fd0 tx=0x7f4c0c005850 comp rx=0 tx=0).stop 2024-03-26T08:31:04.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.947+0000 7f4c22a00640 1 -- 172.21.15.43:0/3484520920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c1c104a70 msgr2=0x7f4c1c197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:04.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.947+0000 7f4c22a00640 1 --2- 172.21.15.43:0/3484520920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c1c104a70 0x7f4c1c197e90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4c08005ab0 tx=0x7f4c080042f0 comp rx=0 tx=0).stop 2024-03-26T08:31:04.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.947+0000 7f4c22a00640 1 -- 172.21.15.43:0/3484520920 shutdown_connections 2024-03-26T08:31:04.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.947+0000 7f4c22a00640 1 --2- 172.21.15.43:0/3484520920 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f4c000a2800 0x7f4c000a4cc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:04.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.947+0000 7f4c22a00640 1 --2- 172.21.15.43:0/3484520920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c1c1073a0 0x7f4c1c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:04.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.947+0000 7f4c22a00640 1 --2- 172.21.15.43:0/3484520920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c1c104a70 0x7f4c1c197e90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:04.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.947+0000 7f4c22a00640 1 --2- 172.21.15.43:0/3484520920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c1c102140 0x7f4c1c197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:04.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.947+0000 7f4c22a00640 1 -- 172.21.15.43:0/3484520920 >> 172.21.15.43:0/3484520920 conn(0x7f4c1c0fbb50 msgr2=0x7f4c1c102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:04.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.947+0000 7f4c22a00640 1 -- 172.21.15.43:0/3484520920 shutdown_connections 2024-03-26T08:31:04.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:04.947+0000 7f4c22a00640 1 -- 172.21.15.43:0/3484520920 wait complete. 2024-03-26T08:31:05.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:05 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3484520920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:05.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:05 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3484520920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:05.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:05 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3484520920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:05.455 INFO:journalctl@ceph.mds.g.smithi043.stdout:Mar 26 08:31:05 smithi043 podman[70652]: 2024-03-26 08:31:05.177274895 +0000 UTC m=+0.038481544 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:31:05.456 INFO:journalctl@ceph.mds.g.smithi043.stdout:Mar 26 08:31:05 smithi043 podman[70652]: 2024-03-26 08:31:05.462321336 +0000 UTC m=+0.323528019 container create 27c24afd4d0533bedfb7e0211faa40f980b92603459768cad4de403d51b5a6a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-g, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T08:31:06.015 INFO:journalctl@ceph.mds.g.smithi043.stdout:Mar 26 08:31:05 smithi043 podman[70652]: 2024-03-26 08:31:05.741840394 +0000 UTC m=+0.603047093 container init 27c24afd4d0533bedfb7e0211faa40f980b92603459768cad4de403d51b5a6a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-g, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T08:31:06.015 INFO:journalctl@ceph.mds.g.smithi043.stdout:Mar 26 08:31:05 smithi043 podman[70652]: 2024-03-26 08:31:05.750948229 +0000 UTC m=+0.612154892 container start 27c24afd4d0533bedfb7e0211faa40f980b92603459768cad4de403d51b5a6a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-g, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T08:31:06.015 INFO:journalctl@ceph.mds.g.smithi043.stdout:Mar 26 08:31:05 smithi043 ceph-mds[70675]: -- 172.21.15.43:0/3884800490 <== mon.0 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56097086ba40 con 0x56097083ec00 2024-03-26T08:31:06.015 INFO:journalctl@ceph.mds.g.smithi043.stdout:Mar 26 08:31:05 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-g[70668]: starting mds.g at 2024-03-26T08:31:06.015 INFO:journalctl@ceph.mds.g.smithi043.stdout:Mar 26 08:31:05 smithi043 bash[70652]: 27c24afd4d0533bedfb7e0211faa40f980b92603459768cad4de403d51b5a6a5 2024-03-26T08:31:06.015 INFO:journalctl@ceph.mds.g.smithi043.stdout:Mar 26 08:31:05 smithi043 systemd[1]: Started Ceph mds.g for 88759420-eb49-11ee-95d0-87774f69a715. 2024-03-26T08:31:06.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:06 smithi183 ceph-mon[35818]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:06.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:06 smithi052 ceph-mon[35071]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:06.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:06 smithi043 ceph-mon[31929]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:06.961 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:31:07.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.190+0000 7fde47800640 1 -- 172.21.15.43:0/688752981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde401012a0 msgr2=0x7fde40101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:07.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.190+0000 7fde47800640 1 --2- 172.21.15.43:0/688752981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde401012a0 0x7fde40101720 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fde38009ca0 tx=0x7fde3802c190 comp rx=0 tx=0).stop 2024-03-26T08:31:07.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.190+0000 7fde47800640 1 -- 172.21.15.43:0/688752981 shutdown_connections 2024-03-26T08:31:07.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.190+0000 7fde47800640 1 --2- 172.21.15.43:0/688752981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde401012a0 0x7fde40101720 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:07.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.190+0000 7fde47800640 1 --2- 172.21.15.43:0/688752981 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde40100900 0x7fde40100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:07.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.190+0000 7fde47800640 1 --2- 172.21.15.43:0/688752981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde400fffa0 0x7fde401003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:07.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.190+0000 7fde47800640 1 -- 172.21.15.43:0/688752981 >> 172.21.15.43:0/688752981 conn(0x7fde400fbb20 msgr2=0x7fde400fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:07.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.191+0000 7fde47800640 1 -- 172.21.15.43:0/688752981 shutdown_connections 2024-03-26T08:31:07.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.191+0000 7fde47800640 1 -- 172.21.15.43:0/688752981 wait complete. 2024-03-26T08:31:07.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.192+0000 7fde47800640 1 Processor -- start 2024-03-26T08:31:07.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.192+0000 7fde47800640 1 -- start start 2024-03-26T08:31:07.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.192+0000 7fde47800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde400fffa0 0x7fde40199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:07.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.192+0000 7fde47800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde40100900 0x7fde40199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:07.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.192+0000 7fde47800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde401012a0 0x7fde401a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:07.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.192+0000 7fde47800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4010f3e0 con 0x7fde401012a0 2024-03-26T08:31:07.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.192+0000 7fde47800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4010f260 con 0x7fde40100900 2024-03-26T08:31:07.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.192+0000 7fde47800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4010f560 con 0x7fde400fffa0 2024-03-26T08:31:07.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.192+0000 7fde45800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde401012a0 0x7fde401a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:07.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.192+0000 7fde45800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde401012a0 0x7fde401a1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50038/0 (socket says 172.21.15.43:50038) 2024-03-26T08:31:07.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.192+0000 7fde45800640 1 -- 172.21.15.43:0/3313180916 learned_addr learned my addr 172.21.15.43:0/3313180916 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:07.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.193+0000 7fde44e00640 1 --2- 172.21.15.43:0/3313180916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde400fffa0 0x7fde40199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:07.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.193+0000 7fde3fe00640 1 --2- 172.21.15.43:0/3313180916 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde40100900 0x7fde40199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:07.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.193+0000 7fde45800640 1 -- 172.21.15.43:0/3313180916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde400fffa0 msgr2=0x7fde40199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:07.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.193+0000 7fde45800640 1 --2- 172.21.15.43:0/3313180916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde400fffa0 0x7fde40199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:07.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.193+0000 7fde45800640 1 -- 172.21.15.43:0/3313180916 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde40100900 msgr2=0x7fde40199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:07.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.193+0000 7fde45800640 1 --2- 172.21.15.43:0/3313180916 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde40100900 0x7fde40199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:07.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.193+0000 7fde45800640 1 -- 172.21.15.43:0/3313180916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde401a16a0 con 0x7fde401012a0 2024-03-26T08:31:07.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.193+0000 7fde45800640 1 --2- 172.21.15.43:0/3313180916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde401012a0 0x7fde401a1030 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fde3802c6d0 tx=0x7fde38003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:07.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.194+0000 7fde3d600640 1 -- 172.21.15.43:0/3313180916 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde38039040 con 0x7fde401012a0 2024-03-26T08:31:07.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.194+0000 7fde3d600640 1 -- 172.21.15.43:0/3313180916 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fde38044070 con 0x7fde401012a0 2024-03-26T08:31:07.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.194+0000 7fde3d600640 1 -- 172.21.15.43:0/3313180916 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde380043c0 con 0x7fde401012a0 2024-03-26T08:31:07.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.194+0000 7fde47800640 1 -- 172.21.15.43:0/3313180916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde401a1930 con 0x7fde401012a0 2024-03-26T08:31:07.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.194+0000 7fde47800640 1 -- 172.21.15.43:0/3313180916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde401a1dd0 con 0x7fde401012a0 2024-03-26T08:31:07.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.195+0000 7fde47800640 1 -- 172.21.15.43:0/3313180916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4010aeb0 con 0x7fde401012a0 2024-03-26T08:31:07.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.197+0000 7fde3d600640 1 -- 172.21.15.43:0/3313180916 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fde38004560 con 0x7fde401012a0 2024-03-26T08:31:07.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.199+0000 7fde3d600640 1 --2- 172.21.15.43:0/3313180916 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fde280a2880 0x7fde280a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:07.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.199+0000 7fde3d600640 1 -- 172.21.15.43:0/3313180916 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fde380f0d40 con 0x7fde401012a0 2024-03-26T08:31:07.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.199+0000 7fde44e00640 1 --2- 172.21.15.43:0/3313180916 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fde280a2880 0x7fde280a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:07.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.200+0000 7fde44e00640 1 --2- 172.21.15.43:0/3313180916 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fde280a2880 0x7fde280a4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fde34005fd0 tx=0x7fde34005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:07.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.204+0000 7fde3d600640 1 -- 172.21.15.43:0/3313180916 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fde380be1e0 con 0x7fde401012a0 2024-03-26T08:31:07.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:07 smithi183 ceph-mon[35818]: mds.? [v2:172.21.15.43:6838/1284655682,v1:172.21.15.43:6839/1284655682] up:boot 2024-03-26T08:31:07.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:07 smithi183 ceph-mon[35818]: fsmap 10 up:standby 2024-03-26T08:31:07.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:07 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T08:31:07.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:07 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:07.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:07 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:07.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:07 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:07.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:07 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:31:07.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:07 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:31:07.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:07 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:31:07.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:07 smithi183 ceph-mon[35818]: Deploying daemon mds.a on smithi043 2024-03-26T08:31:07.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:07 smithi052 ceph-mon[35071]: mds.? [v2:172.21.15.43:6838/1284655682,v1:172.21.15.43:6839/1284655682] up:boot 2024-03-26T08:31:07.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:07 smithi052 ceph-mon[35071]: fsmap 10 up:standby 2024-03-26T08:31:07.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:07 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T08:31:07.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:07 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:07.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:07 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:07.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:07 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:07.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:07 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:31:07.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:07 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:31:07.323 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:07 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:31:07.324 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:07 smithi052 ceph-mon[35071]: Deploying daemon mds.a on smithi043 2024-03-26T08:31:07.365 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:07 smithi043 ceph-mon[31929]: mds.? [v2:172.21.15.43:6838/1284655682,v1:172.21.15.43:6839/1284655682] up:boot 2024-03-26T08:31:07.365 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:07 smithi043 ceph-mon[31929]: fsmap 10 up:standby 2024-03-26T08:31:07.365 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:07 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T08:31:07.365 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:07 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:07.366 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:07 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:07.366 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:07 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:07.366 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:07 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:31:07.366 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:07 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:31:07.366 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:07 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:31:07.366 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:07 smithi043 ceph-mon[31929]: Deploying daemon mds.a on smithi043 2024-03-26T08:31:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.457+0000 7fde47800640 1 -- 172.21.15.43:0/3313180916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde4002d070 con 0x7fde401012a0 2024-03-26T08:31:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.459+0000 7fde3d600640 1 -- 172.21.15.43:0/3313180916 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8453 (secure 0 0 0) 0x7fde380bdae0 con 0x7fde401012a0 2024-03-26T08:31:07.452 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:31:07.452 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1452206807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1452206807},{"type":"v1","addr":"172.21.15.43:6835","nonce":1452206807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3151775467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3151775467},{"type":"v1","addr":"172.21.15.43:6837","nonce":3151775467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/1284655682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":1284655682},{"type":"v1","addr":"172.21.15.43:6839","nonce":1284655682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1842295306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1842295306},{"type":"v1","addr":"172.21.15.183:6835","nonce":1842295306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2267781666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2267781666},{"type":"v1","addr":"172.21.15.183:6837","nonce":2267781666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3572429948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3572429948},{"type":"v1","addr":"172.21.15.183:6839","nonce":3572429948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34448,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/1108327179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108327179},{"type":"v1","addr":"172.21.15.52:6833","nonce":1108327179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34487,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/742057410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":742057410},{"type":"v1","addr":"172.21.15.52:6835","nonce":742057410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34517,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/748239918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":748239918},{"type":"v1","addr":"172.21.15.52:6837","nonce":748239918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-26T08:31:07.453 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 11 2024-03-26T08:31:07.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.461+0000 7fde47800640 1 -- 172.21.15.43:0/3313180916 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fde280a2880 msgr2=0x7fde280a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:07.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.461+0000 7fde47800640 1 --2- 172.21.15.43:0/3313180916 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fde280a2880 0x7fde280a4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fde34005fd0 tx=0x7fde34005850 comp rx=0 tx=0).stop 2024-03-26T08:31:07.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.461+0000 7fde47800640 1 -- 172.21.15.43:0/3313180916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde401012a0 msgr2=0x7fde401a1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:07.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.461+0000 7fde47800640 1 --2- 172.21.15.43:0/3313180916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde401012a0 0x7fde401a1030 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fde3802c6d0 tx=0x7fde38003ba0 comp rx=0 tx=0).stop 2024-03-26T08:31:07.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.462+0000 7fde47800640 1 -- 172.21.15.43:0/3313180916 shutdown_connections 2024-03-26T08:31:07.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.462+0000 7fde47800640 1 --2- 172.21.15.43:0/3313180916 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fde280a2880 0x7fde280a4d40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:07.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.462+0000 7fde47800640 1 --2- 172.21.15.43:0/3313180916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde401012a0 0x7fde401a1030 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:07.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.462+0000 7fde47800640 1 --2- 172.21.15.43:0/3313180916 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde40100900 0x7fde40199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:07.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.462+0000 7fde47800640 1 --2- 172.21.15.43:0/3313180916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde400fffa0 0x7fde40199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:07.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.462+0000 7fde47800640 1 -- 172.21.15.43:0/3313180916 >> 172.21.15.43:0/3313180916 conn(0x7fde400fbb20 msgr2=0x7fde400fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:07.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.462+0000 7fde47800640 1 -- 172.21.15.43:0/3313180916 shutdown_connections 2024-03-26T08:31:07.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:07.462+0000 7fde47800640 1 -- 172.21.15.43:0/3313180916 wait complete. 2024-03-26T08:31:08.341 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:08 smithi043 ceph-mon[31929]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:08.341 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:08 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3313180916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:08.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:08 smithi183 ceph-mon[35818]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:08.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:08 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3313180916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:08.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:08 smithi052 ceph-mon[35071]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:08.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:08 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3313180916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:09.402 INFO:journalctl@ceph.mds.a.smithi043.stdout:Mar 26 08:31:09 smithi043 systemd[1]: Starting Ceph mds.a for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:31:09.474 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:31:09.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.705+0000 7fa022000640 1 -- 172.21.15.43:0/2584318932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa01c1016c0 msgr2=0x7fa01c101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:09.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.705+0000 7fa022000640 1 --2- 172.21.15.43:0/2584318932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa01c1016c0 0x7fa01c101ae0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa00c009900 tx=0x7fa00c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:31:09.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.705+0000 7fa022000640 1 -- 172.21.15.43:0/2584318932 shutdown_connections 2024-03-26T08:31:09.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.705+0000 7fa022000640 1 --2- 172.21.15.43:0/2584318932 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa01c103270 0x7fa01c109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:09.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.705+0000 7fa022000640 1 --2- 172.21.15.43:0/2584318932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa01c1028b0 0x7fa01c102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:09.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.705+0000 7fa022000640 1 --2- 172.21.15.43:0/2584318932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa01c1016c0 0x7fa01c101ae0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:09.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.705+0000 7fa022000640 1 -- 172.21.15.43:0/2584318932 >> 172.21.15.43:0/2584318932 conn(0x7fa01c0fcc50 msgr2=0x7fa01c0ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:09.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.705+0000 7fa022000640 1 -- 172.21.15.43:0/2584318932 shutdown_connections 2024-03-26T08:31:09.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.705+0000 7fa022000640 1 -- 172.21.15.43:0/2584318932 wait complete. 2024-03-26T08:31:09.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.706+0000 7fa022000640 1 Processor -- start 2024-03-26T08:31:09.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.706+0000 7fa022000640 1 -- start start 2024-03-26T08:31:09.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.707+0000 7fa022000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa01c1016c0 0x7fa01c197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:09.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.707+0000 7fa022000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa01c1028b0 0x7fa01c197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:09.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.707+0000 7fa022000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa01c103270 0x7fa01c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:09.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.707+0000 7fa022000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c10d1f0 con 0x7fa01c1028b0 2024-03-26T08:31:09.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.707+0000 7fa022000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c10d070 con 0x7fa01c103270 2024-03-26T08:31:09.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.707+0000 7fa022000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c10d370 con 0x7fa01c1016c0 2024-03-26T08:31:09.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.707+0000 7fa01aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa01c1028b0 0x7fa01c197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:09.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.707+0000 7fa01aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa01c1028b0 0x7fa01c197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50052/0 (socket says 172.21.15.43:50052) 2024-03-26T08:31:09.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.707+0000 7fa01aa00640 1 -- 172.21.15.43:0/4154316435 learned_addr learned my addr 172.21.15.43:0/4154316435 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:09.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.707+0000 7fa01b400640 1 --2- 172.21.15.43:0/4154316435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa01c1016c0 0x7fa01c197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:09.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.707+0000 7fa01be00640 1 --2- 172.21.15.43:0/4154316435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa01c103270 0x7fa01c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:09.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.707+0000 7fa01aa00640 1 -- 172.21.15.43:0/4154316435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa01c1016c0 msgr2=0x7fa01c197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:09.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.707+0000 7fa01aa00640 1 --2- 172.21.15.43:0/4154316435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa01c1016c0 0x7fa01c197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:09.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.707+0000 7fa01aa00640 1 -- 172.21.15.43:0/4154316435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa01c103270 msgr2=0x7fa01c19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:09.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.707+0000 7fa01aa00640 1 --2- 172.21.15.43:0/4154316435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa01c103270 0x7fa01c19eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:09.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.708+0000 7fa01aa00640 1 -- 172.21.15.43:0/4154316435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01c19f540 con 0x7fa01c1028b0 2024-03-26T08:31:09.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.708+0000 7fa01aa00640 1 --2- 172.21.15.43:0/4154316435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa01c1028b0 0x7fa01c197e60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa00800d770 tx=0x7fa00800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:09.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.708+0000 7fa013e00640 1 -- 172.21.15.43:0/4154316435 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa008017070 con 0x7fa01c1028b0 2024-03-26T08:31:09.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.708+0000 7fa013e00640 1 -- 172.21.15.43:0/4154316435 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa0080054d0 con 0x7fa01c1028b0 2024-03-26T08:31:09.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.708+0000 7fa022000640 1 -- 172.21.15.43:0/4154316435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa01c19f830 con 0x7fa01c1028b0 2024-03-26T08:31:09.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.708+0000 7fa013e00640 1 -- 172.21.15.43:0/4154316435 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa008004cf0 con 0x7fa01c1028b0 2024-03-26T08:31:09.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.708+0000 7fa022000640 1 -- 172.21.15.43:0/4154316435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa01c19fd30 con 0x7fa01c1028b0 2024-03-26T08:31:09.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.710+0000 7fa022000640 1 -- 172.21.15.43:0/4154316435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa01c108ab0 con 0x7fa01c1028b0 2024-03-26T08:31:09.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.712+0000 7fa013e00640 1 -- 172.21.15.43:0/4154316435 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa008004e90 con 0x7fa01c1028b0 2024-03-26T08:31:09.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.714+0000 7fa013e00640 1 --2- 172.21.15.43:0/4154316435 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa0000a2800 0x7fa0000a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:09.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.714+0000 7fa013e00640 1 -- 172.21.15.43:0/4154316435 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa0080ce170 con 0x7fa01c1028b0 2024-03-26T08:31:09.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.714+0000 7fa01b400640 1 --2- 172.21.15.43:0/4154316435 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa0000a2800 0x7fa0000a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:09.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.715+0000 7fa01b400640 1 --2- 172.21.15.43:0/4154316435 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa0000a2800 0x7fa0000a4cc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa00c006010 tx=0x7fa00c005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:09.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.719+0000 7fa013e00640 1 -- 172.21.15.43:0/4154316435 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa00809b610 con 0x7fa01c1028b0 2024-03-26T08:31:09.775 INFO:journalctl@ceph.mds.a.smithi043.stdout:Mar 26 08:31:09 smithi043 podman[71006]: 2024-03-26 08:31:09.417466382 +0000 UTC m=+0.032444213 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:31:09.775 INFO:journalctl@ceph.mds.a.smithi043.stdout:Mar 26 08:31:09 smithi043 podman[71006]: 2024-03-26 08:31:09.535825029 +0000 UTC m=+0.150802884 container create e7393de82697c3f3f2012ebbc3a63525f5f9f893d97d2ebbcb78d7e7c23b5cb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-26T08:31:09.775 INFO:journalctl@ceph.mds.a.smithi043.stdout:Mar 26 08:31:09 smithi043 podman[71006]: 2024-03-26 08:31:09.781955071 +0000 UTC m=+0.396932933 container init e7393de82697c3f3f2012ebbc3a63525f5f9f893d97d2ebbcb78d7e7c23b5cb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240319) 2024-03-26T08:31:09.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.981+0000 7fa022000640 1 -- 172.21.15.43:0/4154316435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa01c02d070 con 0x7fa01c1028b0 2024-03-26T08:31:09.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.986+0000 7fa013e00640 1 -- 172.21.15.43:0/4154316435 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8453 (secure 0 0 0) 0x7fa00809af10 con 0x7fa01c1028b0 2024-03-26T08:31:09.979 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:31:09.980 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1452206807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1452206807},{"type":"v1","addr":"172.21.15.43:6835","nonce":1452206807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3151775467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3151775467},{"type":"v1","addr":"172.21.15.43:6837","nonce":3151775467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/1284655682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":1284655682},{"type":"v1","addr":"172.21.15.43:6839","nonce":1284655682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1842295306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1842295306},{"type":"v1","addr":"172.21.15.183:6835","nonce":1842295306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2267781666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2267781666},{"type":"v1","addr":"172.21.15.183:6837","nonce":2267781666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3572429948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3572429948},{"type":"v1","addr":"172.21.15.183:6839","nonce":3572429948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34448,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/1108327179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108327179},{"type":"v1","addr":"172.21.15.52:6833","nonce":1108327179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34487,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/742057410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":742057410},{"type":"v1","addr":"172.21.15.52:6835","nonce":742057410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34517,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/748239918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":748239918},{"type":"v1","addr":"172.21.15.52:6837","nonce":748239918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-26T08:31:09.980 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 11 2024-03-26T08:31:09.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.989+0000 7fa022000640 1 -- 172.21.15.43:0/4154316435 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa0000a2800 msgr2=0x7fa0000a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:09.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.989+0000 7fa022000640 1 --2- 172.21.15.43:0/4154316435 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa0000a2800 0x7fa0000a4cc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa00c006010 tx=0x7fa00c005be0 comp rx=0 tx=0).stop 2024-03-26T08:31:09.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.989+0000 7fa022000640 1 -- 172.21.15.43:0/4154316435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa01c1028b0 msgr2=0x7fa01c197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:09.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.989+0000 7fa022000640 1 --2- 172.21.15.43:0/4154316435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa01c1028b0 0x7fa01c197e60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa00800d770 tx=0x7fa00800dc30 comp rx=0 tx=0).stop 2024-03-26T08:31:09.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.989+0000 7fa022000640 1 -- 172.21.15.43:0/4154316435 shutdown_connections 2024-03-26T08:31:09.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.989+0000 7fa022000640 1 --2- 172.21.15.43:0/4154316435 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa0000a2800 0x7fa0000a4cc0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:09.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.989+0000 7fa022000640 1 --2- 172.21.15.43:0/4154316435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa01c103270 0x7fa01c19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:09.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.989+0000 7fa022000640 1 --2- 172.21.15.43:0/4154316435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa01c1028b0 0x7fa01c197e60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:09.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.989+0000 7fa022000640 1 --2- 172.21.15.43:0/4154316435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa01c1016c0 0x7fa01c197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:09.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.989+0000 7fa022000640 1 -- 172.21.15.43:0/4154316435 >> 172.21.15.43:0/4154316435 conn(0x7fa01c0fcc50 msgr2=0x7fa01c105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:09.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.989+0000 7fa022000640 1 -- 172.21.15.43:0/4154316435 shutdown_connections 2024-03-26T08:31:09.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:09.989+0000 7fa022000640 1 -- 172.21.15.43:0/4154316435 wait complete. 2024-03-26T08:31:10.105 INFO:journalctl@ceph.mds.a.smithi043.stdout:Mar 26 08:31:09 smithi043 podman[71006]: 2024-03-26 08:31:09.790603614 +0000 UTC m=+0.405581475 container start e7393de82697c3f3f2012ebbc3a63525f5f9f893d97d2ebbcb78d7e7c23b5cb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-26T08:31:10.105 INFO:journalctl@ceph.mds.a.smithi043.stdout:Mar 26 08:31:09 smithi043 ceph-mds[71064]: -- 172.21.15.43:0/3978709728 <== mon.0 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c04d1b1a40 con 0x55c04d184c00 2024-03-26T08:31:10.105 INFO:journalctl@ceph.mds.a.smithi043.stdout:Mar 26 08:31:09 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-a[71041]: starting mds.a at 2024-03-26T08:31:10.105 INFO:journalctl@ceph.mds.a.smithi043.stdout:Mar 26 08:31:09 smithi043 bash[71006]: e7393de82697c3f3f2012ebbc3a63525f5f9f893d97d2ebbcb78d7e7c23b5cb1 2024-03-26T08:31:10.105 INFO:journalctl@ceph.mds.a.smithi043.stdout:Mar 26 08:31:09 smithi043 systemd[1]: Started Ceph mds.a for 88759420-eb49-11ee-95d0-87774f69a715. 2024-03-26T08:31:10.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:10 smithi043 ceph-mon[31929]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:10.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:10 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:10.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:10 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/4154316435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:10.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:10 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:10.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:10 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:10.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:10 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:31:10.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:10 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:31:10.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:10 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:31:10.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:10 smithi183 ceph-mon[35818]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:10.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:10 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:10.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:10 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/4154316435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:10.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:10 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:10.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:10 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:10.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:10 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:31:10.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:10 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:31:10.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:10 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:31:10.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:10 smithi052 ceph-mon[35071]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:10.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:10 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:10.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:10 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/4154316435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:10.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:10 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:10.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:10 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:10.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:10 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:31:10.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:10 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:31:10.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:10 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:31:11.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:11 smithi043 ceph-mon[31929]: Deploying daemon mds.b on smithi052 2024-03-26T08:31:11.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:11 smithi043 ceph-mon[31929]: mds.? [v2:172.21.15.43:6840/3081085787,v1:172.21.15.43:6841/3081085787] up:boot 2024-03-26T08:31:11.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:11 smithi043 ceph-mon[31929]: fsmap 11 up:standby 2024-03-26T08:31:11.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:11 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T08:31:11.406 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:11 smithi052 ceph-mon[35071]: Deploying daemon mds.b on smithi052 2024-03-26T08:31:11.406 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:11 smithi052 ceph-mon[35071]: mds.? [v2:172.21.15.43:6840/3081085787,v1:172.21.15.43:6841/3081085787] up:boot 2024-03-26T08:31:11.406 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:11 smithi052 ceph-mon[35071]: fsmap 11 up:standby 2024-03-26T08:31:11.406 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:11 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T08:31:11.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:11 smithi183 ceph-mon[35818]: Deploying daemon mds.b on smithi052 2024-03-26T08:31:11.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:11 smithi183 ceph-mon[35818]: mds.? [v2:172.21.15.43:6840/3081085787,v1:172.21.15.43:6841/3081085787] up:boot 2024-03-26T08:31:11.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:11 smithi183 ceph-mon[35818]: fsmap 11 up:standby 2024-03-26T08:31:11.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:11 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T08:31:12.004 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:31:12.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.244+0000 7fb626800640 1 -- 172.21.15.43:0/2726803811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6201052b0 msgr2=0x7fb62010bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:12.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.244+0000 7fb626800640 1 --2- 172.21.15.43:0/2726803811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6201052b0 0x7fb62010bb40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb618009900 tx=0x7fb61802bc30 comp rx=0 tx=0).stop 2024-03-26T08:31:12.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.244+0000 7fb626800640 1 -- 172.21.15.43:0/2726803811 shutdown_connections 2024-03-26T08:31:12.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.244+0000 7fb626800640 1 --2- 172.21.15.43:0/2726803811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6201052b0 0x7fb62010bb40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:12.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.244+0000 7fb626800640 1 --2- 172.21.15.43:0/2726803811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6201048f0 0x7fb620104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:12.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.244+0000 7fb626800640 1 --2- 172.21.15.43:0/2726803811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb620103700 0x7fb620103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:12.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.244+0000 7fb626800640 1 -- 172.21.15.43:0/2726803811 >> 172.21.15.43:0/2726803811 conn(0x7fb6200fec70 msgr2=0x7fb6201010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:12.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.244+0000 7fb626800640 1 -- 172.21.15.43:0/2726803811 shutdown_connections 2024-03-26T08:31:12.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.244+0000 7fb626800640 1 -- 172.21.15.43:0/2726803811 wait complete. 2024-03-26T08:31:12.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.245+0000 7fb626800640 1 Processor -- start 2024-03-26T08:31:12.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.245+0000 7fb626800640 1 -- start start 2024-03-26T08:31:12.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.246+0000 7fb626800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb620103700 0x7fb6201998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:12.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.246+0000 7fb626800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6201048f0 0x7fb620199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:12.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.246+0000 7fb626800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6201052b0 0x7fb6201a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:12.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.246+0000 7fb626800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62010f280 con 0x7fb6201048f0 2024-03-26T08:31:12.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.246+0000 7fb626800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62010f100 con 0x7fb6201052b0 2024-03-26T08:31:12.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.246+0000 7fb626800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62010f400 con 0x7fb620103700 2024-03-26T08:31:12.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.246+0000 7fb624a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6201048f0 0x7fb620199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:12.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.246+0000 7fb624a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6201048f0 0x7fb620199e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50092/0 (socket says 172.21.15.43:50092) 2024-03-26T08:31:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.246+0000 7fb624a00640 1 -- 172.21.15.43:0/3949694137 learned_addr learned my addr 172.21.15.43:0/3949694137 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.246+0000 7fb625400640 1 --2- 172.21.15.43:0/3949694137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb620103700 0x7fb6201998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:12.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.247+0000 7fb625e00640 1 --2- 172.21.15.43:0/3949694137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6201052b0 0x7fb6201a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:12.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.247+0000 7fb624a00640 1 -- 172.21.15.43:0/3949694137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb620103700 msgr2=0x7fb6201998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:12.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.247+0000 7fb624a00640 1 --2- 172.21.15.43:0/3949694137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb620103700 0x7fb6201998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:12.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.247+0000 7fb624a00640 1 -- 172.21.15.43:0/3949694137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6201052b0 msgr2=0x7fb6201a0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:12.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.247+0000 7fb624a00640 1 --2- 172.21.15.43:0/3949694137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6201052b0 0x7fb6201a0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:12.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.247+0000 7fb624a00640 1 -- 172.21.15.43:0/3949694137 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6201a14e0 con 0x7fb6201048f0 2024-03-26T08:31:12.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.247+0000 7fb625e00640 1 --2- 172.21.15.43:0/3949694137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6201052b0 0x7fb6201a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:31:12.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.247+0000 7fb624a00640 1 --2- 172.21.15.43:0/3949694137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6201048f0 0x7fb620199e00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00d770 tx=0x7fb60c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:12.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.247+0000 7fb61e000640 1 -- 172.21.15.43:0/3949694137 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c017070 con 0x7fb6201048f0 2024-03-26T08:31:12.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.247+0000 7fb61e000640 1 -- 172.21.15.43:0/3949694137 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb60c0054d0 con 0x7fb6201048f0 2024-03-26T08:31:12.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.247+0000 7fb626800640 1 -- 172.21.15.43:0/3949694137 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6201a17d0 con 0x7fb6201048f0 2024-03-26T08:31:12.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.248+0000 7fb61e000640 1 -- 172.21.15.43:0/3949694137 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c004cf0 con 0x7fb6201048f0 2024-03-26T08:31:12.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.248+0000 7fb626800640 1 -- 172.21.15.43:0/3949694137 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6201a1cd0 con 0x7fb6201048f0 2024-03-26T08:31:12.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.249+0000 7fb626800640 1 -- 172.21.15.43:0/3949694137 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb62010aa20 con 0x7fb6201048f0 2024-03-26T08:31:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.251+0000 7fb61e000640 1 -- 172.21.15.43:0/3949694137 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb60c004e90 con 0x7fb6201048f0 2024-03-26T08:31:12.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.253+0000 7fb61e000640 1 --2- 172.21.15.43:0/3949694137 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb6040a2800 0x7fb6040a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:12.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.253+0000 7fb61e000640 1 -- 172.21.15.43:0/3949694137 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb60c0ce0c0 con 0x7fb6201048f0 2024-03-26T08:31:12.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.253+0000 7fb625400640 1 --2- 172.21.15.43:0/3949694137 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb6040a2800 0x7fb6040a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:12.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.254+0000 7fb625400640 1 --2- 172.21.15.43:0/3949694137 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb6040a2800 0x7fb6040a4cc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb620104750 tx=0x7fb610009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:12.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.257+0000 7fb61e000640 1 -- 172.21.15.43:0/3949694137 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb60c09b560 con 0x7fb6201048f0 2024-03-26T08:31:12.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:12 smithi043 ceph-mon[31929]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:12.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.501+0000 7fb626800640 1 -- 172.21.15.43:0/3949694137 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb62002d070 con 0x7fb6201048f0 2024-03-26T08:31:12.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.503+0000 7fb61e000640 1 -- 172.21.15.43:0/3949694137 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9245 (secure 0 0 0) 0x7fb60c09ae60 con 0x7fb6201048f0 2024-03-26T08:31:12.499 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:31:12.499 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1452206807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1452206807},{"type":"v1","addr":"172.21.15.43:6835","nonce":1452206807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3151775467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3151775467},{"type":"v1","addr":"172.21.15.43:6837","nonce":3151775467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/1284655682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":1284655682},{"type":"v1","addr":"172.21.15.43:6839","nonce":1284655682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3081085787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3081085787},{"type":"v1","addr":"172.21.15.43:6841","nonce":3081085787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1842295306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1842295306},{"type":"v1","addr":"172.21.15.183:6835","nonce":1842295306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2267781666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2267781666},{"type":"v1","addr":"172.21.15.183:6837","nonce":2267781666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3572429948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3572429948},{"type":"v1","addr":"172.21.15.183:6839","nonce":3572429948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34448,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/1108327179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108327179},{"type":"v1","addr":"172.21.15.52:6833","nonce":1108327179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34487,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/742057410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":742057410},{"type":"v1","addr":"172.21.15.52:6835","nonce":742057410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34517,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/748239918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":748239918},{"type":"v1","addr":"172.21.15.52:6837","nonce":748239918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-26T08:31:12.499 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 12 2024-03-26T08:31:12.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.506+0000 7fb626800640 1 -- 172.21.15.43:0/3949694137 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb6040a2800 msgr2=0x7fb6040a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:12.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.506+0000 7fb626800640 1 --2- 172.21.15.43:0/3949694137 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb6040a2800 0x7fb6040a4cc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb620104750 tx=0x7fb610009670 comp rx=0 tx=0).stop 2024-03-26T08:31:12.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.506+0000 7fb626800640 1 -- 172.21.15.43:0/3949694137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6201048f0 msgr2=0x7fb620199e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:12.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.506+0000 7fb626800640 1 --2- 172.21.15.43:0/3949694137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6201048f0 0x7fb620199e00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00d770 tx=0x7fb60c00dc30 comp rx=0 tx=0).stop 2024-03-26T08:31:12.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.506+0000 7fb626800640 1 -- 172.21.15.43:0/3949694137 shutdown_connections 2024-03-26T08:31:12.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.506+0000 7fb626800640 1 --2- 172.21.15.43:0/3949694137 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb6040a2800 0x7fb6040a4cc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:12.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.506+0000 7fb626800640 1 --2- 172.21.15.43:0/3949694137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6201052b0 0x7fb6201a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:12.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.506+0000 7fb626800640 1 --2- 172.21.15.43:0/3949694137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6201048f0 0x7fb620199e00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:12.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.506+0000 7fb626800640 1 --2- 172.21.15.43:0/3949694137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb620103700 0x7fb6201998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:12.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.506+0000 7fb626800640 1 -- 172.21.15.43:0/3949694137 >> 172.21.15.43:0/3949694137 conn(0x7fb6200fec70 msgr2=0x7fb620107bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:12.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.506+0000 7fb626800640 1 -- 172.21.15.43:0/3949694137 shutdown_connections 2024-03-26T08:31:12.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:12.506+0000 7fb626800640 1 -- 172.21.15.43:0/3949694137 wait complete. 2024-03-26T08:31:12.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:12 smithi183 ceph-mon[35818]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:12.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:12 smithi052 ceph-mon[35071]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:13.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:13 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3949694137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:13.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:13 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3949694137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:13.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:13 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3949694137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:13.793 INFO:journalctl@ceph.mds.b.smithi052.stdout:Mar 26 08:31:13 smithi052 systemd[1]: Starting Ceph mds.b for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:31:14.073 INFO:journalctl@ceph.mds.b.smithi052.stdout:Mar 26 08:31:13 smithi052 podman[65588]: 2024-03-26 08:31:13.709270594 +0000 UTC m=+0.036692565 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:31:14.073 INFO:journalctl@ceph.mds.b.smithi052.stdout:Mar 26 08:31:13 smithi052 podman[65588]: 2024-03-26 08:31:13.862178112 +0000 UTC m=+0.189600106 container create 233fc0c0ef08c114f3418eb2e84b2474cc0f97f106577ebb6e2995081eada7f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-b, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T08:31:14.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:14 smithi043 ceph-mon[31929]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:14.520 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:31:14.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:14 smithi183 ceph-mon[35818]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:14.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:14 smithi052 ceph-mon[35071]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:14.573 INFO:journalctl@ceph.mds.b.smithi052.stdout:Mar 26 08:31:14 smithi052 podman[65588]: 2024-03-26 08:31:14.124321671 +0000 UTC m=+0.451743709 container init 233fc0c0ef08c114f3418eb2e84b2474cc0f97f106577ebb6e2995081eada7f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-b, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-26T08:31:14.573 INFO:journalctl@ceph.mds.b.smithi052.stdout:Mar 26 08:31:14 smithi052 podman[65588]: 2024-03-26 08:31:14.135270736 +0000 UTC m=+0.462692703 container start 233fc0c0ef08c114f3418eb2e84b2474cc0f97f106577ebb6e2995081eada7f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-b, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-26T08:31:14.573 INFO:journalctl@ceph.mds.b.smithi052.stdout:Mar 26 08:31:14 smithi052 ceph-mds[65607]: -- 172.21.15.52:0/4129283509 <== mon.2 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ac0aae7a40 con 0x55ac0aabb000 2024-03-26T08:31:14.573 INFO:journalctl@ceph.mds.b.smithi052.stdout:Mar 26 08:31:14 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-b[65603]: starting mds.b at 2024-03-26T08:31:14.573 INFO:journalctl@ceph.mds.b.smithi052.stdout:Mar 26 08:31:14 smithi052 bash[65588]: 233fc0c0ef08c114f3418eb2e84b2474cc0f97f106577ebb6e2995081eada7f3 2024-03-26T08:31:14.573 INFO:journalctl@ceph.mds.b.smithi052.stdout:Mar 26 08:31:14 smithi052 systemd[1]: Started Ceph mds.b for 88759420-eb49-11ee-95d0-87774f69a715. 2024-03-26T08:31:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.725+0000 7f740f200640 1 -- 172.21.15.43:0/1293550470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74081021c0 msgr2=0x7f74081045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.725+0000 7f740f200640 1 --2- 172.21.15.43:0/1293550470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74081021c0 0x7f74081045b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f73f8009900 tx=0x7f73f802bc30 comp rx=0 tx=0).stop 2024-03-26T08:31:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.726+0000 7f740f200640 1 -- 172.21.15.43:0/1293550470 shutdown_connections 2024-03-26T08:31:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.726+0000 7f740f200640 1 --2- 172.21.15.43:0/1293550470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7408107420 0x7f7408109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.726+0000 7f740f200640 1 --2- 172.21.15.43:0/1293550470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7408104af0 0x7f7408106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.726+0000 7f740f200640 1 --2- 172.21.15.43:0/1293550470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74081021c0 0x7f74081045b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.726+0000 7f740f200640 1 -- 172.21.15.43:0/1293550470 >> 172.21.15.43:0/1293550470 conn(0x7f74080fbbd0 msgr2=0x7f74080fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.726+0000 7f740f200640 1 -- 172.21.15.43:0/1293550470 shutdown_connections 2024-03-26T08:31:14.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.726+0000 7f740f200640 1 -- 172.21.15.43:0/1293550470 wait complete. 2024-03-26T08:31:14.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.727+0000 7f740f200640 1 Processor -- start 2024-03-26T08:31:14.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.727+0000 7f740f200640 1 -- start start 2024-03-26T08:31:14.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.727+0000 7f740f200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74081021c0 0x7f7408199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:14.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.727+0000 7f740f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7408104af0 0x7f740819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:14.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.727+0000 7f740f200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7408107420 0x7f74081a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:14.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.727+0000 7f740f200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740810d280 con 0x7f7408104af0 2024-03-26T08:31:14.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.727+0000 7f740f200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740810d100 con 0x7f74081021c0 2024-03-26T08:31:14.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.727+0000 7f740f200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740810d400 con 0x7f7408107420 2024-03-26T08:31:14.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.728+0000 7f7407e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74081021c0 0x7f7408199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:14.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.728+0000 7f7407e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74081021c0 0x7f7408199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.43:58102/0 (socket says 172.21.15.43:58102) 2024-03-26T08:31:14.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.728+0000 7f7407e00640 1 -- 172.21.15.43:0/3944964988 learned_addr learned my addr 172.21.15.43:0/3944964988 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:14.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.728+0000 7f7407400640 1 --2- 172.21.15.43:0/3944964988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7408104af0 0x7f740819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:14.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.728+0000 7f740d200640 1 --2- 172.21.15.43:0/3944964988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7408107420 0x7f74081a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:14.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.728+0000 7f7407e00640 1 -- 172.21.15.43:0/3944964988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7408107420 msgr2=0x7f74081a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:14.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.728+0000 7f7407e00640 1 --2- 172.21.15.43:0/3944964988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7408107420 0x7f74081a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:14.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.728+0000 7f7407e00640 1 -- 172.21.15.43:0/3944964988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7408104af0 msgr2=0x7f740819a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:14.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.728+0000 7f7407e00640 1 --2- 172.21.15.43:0/3944964988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7408104af0 0x7f740819a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:14.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.728+0000 7f7407e00640 1 -- 172.21.15.43:0/3944964988 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74081a1790 con 0x7f74081021c0 2024-03-26T08:31:14.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.728+0000 7f7407400640 1 --2- 172.21.15.43:0/3944964988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7408104af0 0x7f740819a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:31:14.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.729+0000 7f7407e00640 1 --2- 172.21.15.43:0/3944964988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74081021c0 0x7f7408199b70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f73f8000c00 tx=0x7f73f80042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:14.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.729+0000 7f7404c00640 1 -- 172.21.15.43:0/3944964988 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f8043020 con 0x7f74081021c0 2024-03-26T08:31:14.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.729+0000 7f7404c00640 1 -- 172.21.15.43:0/3944964988 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f73f8004aa0 con 0x7f74081021c0 2024-03-26T08:31:14.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.729+0000 7f7404c00640 1 -- 172.21.15.43:0/3944964988 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f8003d00 con 0x7f74081021c0 2024-03-26T08:31:14.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.729+0000 7f740f200640 1 -- 172.21.15.43:0/3944964988 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74081a1a20 con 0x7f74081021c0 2024-03-26T08:31:14.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.729+0000 7f740f200640 1 -- 172.21.15.43:0/3944964988 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74081a1f20 con 0x7f74081021c0 2024-03-26T08:31:14.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.730+0000 7f740f200640 1 -- 172.21.15.43:0/3944964988 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7408108f70 con 0x7f74081021c0 2024-03-26T08:31:14.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.733+0000 7f7404c00640 1 -- 172.21.15.43:0/3944964988 <== mon.2 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f73f8046020 con 0x7f74081021c0 2024-03-26T08:31:14.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.734+0000 7f7404c00640 1 --2- 172.21.15.43:0/3944964988 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f73ec0a2800 0x7f73ec0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:14.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.734+0000 7f7404c00640 1 -- 172.21.15.43:0/3944964988 <== mon.2 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f73f80f0d60 con 0x7f74081021c0 2024-03-26T08:31:14.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.734+0000 7f7407400640 1 --2- 172.21.15.43:0/3944964988 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f73ec0a2800 0x7f73ec0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:14.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.735+0000 7f7407400640 1 --2- 172.21.15.43:0/3944964988 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f73ec0a2800 0x7f73ec0a4cc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f740819b080 tx=0x7f73f400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:14.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.740+0000 7f7404c00640 1 -- 172.21.15.43:0/3944964988 <== mon.2 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f73f804d020 con 0x7f74081021c0 2024-03-26T08:31:14.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.988+0000 7f740f200640 1 -- 172.21.15.43:0/3944964988 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f740802d070 con 0x7f74081021c0 2024-03-26T08:31:14.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.991+0000 7f7404c00640 1 -- 172.21.15.43:0/3944964988 <== mon.2 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9245 (secure 0 0 0) 0x7f73f80be200 con 0x7f74081021c0 2024-03-26T08:31:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:31:14.984 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1452206807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1452206807},{"type":"v1","addr":"172.21.15.43:6835","nonce":1452206807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3151775467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3151775467},{"type":"v1","addr":"172.21.15.43:6837","nonce":3151775467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/1284655682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":1284655682},{"type":"v1","addr":"172.21.15.43:6839","nonce":1284655682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3081085787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3081085787},{"type":"v1","addr":"172.21.15.43:6841","nonce":3081085787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1842295306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1842295306},{"type":"v1","addr":"172.21.15.183:6835","nonce":1842295306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2267781666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2267781666},{"type":"v1","addr":"172.21.15.183:6837","nonce":2267781666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3572429948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3572429948},{"type":"v1","addr":"172.21.15.183:6839","nonce":3572429948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34448,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/1108327179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108327179},{"type":"v1","addr":"172.21.15.52:6833","nonce":1108327179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34487,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/742057410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":742057410},{"type":"v1","addr":"172.21.15.52:6835","nonce":742057410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34517,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/748239918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":748239918},{"type":"v1","addr":"172.21.15.52:6837","nonce":748239918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-26T08:31:14.985 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 12 2024-03-26T08:31:14.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.994+0000 7f740f200640 1 -- 172.21.15.43:0/3944964988 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f73ec0a2800 msgr2=0x7f73ec0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:14.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.994+0000 7f740f200640 1 --2- 172.21.15.43:0/3944964988 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f73ec0a2800 0x7f73ec0a4cc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f740819b080 tx=0x7f73f400b390 comp rx=0 tx=0).stop 2024-03-26T08:31:14.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.994+0000 7f740f200640 1 -- 172.21.15.43:0/3944964988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74081021c0 msgr2=0x7f7408199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:14.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.994+0000 7f740f200640 1 --2- 172.21.15.43:0/3944964988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74081021c0 0x7f7408199b70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f73f8000c00 tx=0x7f73f80042f0 comp rx=0 tx=0).stop 2024-03-26T08:31:14.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.994+0000 7f740f200640 1 -- 172.21.15.43:0/3944964988 shutdown_connections 2024-03-26T08:31:14.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.994+0000 7f740f200640 1 --2- 172.21.15.43:0/3944964988 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f73ec0a2800 0x7f73ec0a4cc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:14.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.994+0000 7f740f200640 1 --2- 172.21.15.43:0/3944964988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7408107420 0x7f74081a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:14.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.994+0000 7f740f200640 1 --2- 172.21.15.43:0/3944964988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7408104af0 0x7f740819a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:14.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.994+0000 7f740f200640 1 --2- 172.21.15.43:0/3944964988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74081021c0 0x7f7408199b70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:14.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.994+0000 7f740f200640 1 -- 172.21.15.43:0/3944964988 >> 172.21.15.43:0/3944964988 conn(0x7f74080fbbd0 msgr2=0x7f74081019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:14.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.994+0000 7f740f200640 1 -- 172.21.15.43:0/3944964988 shutdown_connections 2024-03-26T08:31:14.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:14.994+0000 7f740f200640 1 -- 172.21.15.43:0/3944964988 wait complete. 2024-03-26T08:31:15.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:15 smithi043 ceph-mon[31929]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:15.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:15 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:15.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:15 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:15.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:15 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:15.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:15 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:15.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:15 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:15.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:15 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:31:15.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:15 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3944964988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:15.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:15 smithi043 ceph-mon[31929]: mds.? [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] up:boot 2024-03-26T08:31:15.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:15 smithi043 ceph-mon[31929]: fsmap 12 up:standby 2024-03-26T08:31:15.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:15 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T08:31:15.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:15 smithi183 ceph-mon[35818]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:15.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:15 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:15.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:15 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:15.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:15 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:15.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:15 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:15.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:15 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:15.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:15 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:31:15.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:15 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3944964988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:15.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:15 smithi183 ceph-mon[35818]: mds.? [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] up:boot 2024-03-26T08:31:15.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:15 smithi183 ceph-mon[35818]: fsmap 12 up:standby 2024-03-26T08:31:15.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:15 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T08:31:15.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:15 smithi052 ceph-mon[35071]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:15.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:15 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:15.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:15 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:15.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:15 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:15.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:15 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:15.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:15 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:15.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:15 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:31:15.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:15 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3944964988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:15.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:15 smithi052 ceph-mon[35071]: mds.? [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] up:boot 2024-03-26T08:31:15.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:15 smithi052 ceph-mon[35071]: fsmap 12 up:standby 2024-03-26T08:31:15.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:15 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T08:31:17.006 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:31:17.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.228+0000 7f5b62400640 1 -- 172.21.15.43:0/18630754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b5c107420 msgr2=0x7f5b5c109870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:17.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.228+0000 7f5b62400640 1 --2- 172.21.15.43:0/18630754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b5c107420 0x7f5b5c109870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5b54009ca0 tx=0x7f5b5402c0f0 comp rx=0 tx=0).stop 2024-03-26T08:31:17.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.228+0000 7f5b62400640 1 -- 172.21.15.43:0/18630754 shutdown_connections 2024-03-26T08:31:17.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.228+0000 7f5b62400640 1 --2- 172.21.15.43:0/18630754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b5c107420 0x7f5b5c109870 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:17.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.228+0000 7f5b62400640 1 --2- 172.21.15.43:0/18630754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b5c104af0 0x7f5b5c106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:17.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.228+0000 7f5b62400640 1 --2- 172.21.15.43:0/18630754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b5c1021c0 0x7f5b5c1045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:17.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.228+0000 7f5b62400640 1 -- 172.21.15.43:0/18630754 >> 172.21.15.43:0/18630754 conn(0x7f5b5c0fbbd0 msgr2=0x7f5b5c0fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:17.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.228+0000 7f5b62400640 1 -- 172.21.15.43:0/18630754 shutdown_connections 2024-03-26T08:31:17.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.228+0000 7f5b62400640 1 -- 172.21.15.43:0/18630754 wait complete. 2024-03-26T08:31:17.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.229+0000 7f5b62400640 1 Processor -- start 2024-03-26T08:31:17.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.229+0000 7f5b62400640 1 -- start start 2024-03-26T08:31:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.230+0000 7f5b62400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b5c1021c0 0x7f5b5c1978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.230+0000 7f5b62400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b5c104af0 0x7f5b5c197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.230+0000 7f5b62400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b5c107420 0x7f5b5c19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.230+0000 7f5b62400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c10d220 con 0x7f5b5c1021c0 2024-03-26T08:31:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.230+0000 7f5b62400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c10d0a0 con 0x7f5b5c104af0 2024-03-26T08:31:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.230+0000 7f5b62400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c10d3a0 con 0x7f5b5c107420 2024-03-26T08:31:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.230+0000 7f5b5b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b5c1021c0 0x7f5b5c1978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.230+0000 7f5b5b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b5c1021c0 0x7f5b5c1978a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43958/0 (socket says 172.21.15.43:43958) 2024-03-26T08:31:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.230+0000 7f5b5b400640 1 -- 172.21.15.43:0/2364121368 learned_addr learned my addr 172.21.15.43:0/2364121368 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.230+0000 7f5b5aa00640 1 --2- 172.21.15.43:0/2364121368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b5c104af0 0x7f5b5c197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.230+0000 7f5b5be00640 1 --2- 172.21.15.43:0/2364121368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b5c107420 0x7f5b5c19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.230+0000 7f5b5b400640 1 -- 172.21.15.43:0/2364121368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b5c107420 msgr2=0x7f5b5c19ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.230+0000 7f5b5b400640 1 --2- 172.21.15.43:0/2364121368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b5c107420 0x7f5b5c19ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.230+0000 7f5b5b400640 1 -- 172.21.15.43:0/2364121368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b5c104af0 msgr2=0x7f5b5c197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:17.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.230+0000 7f5b5b400640 1 --2- 172.21.15.43:0/2364121368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b5c104af0 0x7f5b5c197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:17.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.230+0000 7f5b5b400640 1 -- 172.21.15.43:0/2364121368 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5c19f4c0 con 0x7f5b5c1021c0 2024-03-26T08:31:17.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.231+0000 7f5b5b400640 1 --2- 172.21.15.43:0/2364121368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b5c1021c0 0x7f5b5c1978a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c00d950 tx=0x7f5b4c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:17.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.231+0000 7f5b53e00640 1 -- 172.21.15.43:0/2364121368 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c017070 con 0x7f5b5c1021c0 2024-03-26T08:31:17.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.231+0000 7f5b62400640 1 -- 172.21.15.43:0/2364121368 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b5c19f7b0 con 0x7f5b5c1021c0 2024-03-26T08:31:17.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.231+0000 7f5b53e00640 1 -- 172.21.15.43:0/2364121368 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5b4c009c30 con 0x7f5b5c1021c0 2024-03-26T08:31:17.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.231+0000 7f5b53e00640 1 -- 172.21.15.43:0/2364121368 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c00f620 con 0x7f5b5c1021c0 2024-03-26T08:31:17.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.231+0000 7f5b62400640 1 -- 172.21.15.43:0/2364121368 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b5c19fd00 con 0x7f5b5c1021c0 2024-03-26T08:31:17.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.232+0000 7f5b62400640 1 -- 172.21.15.43:0/2364121368 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b5c0fd6c0 con 0x7f5b5c1021c0 2024-03-26T08:31:17.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.234+0000 7f5b53e00640 1 -- 172.21.15.43:0/2364121368 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5b4c0109a0 con 0x7f5b5c1021c0 2024-03-26T08:31:17.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.236+0000 7f5b53e00640 1 --2- 172.21.15.43:0/2364121368 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5b400a2800 0x7f5b400a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:17.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.236+0000 7f5b53e00640 1 -- 172.21.15.43:0/2364121368 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5b4c0ce360 con 0x7f5b5c1021c0 2024-03-26T08:31:17.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.236+0000 7f5b5aa00640 1 --2- 172.21.15.43:0/2364121368 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5b400a2800 0x7f5b400a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:17.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.237+0000 7f5b5aa00640 1 --2- 172.21.15.43:0/2364121368 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5b400a2800 0x7f5b400a4cc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5b48005fd0 tx=0x7f5b48005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:17.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.242+0000 7f5b53e00640 1 -- 172.21.15.43:0/2364121368 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5b4c009de0 con 0x7f5b5c1021c0 2024-03-26T08:31:17.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:17 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:17.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:17 smithi043 ceph-mon[31929]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:17.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.524+0000 7f5b62400640 1 -- 172.21.15.43:0/2364121368 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b5c02d070 con 0x7f5b5c1021c0 2024-03-26T08:31:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.525+0000 7f5b53e00640 1 -- 172.21.15.43:0/2364121368 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10037 (secure 0 0 0) 0x7f5b4c09b800 con 0x7f5b5c1021c0 2024-03-26T08:31:17.518 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:31:17.518 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1452206807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1452206807},{"type":"v1","addr":"172.21.15.43:6835","nonce":1452206807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3151775467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3151775467},{"type":"v1","addr":"172.21.15.43:6837","nonce":3151775467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/1284655682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":1284655682},{"type":"v1","addr":"172.21.15.43:6839","nonce":1284655682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3081085787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3081085787},{"type":"v1","addr":"172.21.15.43:6841","nonce":3081085787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1842295306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1842295306},{"type":"v1","addr":"172.21.15.183:6835","nonce":1842295306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2267781666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2267781666},{"type":"v1","addr":"172.21.15.183:6837","nonce":2267781666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3572429948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3572429948},{"type":"v1","addr":"172.21.15.183:6839","nonce":3572429948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34448,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/1108327179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108327179},{"type":"v1","addr":"172.21.15.52:6833","nonce":1108327179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34487,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/742057410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":742057410},{"type":"v1","addr":"172.21.15.52:6835","nonce":742057410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34517,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/748239918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":748239918},{"type":"v1","addr":"172.21.15.52:6837","nonce":748239918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":34559,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2551965101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2551965101},{"type":"v1","addr":"172.21.15.52:6839","nonce":2551965101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-03-26T08:31:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 13 2024-03-26T08:31:17.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.527+0000 7f5b62400640 1 -- 172.21.15.43:0/2364121368 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5b400a2800 msgr2=0x7f5b400a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:17.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.527+0000 7f5b62400640 1 --2- 172.21.15.43:0/2364121368 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5b400a2800 0x7f5b400a4cc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5b48005fd0 tx=0x7f5b48005850 comp rx=0 tx=0).stop 2024-03-26T08:31:17.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.527+0000 7f5b62400640 1 -- 172.21.15.43:0/2364121368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b5c1021c0 msgr2=0x7f5b5c1978a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:17.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.527+0000 7f5b62400640 1 --2- 172.21.15.43:0/2364121368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b5c1021c0 0x7f5b5c1978a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c00d950 tx=0x7f5b4c00de10 comp rx=0 tx=0).stop 2024-03-26T08:31:17.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.528+0000 7f5b62400640 1 -- 172.21.15.43:0/2364121368 shutdown_connections 2024-03-26T08:31:17.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.528+0000 7f5b62400640 1 --2- 172.21.15.43:0/2364121368 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5b400a2800 0x7f5b400a4cc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:17.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.528+0000 7f5b62400640 1 --2- 172.21.15.43:0/2364121368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b5c107420 0x7f5b5c19ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:17.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.528+0000 7f5b62400640 1 --2- 172.21.15.43:0/2364121368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b5c104af0 0x7f5b5c197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:17.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.528+0000 7f5b62400640 1 --2- 172.21.15.43:0/2364121368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b5c1021c0 0x7f5b5c1978a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:17.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.528+0000 7f5b62400640 1 -- 172.21.15.43:0/2364121368 >> 172.21.15.43:0/2364121368 conn(0x7f5b5c0fbbd0 msgr2=0x7f5b5c102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:17.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.528+0000 7f5b62400640 1 -- 172.21.15.43:0/2364121368 shutdown_connections 2024-03-26T08:31:17.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.528+0000 7f5b62400640 1 -- 172.21.15.43:0/2364121368 wait complete. 2024-03-26T08:31:17.543 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:31:17.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:17 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:17.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:17 smithi183 ceph-mon[35818]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:17.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:17 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:17.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:17 smithi052 ceph-mon[35071]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:17.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.773+0000 7f1f32600640 1 -- 172.21.15.43:0/518293546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f2c077c40 msgr2=0x7f1f2c0784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:17.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.773+0000 7f1f32600640 1 --2- 172.21.15.43:0/518293546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f2c077c40 0x7f1f2c0784e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009ca0 tx=0x7f1f2402c100 comp rx=0 tx=0).stop 2024-03-26T08:31:17.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.773+0000 7f1f32600640 1 -- 172.21.15.43:0/518293546 shutdown_connections 2024-03-26T08:31:17.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.774+0000 7f1f32600640 1 --2- 172.21.15.43:0/518293546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f2c077c40 0x7f1f2c0784e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:17.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.774+0000 7f1f32600640 1 --2- 172.21.15.43:0/518293546 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f2c0772a0 0x7f1f2c077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:17.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.774+0000 7f1f32600640 1 --2- 172.21.15.43:0/518293546 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f2c076060 0x7f1f2c076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:17.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.774+0000 7f1f32600640 1 -- 172.21.15.43:0/518293546 >> 172.21.15.43:0/518293546 conn(0x7f1f2c0fee50 msgr2=0x7f1f2c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:17.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.774+0000 7f1f32600640 1 -- 172.21.15.43:0/518293546 shutdown_connections 2024-03-26T08:31:17.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.774+0000 7f1f32600640 1 -- 172.21.15.43:0/518293546 wait complete. 2024-03-26T08:31:17.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.775+0000 7f1f32600640 1 Processor -- start 2024-03-26T08:31:17.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.775+0000 7f1f32600640 1 -- start start 2024-03-26T08:31:17.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.775+0000 7f1f32600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f2c076060 0x7f1f2c19de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:17.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.775+0000 7f1f32600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f2c0772a0 0x7f1f2c19e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:17.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.775+0000 7f1f32600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f2c077c40 0x7f1f2c1a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:17.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.775+0000 7f1f32600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c113820 con 0x7f1f2c076060 2024-03-26T08:31:17.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.775+0000 7f1f32600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c1136a0 con 0x7f1f2c0772a0 2024-03-26T08:31:17.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.775+0000 7f1f32600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c1139a0 con 0x7f1f2c077c40 2024-03-26T08:31:17.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.775+0000 7f1f2b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f2c076060 0x7f1f2c19de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:17.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.776+0000 7f1f2b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f2c076060 0x7f1f2c19de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43972/0 (socket says 172.21.15.43:43972) 2024-03-26T08:31:17.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.776+0000 7f1f2b400640 1 -- 172.21.15.43:0/3232559326 learned_addr learned my addr 172.21.15.43:0/3232559326 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:17.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.776+0000 7f1f2be00640 1 --2- 172.21.15.43:0/3232559326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f2c077c40 0x7f1f2c1a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:17.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.776+0000 7f1f2aa00640 1 --2- 172.21.15.43:0/3232559326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f2c0772a0 0x7f1f2c19e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:17.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.776+0000 7f1f2b400640 1 -- 172.21.15.43:0/3232559326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f2c077c40 msgr2=0x7f1f2c1a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:17.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.776+0000 7f1f2b400640 1 --2- 172.21.15.43:0/3232559326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f2c077c40 0x7f1f2c1a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:17.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.776+0000 7f1f2b400640 1 -- 172.21.15.43:0/3232559326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f2c0772a0 msgr2=0x7f1f2c19e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:17.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.776+0000 7f1f2b400640 1 --2- 172.21.15.43:0/3232559326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f2c0772a0 0x7f1f2c19e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:17.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.776+0000 7f1f2b400640 1 -- 172.21.15.43:0/3232559326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2c1a5a40 con 0x7f1f2c076060 2024-03-26T08:31:17.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.776+0000 7f1f2b400640 1 --2- 172.21.15.43:0/3232559326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f2c076060 0x7f1f2c19de20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00d770 tx=0x7f1f1c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:17.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.777+0000 7f1f23e00640 1 -- 172.21.15.43:0/3232559326 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c017070 con 0x7f1f2c076060 2024-03-26T08:31:17.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.777+0000 7f1f23e00640 1 -- 172.21.15.43:0/3232559326 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1f1c0054d0 con 0x7f1f2c076060 2024-03-26T08:31:17.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.777+0000 7f1f32600640 1 -- 172.21.15.43:0/3232559326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f2c1a5d30 con 0x7f1f2c076060 2024-03-26T08:31:17.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.777+0000 7f1f23e00640 1 -- 172.21.15.43:0/3232559326 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c004cf0 con 0x7f1f2c076060 2024-03-26T08:31:17.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.777+0000 7f1f32600640 1 -- 172.21.15.43:0/3232559326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2c1a61d0 con 0x7f1f2c076060 2024-03-26T08:31:17.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.778+0000 7f1f32600640 1 -- 172.21.15.43:0/3232559326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f2c1103a0 con 0x7f1f2c076060 2024-03-26T08:31:17.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.781+0000 7f1f23e00640 1 -- 172.21.15.43:0/3232559326 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1f1c004e90 con 0x7f1f2c076060 2024-03-26T08:31:17.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.783+0000 7f1f23e00640 1 --2- 172.21.15.43:0/3232559326 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1f100a2880 0x7f1f100a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:17.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.783+0000 7f1f23e00640 1 -- 172.21.15.43:0/3232559326 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1f1c0cdfa0 con 0x7f1f2c076060 2024-03-26T08:31:17.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.783+0000 7f1f2aa00640 1 --2- 172.21.15.43:0/3232559326 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1f100a2880 0x7f1f100a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:17.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.784+0000 7f1f2aa00640 1 --2- 172.21.15.43:0/3232559326 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1f100a2880 0x7f1f100a4d40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1f18005e10 tx=0x7f1f18005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:17.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.788+0000 7f1f23e00640 1 -- 172.21.15.43:0/3232559326 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1f1c09b4e0 con 0x7f1f2c076060 2024-03-26T08:31:17.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:17.998+0000 7f1f32600640 1 -- 172.21.15.43:0/3232559326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f2c0622a0 con 0x7f1f2c076060 2024-03-26T08:31:17.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.001+0000 7f1f23e00640 1 -- 172.21.15.43:0/3232559326 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18408 (secure 0 0 0) 0x7f1f1c0ce1a0 con 0x7f1f2c076060 2024-03-26T08:31:17.994 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:31:17.995 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":82,"fsid":"88759420-eb49-11ee-95d0-87774f69a715","created":"2024-03-26T08:20:27.817107+0000","modified":"2024-03-26T08:30:11.537628+0000","last_up_change":"2024-03-26T08:30:10.520976+0000","last_in_change":"2024-03-26T08:29:46.394360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:24:54.852427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"28670673-9193-4fa3-9cc0-c1a5a6b27284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6803","nonce":936601146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6805","nonce":936601146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6809","nonce":936601146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6807","nonce":936601146}]},"public_addr":"172.21.15.43:6803/936601146","cluster_addr":"172.21.15.43:6805/936601146","heartbeat_back_addr":"172.21.15.43:6809/936601146","heartbeat_front_addr":"172.21.15.43:6807/936601146","state":["exists","up"]},{"osd":1,"uuid":"fa8ad6b6-277a-42b8-b0f8-b36fc219de2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6801","nonce":3268412328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6803","nonce":3268412328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6807","nonce":3268412328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6805","nonce":3268412328}]},"public_addr":"172.21.15.52:6801/3268412328","cluster_addr":"172.21.15.52:6803/3268412328","heartbeat_back_addr":"172.21.15.52:6807/3268412328","heartbeat_front_addr":"172.21.15.52:6805/3268412328","state":["exists","up"]},{"osd":2,"uuid":"ec0039c3-f9bd-419a-81de-fa4418359808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6801","nonce":2103180908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6803","nonce":2103180908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6807","nonce":2103180908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6805","nonce":2103180908}]},"public_addr":"172.21.15.183:6801/2103180908","cluster_addr":"172.21.15.183:6803/2103180908","heartbeat_back_addr":"172.21.15.183:6807/2103180908","heartbeat_front_addr":"172.21.15.183:6805/2103180908","state":["exists","up"]},{"osd":3,"uuid":"e7dcbf00-6586-4664-a69e-3cb0df8dfd30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6811","nonce":3722846033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6813","nonce":3722846033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6817","nonce":3722846033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6815","nonce":3722846033}]},"public_addr":"172.21.15.43:6811/3722846033","cluster_addr":"172.21.15.43:6813/3722846033","heartbeat_back_addr":"172.21.15.43:6817/3722846033","heartbeat_front_addr":"172.21.15.43:6815/3722846033","state":["exists","up"]},{"osd":4,"uuid":"db089a49-5e98-432d-9687-cdc044114c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6809","nonce":4028436523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6811","nonce":4028436523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6815","nonce":4028436523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6813","nonce":4028436523}]},"public_addr":"172.21.15.52:6809/4028436523","cluster_addr":"172.21.15.52:6811/4028436523","heartbeat_back_addr":"172.21.15.52:6815/4028436523","heartbeat_front_addr":"172.21.15.52:6813/4028436523","state":["exists","up"]},{"osd":5,"uuid":"556ac804-02ea-423b-a599-55baf5be4ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6809","nonce":787132252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6811","nonce":787132252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6815","nonce":787132252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6813","nonce":787132252}]},"public_addr":"172.21.15.183:6809/787132252","cluster_addr":"172.21.15.183:6811/787132252","heartbeat_back_addr":"172.21.15.183:6815/787132252","heartbeat_front_addr":"172.21.15.183:6813/787132252","state":["exists","up"]},{"osd":6,"uuid":"3de2a49d-6cef-4580-9d8a-f90cdfcecd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6819","nonce":4206861158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6821","nonce":4206861158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6825","nonce":4206861158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6823","nonce":4206861158}]},"public_addr":"172.21.15.43:6819/4206861158","cluster_addr":"172.21.15.43:6821/4206861158","heartbeat_back_addr":"172.21.15.43:6825/4206861158","heartbeat_front_addr":"172.21.15.43:6823/4206861158","state":["exists","up"]},{"osd":7,"uuid":"a3b832f7-7d7d-4a62-b679-db6da98275d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6817","nonce":3452064146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6819","nonce":3452064146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6823","nonce":3452064146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6821","nonce":3452064146}]},"public_addr":"172.21.15.52:6817/3452064146","cluster_addr":"172.21.15.52:6819/3452064146","heartbeat_back_addr":"172.21.15.52:6823/3452064146","heartbeat_front_addr":"172.21.15.52:6821/3452064146","state":["exists","up"]},{"osd":8,"uuid":"ddc6b07f-d703-414d-a59a-02316376b13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6817","nonce":1023131725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6819","nonce":1023131725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6823","nonce":1023131725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6821","nonce":1023131725}]},"public_addr":"172.21.15.183:6817/1023131725","cluster_addr":"172.21.15.183:6819/1023131725","heartbeat_back_addr":"172.21.15.183:6823/1023131725","heartbeat_front_addr":"172.21.15.183:6821/1023131725","state":["exists","up"]},{"osd":9,"uuid":"ac16bb0e-6232-45c8-b16a-4d878874c588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6827","nonce":3935195657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6829","nonce":3935195657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6833","nonce":3935195657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6831","nonce":3935195657}]},"public_addr":"172.21.15.43:6827/3935195657","cluster_addr":"172.21.15.43:6829/3935195657","heartbeat_back_addr":"172.21.15.43:6833/3935195657","heartbeat_front_addr":"172.21.15.43:6831/3935195657","state":["exists","up"]},{"osd":10,"uuid":"fd839188-f398-4cff-aa7b-36522ff1a8a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6825","nonce":377628320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6827","nonce":377628320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6831","nonce":377628320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6829","nonce":377628320}]},"public_addr":"172.21.15.52:6825/377628320","cluster_addr":"172.21.15.52:6827/377628320","heartbeat_back_addr":"172.21.15.52:6831/377628320","heartbeat_front_addr":"172.21.15.52:6829/377628320","state":["exists","up"]},{"osd":11,"uuid":"a40282a0-0c64-49ee-b64e-6f3f8119e827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6825","nonce":1373086433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6827","nonce":1373086433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6831","nonce":1373086433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6829","nonce":1373086433}]},"public_addr":"172.21.15.183:6825/1373086433","cluster_addr":"172.21.15.183:6827/1373086433","heartbeat_back_addr":"172.21.15.183:6831/1373086433","heartbeat_front_addr":"172.21.15.183:6829/1373086433","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:23:43.478905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:17.573532+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:50.286610+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:25:27.407663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:03.081294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:36.057416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:12.062013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:47.032271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:21.023201+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:57.364739+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:29:34.039995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:30:08.827577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6801/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/4240775295":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/510203140":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/4251957687":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:6801/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1885244665":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/298320867":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1301092625":"2024-03-27T08:21:36.149092+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:31:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.004+0000 7f1f32600640 1 -- 172.21.15.43:0/3232559326 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1f100a2880 msgr2=0x7f1f100a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.004+0000 7f1f32600640 1 --2- 172.21.15.43:0/3232559326 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1f100a2880 0x7f1f100a4d40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1f18005e10 tx=0x7f1f18005da0 comp rx=0 tx=0).stop 2024-03-26T08:31:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.004+0000 7f1f32600640 1 -- 172.21.15.43:0/3232559326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f2c076060 msgr2=0x7f1f2c19de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.004+0000 7f1f32600640 1 --2- 172.21.15.43:0/3232559326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f2c076060 0x7f1f2c19de20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00d770 tx=0x7f1f1c00dc30 comp rx=0 tx=0).stop 2024-03-26T08:31:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.004+0000 7f1f32600640 1 -- 172.21.15.43:0/3232559326 shutdown_connections 2024-03-26T08:31:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.004+0000 7f1f32600640 1 --2- 172.21.15.43:0/3232559326 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1f100a2880 0x7f1f100a4d40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.004+0000 7f1f32600640 1 --2- 172.21.15.43:0/3232559326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f2c077c40 0x7f1f2c1a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:17.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.004+0000 7f1f32600640 1 --2- 172.21.15.43:0/3232559326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f2c0772a0 0x7f1f2c19e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:17.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.004+0000 7f1f32600640 1 --2- 172.21.15.43:0/3232559326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f2c076060 0x7f1f2c19de20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:17.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.004+0000 7f1f32600640 1 -- 172.21.15.43:0/3232559326 >> 172.21.15.43:0/3232559326 conn(0x7f1f2c0fee50 msgr2=0x7f1f2c1004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:17.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.004+0000 7f1f32600640 1 -- 172.21.15.43:0/3232559326 shutdown_connections 2024-03-26T08:31:17.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.004+0000 7f1f32600640 1 -- 172.21.15.43:0/3232559326 wait complete. 2024-03-26T08:31:18.019 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T08:24:54.852427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T08:31:18.020 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T08:31:18.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.239+0000 7fb571a00640 1 -- 172.21.15.43:0/2874487430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb56c1038e0 msgr2=0x7fb56c103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:18.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.239+0000 7fb571a00640 1 --2- 172.21.15.43:0/2874487430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb56c1038e0 0x7fb56c103d00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb55c009860 tx=0x7fb55c02bb70 comp rx=0 tx=0).stop 2024-03-26T08:31:18.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.240+0000 7fb571a00640 1 -- 172.21.15.43:0/2874487430 shutdown_connections 2024-03-26T08:31:18.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.240+0000 7fb571a00640 1 --2- 172.21.15.43:0/2874487430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb56c105490 0x7fb56c10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:18.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.240+0000 7fb571a00640 1 --2- 172.21.15.43:0/2874487430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb56c104ad0 0x7fb56c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:18.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.240+0000 7fb571a00640 1 --2- 172.21.15.43:0/2874487430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb56c1038e0 0x7fb56c103d00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:18.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.240+0000 7fb571a00640 1 -- 172.21.15.43:0/2874487430 >> 172.21.15.43:0/2874487430 conn(0x7fb56c0fee50 msgr2=0x7fb56c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:18.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.240+0000 7fb571a00640 1 -- 172.21.15.43:0/2874487430 shutdown_connections 2024-03-26T08:31:18.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.240+0000 7fb571a00640 1 -- 172.21.15.43:0/2874487430 wait complete. 2024-03-26T08:31:18.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.241+0000 7fb571a00640 1 Processor -- start 2024-03-26T08:31:18.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.241+0000 7fb571a00640 1 -- start start 2024-03-26T08:31:18.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.241+0000 7fb571a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb56c1038e0 0x7fb56c199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:18.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.241+0000 7fb571a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb56c104ad0 0x7fb56c19a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:18.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.241+0000 7fb571a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb56c105490 0x7fb56c1a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:18.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.241+0000 7fb571a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c10f400 con 0x7fb56c1038e0 2024-03-26T08:31:18.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.241+0000 7fb571a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c10f280 con 0x7fb56c104ad0 2024-03-26T08:31:18.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.241+0000 7fb571a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c10f580 con 0x7fb56c105490 2024-03-26T08:31:18.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.242+0000 7fb56aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb56c1038e0 0x7fb56c199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:18.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.242+0000 7fb56aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb56c1038e0 0x7fb56c199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43988/0 (socket says 172.21.15.43:43988) 2024-03-26T08:31:18.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.242+0000 7fb56aa00640 1 -- 172.21.15.43:0/2851290259 learned_addr learned my addr 172.21.15.43:0/2851290259 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:18.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.242+0000 7fb56a000640 1 --2- 172.21.15.43:0/2851290259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb56c104ad0 0x7fb56c19a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:18.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.242+0000 7fb56b400640 1 --2- 172.21.15.43:0/2851290259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb56c105490 0x7fb56c1a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:18.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.242+0000 7fb56aa00640 1 -- 172.21.15.43:0/2851290259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb56c105490 msgr2=0x7fb56c1a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:18.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.242+0000 7fb56aa00640 1 --2- 172.21.15.43:0/2851290259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb56c105490 0x7fb56c1a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:18.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.242+0000 7fb56aa00640 1 -- 172.21.15.43:0/2851290259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb56c104ad0 msgr2=0x7fb56c19a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:18.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.242+0000 7fb56aa00640 1 --2- 172.21.15.43:0/2851290259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb56c104ad0 0x7fb56c19a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:18.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.242+0000 7fb56aa00640 1 -- 172.21.15.43:0/2851290259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56c1a17d0 con 0x7fb56c1038e0 2024-03-26T08:31:18.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.243+0000 7fb56aa00640 1 --2- 172.21.15.43:0/2851290259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb56c1038e0 0x7fb56c199bb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb55c005940 tx=0x7fb55c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:18.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.243+0000 7fb563400640 1 -- 172.21.15.43:0/2851290259 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c041020 con 0x7fb56c1038e0 2024-03-26T08:31:18.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.243+0000 7fb563400640 1 -- 172.21.15.43:0/2851290259 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb55c0035f0 con 0x7fb56c1038e0 2024-03-26T08:31:18.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.243+0000 7fb571a00640 1 -- 172.21.15.43:0/2851290259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb56c1a1a60 con 0x7fb56c1038e0 2024-03-26T08:31:18.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.243+0000 7fb563400640 1 -- 172.21.15.43:0/2851290259 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c039040 con 0x7fb56c1038e0 2024-03-26T08:31:18.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.243+0000 7fb571a00640 1 -- 172.21.15.43:0/2851290259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb56c1a1f50 con 0x7fb56c1038e0 2024-03-26T08:31:18.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.244+0000 7fb571a00640 1 -- 172.21.15.43:0/2851290259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb56c10ad20 con 0x7fb56c1038e0 2024-03-26T08:31:18.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.247+0000 7fb563400640 1 -- 172.21.15.43:0/2851290259 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb55c044020 con 0x7fb56c1038e0 2024-03-26T08:31:18.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.249+0000 7fb563400640 1 --2- 172.21.15.43:0/2851290259 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb5500a2800 0x7fb5500a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:18.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.249+0000 7fb563400640 1 -- 172.21.15.43:0/2851290259 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb55c0f0440 con 0x7fb56c1038e0 2024-03-26T08:31:18.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.249+0000 7fb56a000640 1 --2- 172.21.15.43:0/2851290259 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb5500a2800 0x7fb5500a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:18.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.250+0000 7fb56a000640 1 --2- 172.21.15.43:0/2851290259 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb5500a2800 0x7fb5500a4cc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb56c19b0c0 tx=0x7fb558005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:18.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.252+0000 7fb563400640 1 -- 172.21.15.43:0/2851290259 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb55c04b020 con 0x7fb56c1038e0 2024-03-26T08:31:18.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.458+0000 7fb571a00640 1 -- 172.21.15.43:0/2851290259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb56c0622a0 con 0x7fb56c1038e0 2024-03-26T08:31:18.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.459+0000 7fb563400640 1 -- 172.21.15.43:0/2851290259 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb55c0bda00 con 0x7fb56c1038e0 2024-03-26T08:31:18.451 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-26T08:31:18.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.461+0000 7fb571a00640 1 -- 172.21.15.43:0/2851290259 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb5500a2800 msgr2=0x7fb5500a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:18.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.461+0000 7fb571a00640 1 --2- 172.21.15.43:0/2851290259 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb5500a2800 0x7fb5500a4cc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb56c19b0c0 tx=0x7fb558005f90 comp rx=0 tx=0).stop 2024-03-26T08:31:18.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.461+0000 7fb571a00640 1 -- 172.21.15.43:0/2851290259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb56c1038e0 msgr2=0x7fb56c199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:18.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.461+0000 7fb571a00640 1 --2- 172.21.15.43:0/2851290259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb56c1038e0 0x7fb56c199bb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb55c005940 tx=0x7fb55c0042f0 comp rx=0 tx=0).stop 2024-03-26T08:31:18.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.461+0000 7fb571a00640 1 -- 172.21.15.43:0/2851290259 shutdown_connections 2024-03-26T08:31:18.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.461+0000 7fb571a00640 1 --2- 172.21.15.43:0/2851290259 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb5500a2800 0x7fb5500a4cc0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:18.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.461+0000 7fb571a00640 1 --2- 172.21.15.43:0/2851290259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb56c105490 0x7fb56c1a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:18.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.461+0000 7fb571a00640 1 --2- 172.21.15.43:0/2851290259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb56c104ad0 0x7fb56c19a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:18.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.461+0000 7fb571a00640 1 --2- 172.21.15.43:0/2851290259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb56c1038e0 0x7fb56c199bb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:18.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.461+0000 7fb571a00640 1 -- 172.21.15.43:0/2851290259 >> 172.21.15.43:0/2851290259 conn(0x7fb56c0fee50 msgr2=0x7fb56c107ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:18.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.461+0000 7fb571a00640 1 -- 172.21.15.43:0/2851290259 shutdown_connections 2024-03-26T08:31:18.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.461+0000 7fb571a00640 1 -- 172.21.15.43:0/2851290259 wait complete. 2024-03-26T08:31:18.474 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-26T08:31:18.541 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:18 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2364121368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:18.541 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:18 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:18.541 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:18 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:18.541 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:18 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:18.541 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:18 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:18.541 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:18 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3232559326' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:31:18.542 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:18 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:18.542 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:18 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:18.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:18 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2364121368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:18.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:18 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:18.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:18 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:18.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:18 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:18.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:18 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:18.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:18 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3232559326' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:31:18.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:18 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:18.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:18 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:18.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:18 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2364121368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:18.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:18 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:18.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:18 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:18.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:18 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:18.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:18 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:18.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:18 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3232559326' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:31:18.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:18 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:18.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:18 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:18.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.688+0000 7f0cdc000640 1 -- 172.21.15.43:0/3054134874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd4103270 msgr2=0x7f0cd4109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:18.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.688+0000 7f0cdc000640 1 --2- 172.21.15.43:0/3054134874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd4103270 0x7f0cd4109b00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc009ca0 tx=0x7f0ccc02c0f0 comp rx=0 tx=0).stop 2024-03-26T08:31:18.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.688+0000 7f0cdc000640 1 -- 172.21.15.43:0/3054134874 shutdown_connections 2024-03-26T08:31:18.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.688+0000 7f0cdc000640 1 --2- 172.21.15.43:0/3054134874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd4103270 0x7f0cd4109b00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:18.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.688+0000 7f0cdc000640 1 --2- 172.21.15.43:0/3054134874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0cd41028b0 0x7f0cd4102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:18.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.688+0000 7f0cdc000640 1 --2- 172.21.15.43:0/3054134874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cd41016c0 0x7f0cd4101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:18.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.688+0000 7f0cdc000640 1 -- 172.21.15.43:0/3054134874 >> 172.21.15.43:0/3054134874 conn(0x7f0cd40fcc50 msgr2=0x7f0cd40ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:18.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.688+0000 7f0cdc000640 1 -- 172.21.15.43:0/3054134874 shutdown_connections 2024-03-26T08:31:18.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.688+0000 7f0cdc000640 1 -- 172.21.15.43:0/3054134874 wait complete. 2024-03-26T08:31:18.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.690+0000 7f0cdc000640 1 Processor -- start 2024-03-26T08:31:18.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.690+0000 7f0cdc000640 1 -- start start 2024-03-26T08:31:18.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.690+0000 7f0cdc000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0cd41016c0 0x7f0cd4075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:18.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.690+0000 7f0cdc000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cd41028b0 0x7f0cd4073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:18.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.691+0000 7f0cdc000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd4103270 0x7f0cd4073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:18.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.691+0000 7f0cdc000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd410d1d0 con 0x7f0cd4103270 2024-03-26T08:31:18.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.691+0000 7f0cdc000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd410d050 con 0x7f0cd41016c0 2024-03-26T08:31:18.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.691+0000 7f0cdc000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd410d350 con 0x7f0cd41028b0 2024-03-26T08:31:18.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.691+0000 7f0cda000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd4103270 0x7f0cd4073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:18.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.691+0000 7f0cda000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd4103270 0x7f0cd4073fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44012/0 (socket says 172.21.15.43:44012) 2024-03-26T08:31:18.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.691+0000 7f0cda000640 1 -- 172.21.15.43:0/2018213704 learned_addr learned my addr 172.21.15.43:0/2018213704 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:18.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.691+0000 7f0cd9600640 1 --2- 172.21.15.43:0/2018213704 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0cd41016c0 0x7f0cd4075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:18.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.691+0000 7f0cd8c00640 1 --2- 172.21.15.43:0/2018213704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cd41028b0 0x7f0cd4073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:18.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.691+0000 7f0cda000640 1 -- 172.21.15.43:0/2018213704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cd41028b0 msgr2=0x7f0cd4073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:18.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.691+0000 7f0cda000640 1 --2- 172.21.15.43:0/2018213704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cd41028b0 0x7f0cd4073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:18.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.691+0000 7f0cda000640 1 -- 172.21.15.43:0/2018213704 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0cd41016c0 msgr2=0x7f0cd4075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:18.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.691+0000 7f0cda000640 1 --2- 172.21.15.43:0/2018213704 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0cd41016c0 0x7f0cd4075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:18.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.691+0000 7f0cda000640 1 -- 172.21.15.43:0/2018213704 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd4074860 con 0x7f0cd4103270 2024-03-26T08:31:18.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.692+0000 7f0cda000640 1 --2- 172.21.15.43:0/2018213704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd4103270 0x7f0cd4073fd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc02c700 tx=0x7f0ccc0328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:18.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.692+0000 7f0cd2000640 1 -- 172.21.15.43:0/2018213704 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc041020 con 0x7f0cd4103270 2024-03-26T08:31:18.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.692+0000 7f0cd2000640 1 -- 172.21.15.43:0/2018213704 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0ccc003e20 con 0x7f0cd4103270 2024-03-26T08:31:18.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.692+0000 7f0cdc000640 1 -- 172.21.15.43:0/2018213704 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd41a1980 con 0x7f0cd4103270 2024-03-26T08:31:18.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.692+0000 7f0cd2000640 1 -- 172.21.15.43:0/2018213704 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc039040 con 0x7f0cd4103270 2024-03-26T08:31:18.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.692+0000 7f0cdc000640 1 -- 172.21.15.43:0/2018213704 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd41a1ed0 con 0x7f0cd4103270 2024-03-26T08:31:18.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.694+0000 7f0cdc000640 1 -- 172.21.15.43:0/2018213704 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd40688e0 con 0x7f0cd4103270 2024-03-26T08:31:18.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.696+0000 7f0cd2000640 1 -- 172.21.15.43:0/2018213704 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0ccc044020 con 0x7f0cd4103270 2024-03-26T08:31:18.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.698+0000 7f0cd2000640 1 --2- 172.21.15.43:0/2018213704 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0cbc0a2800 0x7f0cbc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:18.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.698+0000 7f0cd2000640 1 -- 172.21.15.43:0/2018213704 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0ccc0f0b70 con 0x7f0cd4103270 2024-03-26T08:31:18.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.699+0000 7f0cd9600640 1 --2- 172.21.15.43:0/2018213704 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0cbc0a2800 0x7f0cbc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:18.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.699+0000 7f0cd9600640 1 --2- 172.21.15.43:0/2018213704 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0cbc0a2800 0x7f0cbc0a4cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8005fd0 tx=0x7f0cc8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:18.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.702+0000 7f0cd2000640 1 -- 172.21.15.43:0/2018213704 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0ccc04b020 con 0x7f0cd4103270 2024-03-26T08:31:18.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.915+0000 7f0cdc000640 1 -- 172.21.15.43:0/2018213704 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0cd40622a0 con 0x7f0cd4103270 2024-03-26T08:31:18.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.915+0000 7f0cd2000640 1 -- 172.21.15.43:0/2018213704 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0ccc03b2f0 con 0x7f0cd4103270 2024-03-26T08:31:18.908 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:31:18.908 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-03-26T08:31:18.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:31:18.909 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2024-03-26T08:31:18.909 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2024-03-26T08:31:18.909 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:31:18.909 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-03-26T08:31:18.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.918+0000 7f0cdc000640 1 -- 172.21.15.43:0/2018213704 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0cbc0a2800 msgr2=0x7f0cbc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:18.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.918+0000 7f0cdc000640 1 --2- 172.21.15.43:0/2018213704 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0cbc0a2800 0x7f0cbc0a4cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8005fd0 tx=0x7f0cc8005850 comp rx=0 tx=0).stop 2024-03-26T08:31:18.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.918+0000 7f0cdc000640 1 -- 172.21.15.43:0/2018213704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd4103270 msgr2=0x7f0cd4073fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:18.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.918+0000 7f0cdc000640 1 --2- 172.21.15.43:0/2018213704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd4103270 0x7f0cd4073fd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc02c700 tx=0x7f0ccc0328a0 comp rx=0 tx=0).stop 2024-03-26T08:31:18.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.919+0000 7f0cdc000640 1 -- 172.21.15.43:0/2018213704 shutdown_connections 2024-03-26T08:31:18.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.919+0000 7f0cdc000640 1 --2- 172.21.15.43:0/2018213704 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0cbc0a2800 0x7f0cbc0a4cc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:18.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.919+0000 7f0cdc000640 1 --2- 172.21.15.43:0/2018213704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cd4103270 0x7f0cd4073fd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:18.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.919+0000 7f0cdc000640 1 --2- 172.21.15.43:0/2018213704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cd41028b0 0x7f0cd4073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:18.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.919+0000 7f0cdc000640 1 --2- 172.21.15.43:0/2018213704 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0cd41016c0 0x7f0cd4075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:18.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.919+0000 7f0cdc000640 1 -- 172.21.15.43:0/2018213704 >> 172.21.15.43:0/2018213704 conn(0x7f0cd40fcc50 msgr2=0x7f0cd4105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:18.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.919+0000 7f0cdc000640 1 -- 172.21.15.43:0/2018213704 shutdown_connections 2024-03-26T08:31:18.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:18.919+0000 7f0cdc000640 1 -- 172.21.15.43:0/2018213704 wait complete. 2024-03-26T08:31:18.931 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-26T08:31:18.932 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-26T08:31:19.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.162+0000 7f3534800640 1 -- 172.21.15.43:0/3378712408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f352c1038e0 msgr2=0x7f352c103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:19.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.162+0000 7f3534800640 1 --2- 172.21.15.43:0/3378712408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f352c1038e0 0x7f352c103d00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3524009860 tx=0x7f352402bb70 comp rx=0 tx=0).stop 2024-03-26T08:31:19.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.162+0000 7f3534800640 1 -- 172.21.15.43:0/3378712408 shutdown_connections 2024-03-26T08:31:19.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.162+0000 7f3534800640 1 --2- 172.21.15.43:0/3378712408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f352c105490 0x7f352c10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:19.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.162+0000 7f3534800640 1 --2- 172.21.15.43:0/3378712408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f352c104ad0 0x7f352c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:19.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.162+0000 7f3534800640 1 --2- 172.21.15.43:0/3378712408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f352c1038e0 0x7f352c103d00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:19.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.162+0000 7f3534800640 1 -- 172.21.15.43:0/3378712408 >> 172.21.15.43:0/3378712408 conn(0x7f352c0fee50 msgr2=0x7f352c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:19.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.162+0000 7f3534800640 1 -- 172.21.15.43:0/3378712408 shutdown_connections 2024-03-26T08:31:19.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.162+0000 7f3534800640 1 -- 172.21.15.43:0/3378712408 wait complete. 2024-03-26T08:31:19.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.163+0000 7f3534800640 1 Processor -- start 2024-03-26T08:31:19.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.163+0000 7f3534800640 1 -- start start 2024-03-26T08:31:19.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.164+0000 7f3534800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f352c1038e0 0x7f352c199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:19.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.164+0000 7f3534800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f352c104ad0 0x7f352c19a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:19.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.164+0000 7f3534800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f352c105490 0x7f352c1a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:19.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.164+0000 7f3534800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c10f3e0 con 0x7f352c105490 2024-03-26T08:31:19.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.164+0000 7f3534800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c10f260 con 0x7f352c104ad0 2024-03-26T08:31:19.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.164+0000 7f3534800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c10f560 con 0x7f352c1038e0 2024-03-26T08:31:19.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.164+0000 7f3532800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f352c105490 0x7f352c1a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:19.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.164+0000 7f3532800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f352c105490 0x7f352c1a1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44022/0 (socket says 172.21.15.43:44022) 2024-03-26T08:31:19.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.164+0000 7f3532800640 1 -- 172.21.15.43:0/2972934693 learned_addr learned my addr 172.21.15.43:0/2972934693 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:19.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.164+0000 7f3531400640 1 --2- 172.21.15.43:0/2972934693 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f352c104ad0 0x7f352c19a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:19.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.164+0000 7f3531e00640 1 --2- 172.21.15.43:0/2972934693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f352c1038e0 0x7f352c199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:19.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.164+0000 7f3532800640 1 -- 172.21.15.43:0/2972934693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f352c1038e0 msgr2=0x7f352c199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:19.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.164+0000 7f3532800640 1 --2- 172.21.15.43:0/2972934693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f352c1038e0 0x7f352c199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:19.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.164+0000 7f3532800640 1 -- 172.21.15.43:0/2972934693 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f352c104ad0 msgr2=0x7f352c19a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:19.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.164+0000 7f3532800640 1 --2- 172.21.15.43:0/2972934693 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f352c104ad0 0x7f352c19a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:19.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.164+0000 7f3532800640 1 -- 172.21.15.43:0/2972934693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352c1a1790 con 0x7f352c105490 2024-03-26T08:31:19.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.165+0000 7f3532800640 1 --2- 172.21.15.43:0/2972934693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f352c105490 0x7f352c1a1120 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f352800eb10 tx=0x7f352800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:19.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.165+0000 7f3522a00640 1 -- 172.21.15.43:0/2972934693 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3528004b20 con 0x7f352c105490 2024-03-26T08:31:19.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.165+0000 7f3522a00640 1 -- 172.21.15.43:0/2972934693 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3528004cc0 con 0x7f352c105490 2024-03-26T08:31:19.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.165+0000 7f3534800640 1 -- 172.21.15.43:0/2972934693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f352c1a1a80 con 0x7f352c105490 2024-03-26T08:31:19.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.165+0000 7f3522a00640 1 -- 172.21.15.43:0/2972934693 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3528010690 con 0x7f352c105490 2024-03-26T08:31:19.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.165+0000 7f3534800640 1 -- 172.21.15.43:0/2972934693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f352c1a1fd0 con 0x7f352c105490 2024-03-26T08:31:19.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.166+0000 7f3534800640 1 -- 172.21.15.43:0/2972934693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f352c10ad70 con 0x7f352c105490 2024-03-26T08:31:19.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.168+0000 7f3522a00640 1 -- 172.21.15.43:0/2972934693 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f3528005020 con 0x7f352c105490 2024-03-26T08:31:19.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.170+0000 7f3522a00640 1 --2- 172.21.15.43:0/2972934693 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f35180a2880 0x7f35180a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.170+0000 7f3522a00640 1 -- 172.21.15.43:0/2972934693 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f35280cece0 con 0x7f352c105490 2024-03-26T08:31:19.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.170+0000 7f3531e00640 1 --2- 172.21.15.43:0/2972934693 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f35180a2880 0x7f35180a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:19.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.171+0000 7f3531e00640 1 --2- 172.21.15.43:0/2972934693 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f35180a2880 0x7f35180a4d40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f352402c180 tx=0x7f3524034980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:19.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.176+0000 7f3522a00640 1 -- 172.21.15.43:0/2972934693 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f352809c180 con 0x7f352c105490 2024-03-26T08:31:19.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:19.388+0000 7f3534800640 1 -- 172.21.15.43:0/2972934693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f352c02d070 con 0x7f352c105490 2024-03-26T08:31:19.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:19 smithi183 ceph-mon[35818]: pgmap v363: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:19.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:19 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2851290259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:31:19.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:19 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2018213704' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-26T08:31:19.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:19 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:31:19.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:19 smithi052 ceph-mon[35071]: pgmap v363: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:19.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:19 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2851290259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:31:19.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:19 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2018213704' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-26T08:31:19.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:19 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:31:19.653 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:19 smithi043 ceph-mon[31929]: pgmap v363: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:19.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:19 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2851290259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:31:19.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:19 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2018213704' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-26T08:31:19.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:19 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:31:20.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.303+0000 7f3522a00640 1 -- 172.21.15.43:0/2972934693 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7f352809ba80 con 0x7f352c105490 2024-03-26T08:31:20.297 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2024-03-26T08:31:20.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.307+0000 7f3534800640 1 -- 172.21.15.43:0/2972934693 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f35180a2880 msgr2=0x7f35180a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:20.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.307+0000 7f3534800640 1 --2- 172.21.15.43:0/2972934693 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f35180a2880 0x7f35180a4d40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f352402c180 tx=0x7f3524034980 comp rx=0 tx=0).stop 2024-03-26T08:31:20.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.307+0000 7f3534800640 1 -- 172.21.15.43:0/2972934693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f352c105490 msgr2=0x7f352c1a1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:20.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.307+0000 7f3534800640 1 --2- 172.21.15.43:0/2972934693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f352c105490 0x7f352c1a1120 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f352800eb10 tx=0x7f352800c2d0 comp rx=0 tx=0).stop 2024-03-26T08:31:20.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.308+0000 7f3534800640 1 -- 172.21.15.43:0/2972934693 shutdown_connections 2024-03-26T08:31:20.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.308+0000 7f3534800640 1 --2- 172.21.15.43:0/2972934693 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f35180a2880 0x7f35180a4d40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:20.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.308+0000 7f3534800640 1 --2- 172.21.15.43:0/2972934693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f352c105490 0x7f352c1a1120 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:20.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.308+0000 7f3534800640 1 --2- 172.21.15.43:0/2972934693 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f352c104ad0 0x7f352c19a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:20.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.308+0000 7f3534800640 1 --2- 172.21.15.43:0/2972934693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f352c1038e0 0x7f352c199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:20.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.308+0000 7f3534800640 1 -- 172.21.15.43:0/2972934693 >> 172.21.15.43:0/2972934693 conn(0x7f352c0fee50 msgr2=0x7f352c107f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:20.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.308+0000 7f3534800640 1 -- 172.21.15.43:0/2972934693 shutdown_connections 2024-03-26T08:31:20.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.308+0000 7f3534800640 1 -- 172.21.15.43:0/2972934693 wait complete. 2024-03-26T08:31:20.328 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-26T08:31:20.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.542+0000 7f517ec00640 1 -- 172.21.15.43:0/2802492863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5178101210 msgr2=0x7f5178101690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:20.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.542+0000 7f517ec00640 1 --2- 172.21.15.43:0/2802492863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5178101210 0x7f5178101690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5170009ca0 tx=0x7f517002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:31:20.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.542+0000 7f517ec00640 1 -- 172.21.15.43:0/2802492863 shutdown_connections 2024-03-26T08:31:20.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.542+0000 7f517ec00640 1 --2- 172.21.15.43:0/2802492863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5178101210 0x7f5178101690 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:20.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.542+0000 7f517ec00640 1 --2- 172.21.15.43:0/2802492863 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5178100870 0x7f5178100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:20.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.542+0000 7f517ec00640 1 --2- 172.21.15.43:0/2802492863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51780fff10 0x7f5178100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:20.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.542+0000 7f517ec00640 1 -- 172.21.15.43:0/2802492863 >> 172.21.15.43:0/2802492863 conn(0x7f51780fbb30 msgr2=0x7f51780fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:20.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.542+0000 7f517ec00640 1 -- 172.21.15.43:0/2802492863 shutdown_connections 2024-03-26T08:31:20.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.542+0000 7f517ec00640 1 -- 172.21.15.43:0/2802492863 wait complete. 2024-03-26T08:31:20.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.543+0000 7f517ec00640 1 Processor -- start 2024-03-26T08:31:20.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.544+0000 7f517ec00640 1 -- start start 2024-03-26T08:31:20.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.544+0000 7f517ec00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51780fff10 0x7f5178199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:20.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.544+0000 7f517ec00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5178100870 0x7f5178199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:20.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.544+0000 7f517ec00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5178101210 0x7f51781a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:20.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.544+0000 7f517ec00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517810f3a0 con 0x7f5178101210 2024-03-26T08:31:20.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.544+0000 7f517ec00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517810f220 con 0x7f5178100870 2024-03-26T08:31:20.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.544+0000 7f517ec00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517810f520 con 0x7f51780fff10 2024-03-26T08:31:20.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.544+0000 7f517cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5178101210 0x7f51781a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:20.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.544+0000 7f517cc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5178101210 0x7f51781a1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44038/0 (socket says 172.21.15.43:44038) 2024-03-26T08:31:20.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.544+0000 7f517cc00640 1 -- 172.21.15.43:0/922516349 learned_addr learned my addr 172.21.15.43:0/922516349 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:20.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.544+0000 7f5177e00640 1 --2- 172.21.15.43:0/922516349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51780fff10 0x7f5178199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:20.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.544+0000 7f5177400640 1 --2- 172.21.15.43:0/922516349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5178100870 0x7f5178199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:20.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.545+0000 7f517cc00640 1 -- 172.21.15.43:0/922516349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51780fff10 msgr2=0x7f5178199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:20.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.545+0000 7f517cc00640 1 --2- 172.21.15.43:0/922516349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51780fff10 0x7f5178199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:20.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.545+0000 7f517cc00640 1 -- 172.21.15.43:0/922516349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5178100870 msgr2=0x7f5178199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:20.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.545+0000 7f517cc00640 1 --2- 172.21.15.43:0/922516349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5178100870 0x7f5178199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:20.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.545+0000 7f517cc00640 1 -- 172.21.15.43:0/922516349 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51781a16c0 con 0x7f5178101210 2024-03-26T08:31:20.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.545+0000 7f5177e00640 1 --2- 172.21.15.43:0/922516349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51780fff10 0x7f5178199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:31:20.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.545+0000 7f517cc00640 1 --2- 172.21.15.43:0/922516349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5178101210 0x7f51781a1050 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5170032380 tx=0x7f5170032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:20.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.545+0000 7f5174c00640 1 -- 172.21.15.43:0/922516349 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5170041020 con 0x7f5178101210 2024-03-26T08:31:20.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.545+0000 7f5174c00640 1 -- 172.21.15.43:0/922516349 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5170003e20 con 0x7f5178101210 2024-03-26T08:31:20.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.545+0000 7f5174c00640 1 -- 172.21.15.43:0/922516349 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5170039040 con 0x7f5178101210 2024-03-26T08:31:20.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.545+0000 7f517ec00640 1 -- 172.21.15.43:0/922516349 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51781a1950 con 0x7f5178101210 2024-03-26T08:31:20.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.546+0000 7f517ec00640 1 -- 172.21.15.43:0/922516349 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51781a1e40 con 0x7f5178101210 2024-03-26T08:31:20.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.547+0000 7f517ec00640 1 -- 172.21.15.43:0/922516349 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f517810ae30 con 0x7f5178101210 2024-03-26T08:31:20.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.549+0000 7f5174c00640 1 -- 172.21.15.43:0/922516349 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5170044020 con 0x7f5178101210 2024-03-26T08:31:20.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.550+0000 7f5174c00640 1 --2- 172.21.15.43:0/922516349 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f515c0a2800 0x7f515c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:20.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.551+0000 7f5174c00640 1 -- 172.21.15.43:0/922516349 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f51700f0b30 con 0x7f5178101210 2024-03-26T08:31:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.551+0000 7f5177e00640 1 --2- 172.21.15.43:0/922516349 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f515c0a2800 0x7f515c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.552+0000 7f5177e00640 1 --2- 172.21.15.43:0/922516349 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f515c0a2800 0x7f515c0a4cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f51680058c0 tx=0x7f5168005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:20.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:20 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:31:20.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:20 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:20.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:20 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2972934693' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T08:31:20.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:20 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:31:20.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.555+0000 7f5174c00640 1 -- 172.21.15.43:0/922516349 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f517004b020 con 0x7f5178101210 2024-03-26T08:31:20.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:20 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:31:20.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:20 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:20.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:20 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2972934693' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T08:31:20.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:20 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:31:20.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:20 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:31:20.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:20 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:20.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:20 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2972934693' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T08:31:20.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:20 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:31:20.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:20.770+0000 7f517ec00640 1 -- 172.21.15.43:0/922516349 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f51780622a0 con 0x7f5178101210 2024-03-26T08:31:21.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.327+0000 7f5174c00640 1 -- 172.21.15.43:0/922516349 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v84) v1 ==== 171+0+0 (secure 0 0 0) 0x7f51700bde60 con 0x7f5178101210 2024-03-26T08:31:21.321 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2024-03-26T08:31:21.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.330+0000 7f517ec00640 1 -- 172.21.15.43:0/922516349 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f515c0a2800 msgr2=0x7f515c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:21.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.330+0000 7f517ec00640 1 --2- 172.21.15.43:0/922516349 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f515c0a2800 0x7f515c0a4cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f51680058c0 tx=0x7f5168005850 comp rx=0 tx=0).stop 2024-03-26T08:31:21.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.330+0000 7f517ec00640 1 -- 172.21.15.43:0/922516349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5178101210 msgr2=0x7f51781a1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:21.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.330+0000 7f517ec00640 1 --2- 172.21.15.43:0/922516349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5178101210 0x7f51781a1050 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5170032380 tx=0x7f5170032820 comp rx=0 tx=0).stop 2024-03-26T08:31:21.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.330+0000 7f517ec00640 1 -- 172.21.15.43:0/922516349 shutdown_connections 2024-03-26T08:31:21.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.330+0000 7f517ec00640 1 --2- 172.21.15.43:0/922516349 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f515c0a2800 0x7f515c0a4cc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:21.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.330+0000 7f517ec00640 1 --2- 172.21.15.43:0/922516349 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5178101210 0x7f51781a1050 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:21.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.330+0000 7f517ec00640 1 --2- 172.21.15.43:0/922516349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5178100870 0x7f5178199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:21.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.330+0000 7f517ec00640 1 --2- 172.21.15.43:0/922516349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51780fff10 0x7f5178199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:21.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.330+0000 7f517ec00640 1 -- 172.21.15.43:0/922516349 >> 172.21.15.43:0/922516349 conn(0x7f51780fbb30 msgr2=0x7f51780fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:21.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.331+0000 7f517ec00640 1 -- 172.21.15.43:0/922516349 shutdown_connections 2024-03-26T08:31:21.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.331+0000 7f517ec00640 1 -- 172.21.15.43:0/922516349 wait complete. 2024-03-26T08:31:21.362 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-26T08:31:21.419 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:21 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2972934693' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-26T08:31:21.419 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:21 smithi043 ceph-mon[31929]: osdmap e83: 12 total, 12 up, 12 in 2024-03-26T08:31:21.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.555+0000 7f80ede00640 1 -- 172.21.15.43:0/2507394361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80e810dc80 msgr2=0x7f80e8110070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:21.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.555+0000 7f80ede00640 1 --2- 172.21.15.43:0/2507394361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80e810dc80 0x7f80e8110070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009930 tx=0x7f80e002bc30 comp rx=0 tx=0).stop 2024-03-26T08:31:21.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.555+0000 7f80ede00640 1 -- 172.21.15.43:0/2507394361 shutdown_connections 2024-03-26T08:31:21.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.555+0000 7f80ede00640 1 --2- 172.21.15.43:0/2507394361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80e810dc80 0x7f80e8110070 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:21.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.555+0000 7f80ede00640 1 --2- 172.21.15.43:0/2507394361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80e8074100 0x7f80e8074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:21.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.555+0000 7f80ede00640 1 --2- 172.21.15.43:0/2507394361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80e80756e0 0x7f80e8073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:21.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.555+0000 7f80ede00640 1 -- 172.21.15.43:0/2507394361 >> 172.21.15.43:0/2507394361 conn(0x7f80e80fcc20 msgr2=0x7f80e80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:21.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.555+0000 7f80ede00640 1 -- 172.21.15.43:0/2507394361 shutdown_connections 2024-03-26T08:31:21.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.555+0000 7f80ede00640 1 -- 172.21.15.43:0/2507394361 wait complete. 2024-03-26T08:31:21.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.556+0000 7f80ede00640 1 Processor -- start 2024-03-26T08:31:21.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.556+0000 7f80ede00640 1 -- start start 2024-03-26T08:31:21.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:21 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2972934693' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-26T08:31:21.550 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:21 smithi183 ceph-mon[35818]: osdmap e83: 12 total, 12 up, 12 in 2024-03-26T08:31:21.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.556+0000 7f80ede00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80e8074100 0x7f80e819de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:21.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.556+0000 7f80ede00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80e80756e0 0x7f80e819e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:21.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.556+0000 7f80ede00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80e810dc80 0x7f80e81a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:21.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.556+0000 7f80ede00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e8113820 con 0x7f80e80756e0 2024-03-26T08:31:21.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.556+0000 7f80ede00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e81136a0 con 0x7f80e8074100 2024-03-26T08:31:21.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.556+0000 7f80ede00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e81139a0 con 0x7f80e810dc80 2024-03-26T08:31:21.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.557+0000 7f80e6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80e80756e0 0x7f80e819e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:21.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.557+0000 7f80e6000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80e80756e0 0x7f80e819e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44066/0 (socket says 172.21.15.43:44066) 2024-03-26T08:31:21.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.557+0000 7f80e6000640 1 -- 172.21.15.43:0/3750791709 learned_addr learned my addr 172.21.15.43:0/3750791709 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:21.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.557+0000 7f80e7400640 1 --2- 172.21.15.43:0/3750791709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80e810dc80 0x7f80e81a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:21.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.557+0000 7f80e6a00640 1 --2- 172.21.15.43:0/3750791709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80e8074100 0x7f80e819de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:21.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.557+0000 7f80e6000640 1 -- 172.21.15.43:0/3750791709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80e810dc80 msgr2=0x7f80e81a5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:21.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.557+0000 7f80e6000640 1 --2- 172.21.15.43:0/3750791709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80e810dc80 0x7f80e81a5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:21.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.557+0000 7f80e6000640 1 -- 172.21.15.43:0/3750791709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80e8074100 msgr2=0x7f80e819de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:21.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.557+0000 7f80e6000640 1 --2- 172.21.15.43:0/3750791709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80e8074100 0x7f80e819de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:21.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.557+0000 7f80e6000640 1 -- 172.21.15.43:0/3750791709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e81a5aa0 con 0x7f80e80756e0 2024-03-26T08:31:21.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.558+0000 7f80e6000640 1 --2- 172.21.15.43:0/3750791709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80e80756e0 0x7f80e819e3c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f80d400c920 tx=0x7f80d400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:21.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.558+0000 7f80eca00640 1 -- 172.21.15.43:0/3750791709 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d4007c70 con 0x7f80e80756e0 2024-03-26T08:31:21.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.558+0000 7f80eca00640 1 -- 172.21.15.43:0/3750791709 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f80d4007e10 con 0x7f80e80756e0 2024-03-26T08:31:21.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.558+0000 7f80ede00640 1 -- 172.21.15.43:0/3750791709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e81a5d90 con 0x7f80e80756e0 2024-03-26T08:31:21.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.558+0000 7f80eca00640 1 -- 172.21.15.43:0/3750791709 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d4018820 con 0x7f80e80756e0 2024-03-26T08:31:21.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.558+0000 7f80ede00640 1 -- 172.21.15.43:0/3750791709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e81a6260 con 0x7f80e80756e0 2024-03-26T08:31:21.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.559+0000 7f80ede00640 1 -- 172.21.15.43:0/3750791709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80e8103d40 con 0x7f80e80756e0 2024-03-26T08:31:21.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.562+0000 7f80eca00640 1 -- 172.21.15.43:0/3750791709 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f80d40189c0 con 0x7f80e80756e0 2024-03-26T08:31:21.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.563+0000 7f80eca00640 1 --2- 172.21.15.43:0/3750791709 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f80cc0a2880 0x7f80cc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:21.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.564+0000 7f80eca00640 1 -- 172.21.15.43:0/3750791709 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f80d40ced50 con 0x7f80e80756e0 2024-03-26T08:31:21.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.564+0000 7f80e6a00640 1 --2- 172.21.15.43:0/3750791709 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f80cc0a2880 0x7f80cc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:21.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.564+0000 7f80e6a00640 1 --2- 172.21.15.43:0/3750791709 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f80cc0a2880 0x7f80cc0a4d40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f80d8005fd0 tx=0x7f80d8009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:21.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.567+0000 7f80eca00640 1 -- 172.21.15.43:0/3750791709 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f80d4014030 con 0x7f80e80756e0 2024-03-26T08:31:21.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:21 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2972934693' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-26T08:31:21.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:21 smithi052 ceph-mon[35071]: osdmap e83: 12 total, 12 up, 12 in 2024-03-26T08:31:21.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:21.816+0000 7f80ede00640 1 -- 172.21.15.43:0/3750791709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f80e802d070 con 0x7f80e80756e0 2024-03-26T08:31:22.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.367+0000 7f80eca00640 1 -- 172.21.15.43:0/3750791709 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f80d409c010 con 0x7f80e80756e0 2024-03-26T08:31:22.369 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 2 and data pool 3 2024-03-26T08:31:22.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.383+0000 7f80ede00640 1 -- 172.21.15.43:0/3750791709 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f80cc0a2880 msgr2=0x7f80cc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:22.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.383+0000 7f80ede00640 1 --2- 172.21.15.43:0/3750791709 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f80cc0a2880 0x7f80cc0a4d40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f80d8005fd0 tx=0x7f80d8009870 comp rx=0 tx=0).stop 2024-03-26T08:31:22.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.383+0000 7f80ede00640 1 -- 172.21.15.43:0/3750791709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80e80756e0 msgr2=0x7f80e819e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:22.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.383+0000 7f80ede00640 1 --2- 172.21.15.43:0/3750791709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80e80756e0 0x7f80e819e3c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f80d400c920 tx=0x7f80d400cde0 comp rx=0 tx=0).stop 2024-03-26T08:31:22.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.383+0000 7f80ede00640 1 -- 172.21.15.43:0/3750791709 shutdown_connections 2024-03-26T08:31:22.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.383+0000 7f80ede00640 1 --2- 172.21.15.43:0/3750791709 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f80cc0a2880 0x7f80cc0a4d40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:22.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.383+0000 7f80ede00640 1 --2- 172.21.15.43:0/3750791709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80e810dc80 0x7f80e81a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:22.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.383+0000 7f80ede00640 1 --2- 172.21.15.43:0/3750791709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80e80756e0 0x7f80e819e3c0 secure :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f80d400c920 tx=0x7f80d400cde0 comp rx=0 tx=0).stop 2024-03-26T08:31:22.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.383+0000 7f80ede00640 1 --2- 172.21.15.43:0/3750791709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80e8074100 0x7f80e819de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:22.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.383+0000 7f80ede00640 1 -- 172.21.15.43:0/3750791709 >> 172.21.15.43:0/3750791709 conn(0x7f80e80fcc20 msgr2=0x7f80e8100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:22.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.387+0000 7f80ede00640 1 -- 172.21.15.43:0/3750791709 shutdown_connections 2024-03-26T08:31:22.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.388+0000 7f80ede00640 1 -- 172.21.15.43:0/3750791709 wait complete. 2024-03-26T08:31:22.410 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-26T08:31:22.411 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-26T08:31:22.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.623+0000 7f6d2fe00640 1 -- 172.21.15.43:0/395441851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d30101120 msgr2=0x7f6d301015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:22.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.623+0000 7f6d2fe00640 1 --2- 172.21.15.43:0/395441851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d30101120 0x7f6d301015a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6d20009860 tx=0x7f6d2002bb70 comp rx=0 tx=0).stop 2024-03-26T08:31:22.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.624+0000 7f6d2fe00640 1 -- 172.21.15.43:0/395441851 shutdown_connections 2024-03-26T08:31:22.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.624+0000 7f6d2fe00640 1 --2- 172.21.15.43:0/395441851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d30101120 0x7f6d301015a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:22.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.624+0000 7f6d2fe00640 1 --2- 172.21.15.43:0/395441851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d30100780 0x7f6d30100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:22.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.624+0000 7f6d2fe00640 1 --2- 172.21.15.43:0/395441851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d300ffe20 0x7f6d30100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:22.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.624+0000 7f6d2fe00640 1 -- 172.21.15.43:0/395441851 >> 172.21.15.43:0/395441851 conn(0x7f6d300fba20 msgr2=0x7f6d300fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:22.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.624+0000 7f6d2fe00640 1 -- 172.21.15.43:0/395441851 shutdown_connections 2024-03-26T08:31:22.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.624+0000 7f6d2fe00640 1 -- 172.21.15.43:0/395441851 wait complete. 2024-03-26T08:31:22.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.625+0000 7f6d2fe00640 1 Processor -- start 2024-03-26T08:31:22.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.625+0000 7f6d2fe00640 1 -- start start 2024-03-26T08:31:22.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.626+0000 7f6d2fe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d300ffe20 0x7f6d30199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:22.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.626+0000 7f6d2fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d30100780 0x7f6d30199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:22.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.626+0000 7f6d2fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d30101120 0x7f6d301a0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:22.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.626+0000 7f6d2fe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3010f2b0 con 0x7f6d30101120 2024-03-26T08:31:22.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.626+0000 7f6d2fe00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3010f130 con 0x7f6d300ffe20 2024-03-26T08:31:22.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.626+0000 7f6d2fe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3010f430 con 0x7f6d30100780 2024-03-26T08:31:22.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.626+0000 7f6d2f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d30101120 0x7f6d301a0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:22.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.626+0000 7f6d2ea00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d300ffe20 0x7f6d30199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:22.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.626+0000 7f6d2f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d30101120 0x7f6d301a0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44080/0 (socket says 172.21.15.43:44080) 2024-03-26T08:31:22.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.626+0000 7f6d2f400640 1 -- 172.21.15.43:0/3127227346 learned_addr learned my addr 172.21.15.43:0/3127227346 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:22.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.626+0000 7f6d2e000640 1 --2- 172.21.15.43:0/3127227346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d30100780 0x7f6d30199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:22.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.627+0000 7f6d2ea00640 1 -- 172.21.15.43:0/3127227346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d30100780 msgr2=0x7f6d30199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:22.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.627+0000 7f6d2ea00640 1 --2- 172.21.15.43:0/3127227346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d30100780 0x7f6d30199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:22.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.627+0000 7f6d2ea00640 1 -- 172.21.15.43:0/3127227346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d30101120 msgr2=0x7f6d301a0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:22.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.627+0000 7f6d2ea00640 1 --2- 172.21.15.43:0/3127227346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d30101120 0x7f6d301a0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:22.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.627+0000 7f6d2ea00640 1 -- 172.21.15.43:0/3127227346 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d301a15a0 con 0x7f6d300ffe20 2024-03-26T08:31:22.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.627+0000 7f6d2f400640 1 --2- 172.21.15.43:0/3127227346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d30101120 0x7f6d301a0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:31:22.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.627+0000 7f6d2ea00640 1 --2- 172.21.15.43:0/3127227346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d300ffe20 0x7f6d30199980 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6d28009fd0 tx=0x7f6d2800c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:22.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.627+0000 7f6d27400640 1 -- 172.21.15.43:0/3127227346 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d28003970 con 0x7f6d300ffe20 2024-03-26T08:31:22.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.627+0000 7f6d2fe00640 1 -- 172.21.15.43:0/3127227346 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d301a1890 con 0x7f6d300ffe20 2024-03-26T08:31:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.628+0000 7f6d27400640 1 -- 172.21.15.43:0/3127227346 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6d28004020 con 0x7f6d300ffe20 2024-03-26T08:31:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.628+0000 7f6d2fe00640 1 -- 172.21.15.43:0/3127227346 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d301a1e30 con 0x7f6d300ffe20 2024-03-26T08:31:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.628+0000 7f6d27400640 1 -- 172.21.15.43:0/3127227346 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2800b950 con 0x7f6d300ffe20 2024-03-26T08:31:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.629+0000 7f6d2fe00640 1 -- 172.21.15.43:0/3127227346 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d3010ad00 con 0x7f6d300ffe20 2024-03-26T08:31:22.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.633+0000 7f6d27400640 1 -- 172.21.15.43:0/3127227346 <== mon.2 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f6d280041d0 con 0x7f6d300ffe20 2024-03-26T08:31:22.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.634+0000 7f6d27400640 1 --2- 172.21.15.43:0/3127227346 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6d140a2800 0x7f6d140a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:22.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.634+0000 7f6d27400640 1 -- 172.21.15.43:0/3127227346 <== mon.2 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6d280cf0d0 con 0x7f6d300ffe20 2024-03-26T08:31:22.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.635+0000 7f6d2e000640 1 --2- 172.21.15.43:0/3127227346 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6d140a2800 0x7f6d140a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:22.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.635+0000 7f6d2e000640 1 --2- 172.21.15.43:0/3127227346 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6d140a2800 0x7f6d140a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6d3019ae90 tx=0x7f6d18005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:22.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.647+0000 7f6d27400640 1 -- 172.21.15.43:0/3127227346 <== mon.2 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6d28024070 con 0x7f6d300ffe20 2024-03-26T08:31:22.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:22 smithi043 ceph-mon[31929]: pgmap v365: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:22.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:22 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/922516349' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-26T08:31:22.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:22 smithi043 ceph-mon[31929]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T08:31:22.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:22 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/922516349' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-26T08:31:22.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:22 smithi043 ceph-mon[31929]: osdmap e84: 12 total, 12 up, 12 in 2024-03-26T08:31:22.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:22 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3750791709' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-26T08:31:22.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:22 smithi043 ceph-mon[31929]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-26T08:31:22.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:22 smithi183 ceph-mon[35818]: pgmap v365: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:22.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:22 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/922516349' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-26T08:31:22.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:22 smithi183 ceph-mon[35818]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T08:31:22.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:22 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/922516349' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-26T08:31:22.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:22 smithi183 ceph-mon[35818]: osdmap e84: 12 total, 12 up, 12 in 2024-03-26T08:31:22.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:22 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3750791709' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-26T08:31:22.800 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:22 smithi183 ceph-mon[35818]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-26T08:31:22.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:22 smithi052 ceph-mon[35071]: pgmap v365: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:22.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:22 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/922516349' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-26T08:31:22.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:22 smithi052 ceph-mon[35071]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T08:31:22.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:22 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/922516349' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-26T08:31:22.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:22 smithi052 ceph-mon[35071]: osdmap e84: 12 total, 12 up, 12 in 2024-03-26T08:31:22.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:22 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3750791709' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-26T08:31:22.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:22 smithi052 ceph-mon[35071]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-26T08:31:22.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:22.902+0000 7f6d2fe00640 1 -- 172.21.15.43:0/3127227346 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f6d3002d070 con 0x7f6d300ffe20 2024-03-26T08:31:23.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.376+0000 7f6d27400640 1 -- 172.21.15.43:0/3127227346 <== mon.2 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v86) v1 ==== 157+0+0 (secure 0 0 0) 0x7f6d2809c2c0 con 0x7f6d300ffe20 2024-03-26T08:31:23.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.380+0000 7f6d2fe00640 1 -- 172.21.15.43:0/3127227346 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6d140a2800 msgr2=0x7f6d140a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:23.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.380+0000 7f6d2fe00640 1 --2- 172.21.15.43:0/3127227346 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6d140a2800 0x7f6d140a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6d3019ae90 tx=0x7f6d18005f90 comp rx=0 tx=0).stop 2024-03-26T08:31:23.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.380+0000 7f6d2fe00640 1 -- 172.21.15.43:0/3127227346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d300ffe20 msgr2=0x7f6d30199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:23.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.380+0000 7f6d2fe00640 1 --2- 172.21.15.43:0/3127227346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d300ffe20 0x7f6d30199980 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6d28009fd0 tx=0x7f6d2800c880 comp rx=0 tx=0).stop 2024-03-26T08:31:23.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.380+0000 7f6d2fe00640 1 -- 172.21.15.43:0/3127227346 shutdown_connections 2024-03-26T08:31:23.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.380+0000 7f6d2fe00640 1 --2- 172.21.15.43:0/3127227346 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6d140a2800 0x7f6d140a4cc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:23.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.380+0000 7f6d2fe00640 1 --2- 172.21.15.43:0/3127227346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d30101120 0x7f6d301a0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:23.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.380+0000 7f6d2fe00640 1 --2- 172.21.15.43:0/3127227346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d30100780 0x7f6d30199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:23.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.380+0000 7f6d2fe00640 1 --2- 172.21.15.43:0/3127227346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d300ffe20 0x7f6d30199980 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:23.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.380+0000 7f6d2fe00640 1 -- 172.21.15.43:0/3127227346 >> 172.21.15.43:0/3127227346 conn(0x7f6d300fba20 msgr2=0x7f6d300fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:23.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.380+0000 7f6d2fe00640 1 -- 172.21.15.43:0/3127227346 shutdown_connections 2024-03-26T08:31:23.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.380+0000 7f6d2fe00640 1 -- 172.21.15.43:0/3127227346 wait complete. 2024-03-26T08:31:23.398 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-26T08:31:23.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:23 smithi052 ceph-mon[35071]: pgmap v367: 97 pgs: 3 creating+peering, 1 active+clean, 93 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:23.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:23 smithi052 ceph-mon[35071]: osdmap e85: 12 total, 12 up, 12 in 2024-03-26T08:31:23.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:23 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3750791709' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-26T08:31:23.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:23 smithi052 ceph-mon[35071]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-26T08:31:23.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:23 smithi052 ceph-mon[35071]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-26T08:31:23.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:23 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:23.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:23 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:23.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:23 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:23.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:23 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:23.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:23 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:23.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:23 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:23.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:23 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:31:23.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:23 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:31:23.574 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:23 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:23.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:23 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3127227346' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T08:31:23.575 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:23 smithi052 ceph-mon[35071]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T08:31:23.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.639+0000 7f87f1600640 1 -- 172.21.15.43:0/1663224018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87ec101f20 msgr2=0x7f87ec104310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:23.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.639+0000 7f87f1600640 1 --2- 172.21.15.43:0/1663224018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87ec101f20 0x7f87ec104310 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f87dc009ca0 tx=0x7f87dc02c0f0 comp rx=0 tx=0).stop 2024-03-26T08:31:23.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.640+0000 7f87f1600640 1 -- 172.21.15.43:0/1663224018 shutdown_connections 2024-03-26T08:31:23.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.640+0000 7f87f1600640 1 --2- 172.21.15.43:0/1663224018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87ec107180 0x7f87ec1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:23.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.640+0000 7f87f1600640 1 --2- 172.21.15.43:0/1663224018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87ec104850 0x7f87ec106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:23.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.640+0000 7f87f1600640 1 --2- 172.21.15.43:0/1663224018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87ec101f20 0x7f87ec104310 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:23.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.640+0000 7f87f1600640 1 -- 172.21.15.43:0/1663224018 >> 172.21.15.43:0/1663224018 conn(0x7f87ec0fb8d0 msgr2=0x7f87ec0fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:23.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.640+0000 7f87f1600640 1 -- 172.21.15.43:0/1663224018 shutdown_connections 2024-03-26T08:31:23.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.640+0000 7f87f1600640 1 -- 172.21.15.43:0/1663224018 wait complete. 2024-03-26T08:31:23.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.641+0000 7f87f1600640 1 Processor -- start 2024-03-26T08:31:23.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.641+0000 7f87f1600640 1 -- start start 2024-03-26T08:31:23.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.642+0000 7f87f1600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87ec101f20 0x7f87ec199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:23.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.642+0000 7f87f1600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87ec104850 0x7f87ec199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:23.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.642+0000 7f87f1600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87ec107180 0x7f87ec1a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:23.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.642+0000 7f87f1600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec10cfe0 con 0x7f87ec101f20 2024-03-26T08:31:23.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.642+0000 7f87f1600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec10ce60 con 0x7f87ec107180 2024-03-26T08:31:23.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.642+0000 7f87f1600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec10d160 con 0x7f87ec104850 2024-03-26T08:31:23.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.642+0000 7f87ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87ec101f20 0x7f87ec199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:23.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.642+0000 7f87ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87ec101f20 0x7f87ec199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44100/0 (socket says 172.21.15.43:44100) 2024-03-26T08:31:23.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.642+0000 7f87ebe00640 1 -- 172.21.15.43:0/2583336604 learned_addr learned my addr 172.21.15.43:0/2583336604 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:23.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.642+0000 7f87eb400640 1 --2- 172.21.15.43:0/2583336604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87ec104850 0x7f87ec199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:23.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.642+0000 7f87f0c00640 1 --2- 172.21.15.43:0/2583336604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87ec107180 0x7f87ec1a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:23.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.642+0000 7f87ebe00640 1 -- 172.21.15.43:0/2583336604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87ec104850 msgr2=0x7f87ec199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:23.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.642+0000 7f87ebe00640 1 --2- 172.21.15.43:0/2583336604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87ec104850 0x7f87ec199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:23.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.642+0000 7f87ebe00640 1 -- 172.21.15.43:0/2583336604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87ec107180 msgr2=0x7f87ec1a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:23.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.642+0000 7f87ebe00640 1 --2- 172.21.15.43:0/2583336604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87ec107180 0x7f87ec1a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:23.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.642+0000 7f87ebe00640 1 -- 172.21.15.43:0/2583336604 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ec1a1550 con 0x7f87ec101f20 2024-03-26T08:31:23.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.642+0000 7f87f0c00640 1 --2- 172.21.15.43:0/2583336604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87ec107180 0x7f87ec1a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:31:23.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.643+0000 7f87ebe00640 1 --2- 172.21.15.43:0/2583336604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87ec101f20 0x7f87ec199930 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f87dc000f80 tx=0x7f87dc0328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:23.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.643+0000 7f87e8c00640 1 -- 172.21.15.43:0/2583336604 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc041020 con 0x7f87ec101f20 2024-03-26T08:31:23.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.643+0000 7f87e8c00640 1 -- 172.21.15.43:0/2583336604 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f87dc003e20 con 0x7f87ec101f20 2024-03-26T08:31:23.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.643+0000 7f87e8c00640 1 -- 172.21.15.43:0/2583336604 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc039040 con 0x7f87ec101f20 2024-03-26T08:31:23.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.643+0000 7f87f1600640 1 -- 172.21.15.43:0/2583336604 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ec1a17e0 con 0x7f87ec101f20 2024-03-26T08:31:23.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.643+0000 7f87f1600640 1 -- 172.21.15.43:0/2583336604 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ec1a1c80 con 0x7f87ec101f20 2024-03-26T08:31:23.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.645+0000 7f87f1600640 1 -- 172.21.15.43:0/2583336604 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87ec108cd0 con 0x7f87ec101f20 2024-03-26T08:31:23.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.647+0000 7f87e8c00640 1 -- 172.21.15.43:0/2583336604 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f87dc044020 con 0x7f87ec101f20 2024-03-26T08:31:23.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.648+0000 7f87e8c00640 1 --2- 172.21.15.43:0/2583336604 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f87d00a2800 0x7f87d00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:23.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.648+0000 7f87e8c00640 1 -- 172.21.15.43:0/2583336604 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8277+0+0 (secure 0 0 0) 0x7f87dc0f0d70 con 0x7f87ec101f20 2024-03-26T08:31:23.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.648+0000 7f87eb400640 1 --2- 172.21.15.43:0/2583336604 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f87d00a2800 0x7f87d00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:23.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.649+0000 7f87eb400640 1 --2- 172.21.15.43:0/2583336604 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f87d00a2800 0x7f87d00a4cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f87ec19ae40 tx=0x7f87d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:23.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.653+0000 7f87e8c00640 1 -- 172.21.15.43:0/2583336604 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f87dc04b020 con 0x7f87ec101f20 2024-03-26T08:31:23.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:23 smithi043 ceph-mon[31929]: pgmap v367: 97 pgs: 3 creating+peering, 1 active+clean, 93 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:23.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:23 smithi043 ceph-mon[31929]: osdmap e85: 12 total, 12 up, 12 in 2024-03-26T08:31:23.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:23 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3750791709' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-26T08:31:23.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:23 smithi043 ceph-mon[31929]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-26T08:31:23.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:23 smithi043 ceph-mon[31929]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-26T08:31:23.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:23 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:23.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:23 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:23.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:23 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:23.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:23 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:23.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:23 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:23.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:23 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:23.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:23 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:31:23.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:23 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:31:23.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:23 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:23.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:23 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3127227346' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T08:31:23.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:23 smithi043 ceph-mon[31929]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T08:31:23.701 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:23 smithi183 ceph-mon[35818]: pgmap v367: 97 pgs: 3 creating+peering, 1 active+clean, 93 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:23.701 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:23 smithi183 ceph-mon[35818]: osdmap e85: 12 total, 12 up, 12 in 2024-03-26T08:31:23.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:23 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3750791709' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-26T08:31:23.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:23 smithi183 ceph-mon[35818]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-26T08:31:23.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:23 smithi183 ceph-mon[35818]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-26T08:31:23.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:23 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:23.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:23 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:23.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:23 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:23.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:23 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:23.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:23 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:23.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:23 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:23.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:23 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:31:23.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:23 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:31:23.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:23 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:31:23.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:23 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3127227346' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T08:31:23.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:23 smithi183 ceph-mon[35818]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T08:31:23.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:23.859+0000 7f87f1600640 1 -- 172.21.15.43:0/2583336604 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f87ec02d070 con 0x7f87ec101f20 2024-03-26T08:31:24.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:24 smithi043 ceph-mon[31929]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-26T08:31:24.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:24 smithi043 ceph-mon[31929]: osdmap e86: 12 total, 12 up, 12 in 2024-03-26T08:31:24.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:24 smithi043 ceph-mon[31929]: mds.? [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] up:active 2024-03-26T08:31:24.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:24 smithi043 ceph-mon[31929]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T08:31:24.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:24 smithi043 ceph-mon[31929]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T08:31:24.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:24 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2583336604' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-26T08:31:24.730 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:24 smithi183 ceph-mon[35818]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-26T08:31:24.730 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:24 smithi183 ceph-mon[35818]: osdmap e86: 12 total, 12 up, 12 in 2024-03-26T08:31:24.730 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:24 smithi183 ceph-mon[35818]: mds.? [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] up:active 2024-03-26T08:31:24.731 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:24 smithi183 ceph-mon[35818]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T08:31:24.731 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:24 smithi183 ceph-mon[35818]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T08:31:24.731 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:24 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2583336604' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-26T08:31:24.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:24 smithi052 ceph-mon[35071]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-26T08:31:24.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:24 smithi052 ceph-mon[35071]: osdmap e86: 12 total, 12 up, 12 in 2024-03-26T08:31:24.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:24 smithi052 ceph-mon[35071]: mds.? [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] up:active 2024-03-26T08:31:24.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:24 smithi052 ceph-mon[35071]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T08:31:24.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:24 smithi052 ceph-mon[35071]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T08:31:24.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:24 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2583336604' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-26T08:31:25.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.398+0000 7f87e8c00640 1 -- 172.21.15.43:0/2583336604 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v88) v1 ==== 228+0+0 (secure 0 0 0) 0x7f87dc0bdde0 con 0x7f87ec101f20 2024-03-26T08:31:25.391 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data_ec' created 2024-03-26T08:31:25.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.402+0000 7f87f1600640 1 -- 172.21.15.43:0/2583336604 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f87d00a2800 msgr2=0x7f87d00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:25.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.402+0000 7f87f1600640 1 --2- 172.21.15.43:0/2583336604 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f87d00a2800 0x7f87d00a4cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f87ec19ae40 tx=0x7f87d4005f90 comp rx=0 tx=0).stop 2024-03-26T08:31:25.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.402+0000 7f87f1600640 1 -- 172.21.15.43:0/2583336604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87ec101f20 msgr2=0x7f87ec199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:25.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.402+0000 7f87f1600640 1 --2- 172.21.15.43:0/2583336604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87ec101f20 0x7f87ec199930 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f87dc000f80 tx=0x7f87dc0328a0 comp rx=0 tx=0).stop 2024-03-26T08:31:25.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.402+0000 7f87f1600640 1 -- 172.21.15.43:0/2583336604 shutdown_connections 2024-03-26T08:31:25.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.402+0000 7f87f1600640 1 --2- 172.21.15.43:0/2583336604 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f87d00a2800 0x7f87d00a4cc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:25.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.402+0000 7f87f1600640 1 --2- 172.21.15.43:0/2583336604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87ec107180 0x7f87ec1a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:25.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.402+0000 7f87f1600640 1 --2- 172.21.15.43:0/2583336604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87ec104850 0x7f87ec199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:25.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.402+0000 7f87f1600640 1 --2- 172.21.15.43:0/2583336604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87ec101f20 0x7f87ec199930 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:25.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.402+0000 7f87f1600640 1 -- 172.21.15.43:0/2583336604 >> 172.21.15.43:0/2583336604 conn(0x7f87ec0fb8d0 msgr2=0x7f87ec1017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:25.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.402+0000 7f87f1600640 1 -- 172.21.15.43:0/2583336604 shutdown_connections 2024-03-26T08:31:25.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.403+0000 7f87f1600640 1 -- 172.21.15.43:0/2583336604 wait complete. 2024-03-26T08:31:25.424 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-26T08:31:25.572 INFO:journalctl@ceph.mds.b.smithi052.stdout:Mar 26 08:31:25 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-b[65603]: 2024-03-26T08:31:25.163+0000 7f6eb8200700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-26T08:31:25.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:25 smithi052 ceph-mon[35071]: pgmap v370: 97 pgs: 3 creating+peering, 46 active+clean, 48 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 6 op/s 2024-03-26T08:31:25.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:25 smithi052 ceph-mon[35071]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-26T08:31:25.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:25 smithi052 ceph-mon[35071]: Cluster is now healthy 2024-03-26T08:31:25.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:25 smithi052 ceph-mon[35071]: osdmap e87: 12 total, 12 up, 12 in 2024-03-26T08:31:25.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.627+0000 7fadfc400640 1 -- 172.21.15.43:0/2867054127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf4103260 msgr2=0x7fadf4109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:25.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.627+0000 7fadfc400640 1 --2- 172.21.15.43:0/2867054127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf4103260 0x7fadf4109af0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fadec009900 tx=0x7fadec02bc30 comp rx=0 tx=0).stop 2024-03-26T08:31:25.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.627+0000 7fadfc400640 1 -- 172.21.15.43:0/2867054127 shutdown_connections 2024-03-26T08:31:25.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.627+0000 7fadfc400640 1 --2- 172.21.15.43:0/2867054127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf4103260 0x7fadf4109af0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:25.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.627+0000 7fadfc400640 1 --2- 172.21.15.43:0/2867054127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadf41028a0 0x7fadf4102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:25.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.627+0000 7fadfc400640 1 --2- 172.21.15.43:0/2867054127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadf41016b0 0x7fadf4101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:25.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.627+0000 7fadfc400640 1 -- 172.21.15.43:0/2867054127 >> 172.21.15.43:0/2867054127 conn(0x7fadf40fcc40 msgr2=0x7fadf40ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:25.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.627+0000 7fadfc400640 1 -- 172.21.15.43:0/2867054127 shutdown_connections 2024-03-26T08:31:25.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.628+0000 7fadfc400640 1 -- 172.21.15.43:0/2867054127 wait complete. 2024-03-26T08:31:25.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.628+0000 7fadfc400640 1 Processor -- start 2024-03-26T08:31:25.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.629+0000 7fadfc400640 1 -- start start 2024-03-26T08:31:25.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.629+0000 7fadfc400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadf41016b0 0x7fadf4199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:25.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.629+0000 7fadfc400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadf41028a0 0x7fadf4199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:25.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.629+0000 7fadfc400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf4103260 0x7fadf41a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:25.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.629+0000 7fadfc400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf410d230 con 0x7fadf4103260 2024-03-26T08:31:25.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.629+0000 7fadfc400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf410d0b0 con 0x7fadf41028a0 2024-03-26T08:31:25.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.629+0000 7fadfc400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf410d3b0 con 0x7fadf41016b0 2024-03-26T08:31:25.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.629+0000 7fadfa400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf4103260 0x7fadf41a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:25.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.629+0000 7fadfa400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf4103260 0x7fadf41a1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53572/0 (socket says 172.21.15.43:53572) 2024-03-26T08:31:25.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.629+0000 7fadfa400640 1 -- 172.21.15.43:0/491071033 learned_addr learned my addr 172.21.15.43:0/491071033 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:25.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.629+0000 7fadf9a00640 1 --2- 172.21.15.43:0/491071033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadf41016b0 0x7fadf4199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:25.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.629+0000 7fadf9000640 1 --2- 172.21.15.43:0/491071033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadf41028a0 0x7fadf4199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:25.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.630+0000 7fadfa400640 1 -- 172.21.15.43:0/491071033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadf41016b0 msgr2=0x7fadf4199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:25.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.630+0000 7fadfa400640 1 --2- 172.21.15.43:0/491071033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadf41016b0 0x7fadf4199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:25.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.630+0000 7fadfa400640 1 -- 172.21.15.43:0/491071033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadf41028a0 msgr2=0x7fadf4199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:25.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.630+0000 7fadfa400640 1 --2- 172.21.15.43:0/491071033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadf41028a0 0x7fadf4199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:25.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.630+0000 7fadfa400640 1 -- 172.21.15.43:0/491071033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf41a16a0 con 0x7fadf4103260 2024-03-26T08:31:25.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.630+0000 7fadfa400640 1 --2- 172.21.15.43:0/491071033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf4103260 0x7fadf41a1030 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fadec036040 tx=0x7fadec005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:25.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.630+0000 7fadf2000640 1 -- 172.21.15.43:0/491071033 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec043020 con 0x7fadf4103260 2024-03-26T08:31:25.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.630+0000 7fadf2000640 1 -- 172.21.15.43:0/491071033 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fadec004410 con 0x7fadf4103260 2024-03-26T08:31:25.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.630+0000 7fadf2000640 1 -- 172.21.15.43:0/491071033 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec003b80 con 0x7fadf4103260 2024-03-26T08:31:25.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.630+0000 7fadfc400640 1 -- 172.21.15.43:0/491071033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf41a1930 con 0x7fadf4103260 2024-03-26T08:31:25.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.630+0000 7fadfc400640 1 -- 172.21.15.43:0/491071033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf41a1e80 con 0x7fadf4103260 2024-03-26T08:31:25.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.631+0000 7fadfc400640 1 -- 172.21.15.43:0/491071033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf41a2170 con 0x7fadf4103260 2024-03-26T08:31:25.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.634+0000 7fadf2000640 1 -- 172.21.15.43:0/491071033 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fadec046020 con 0x7fadf4103260 2024-03-26T08:31:25.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.635+0000 7fadf2000640 1 --2- 172.21.15.43:0/491071033 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fadd80a2800 0x7fadd80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:25.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.635+0000 7fadf2000640 1 -- 172.21.15.43:0/491071033 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8770+0+0 (secure 0 0 0) 0x7fadec0f1430 con 0x7fadf4103260 2024-03-26T08:31:25.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.635+0000 7fadf9a00640 1 --2- 172.21.15.43:0/491071033 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fadd80a2800 0x7fadd80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:25.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.636+0000 7fadf9a00640 1 --2- 172.21.15.43:0/491071033 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fadd80a2800 0x7fadd80a4cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fade40095f0 tx=0x7fade4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:25.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.638+0000 7fadf2000640 1 -- 172.21.15.43:0/491071033 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fadec04d020 con 0x7fadf4103260 2024-03-26T08:31:25.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:25 smithi043 ceph-mon[31929]: pgmap v370: 97 pgs: 3 creating+peering, 46 active+clean, 48 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 6 op/s 2024-03-26T08:31:25.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:25 smithi043 ceph-mon[31929]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-26T08:31:25.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:25 smithi043 ceph-mon[31929]: Cluster is now healthy 2024-03-26T08:31:25.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:25 smithi043 ceph-mon[31929]: osdmap e87: 12 total, 12 up, 12 in 2024-03-26T08:31:25.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:25 smithi183 ceph-mon[35818]: pgmap v370: 97 pgs: 3 creating+peering, 46 active+clean, 48 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 6 op/s 2024-03-26T08:31:25.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:25 smithi183 ceph-mon[35818]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-26T08:31:25.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:25 smithi183 ceph-mon[35818]: Cluster is now healthy 2024-03-26T08:31:25.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:25 smithi183 ceph-mon[35818]: osdmap e87: 12 total, 12 up, 12 in 2024-03-26T08:31:25.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:25.842+0000 7fadfc400640 1 -- 172.21.15.43:0/491071033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fadf41a2340 con 0x7fadf4103260 2024-03-26T08:31:26.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.412+0000 7fadf2000640 1 -- 172.21.15.43:0/491071033 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v89) v1 ==== 169+0+0 (secure 0 0 0) 0x7fadec0be250 con 0x7fadf4103260 2024-03-26T08:31:26.406 INFO:teuthology.orchestra.run.smithi043.stderr:set pool 4 allow_ec_overwrites to true 2024-03-26T08:31:26.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.418+0000 7fadfc400640 1 -- 172.21.15.43:0/491071033 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fadd80a2800 msgr2=0x7fadd80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:26.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.418+0000 7fadfc400640 1 --2- 172.21.15.43:0/491071033 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fadd80a2800 0x7fadd80a4cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fade40095f0 tx=0x7fade4005f90 comp rx=0 tx=0).stop 2024-03-26T08:31:26.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.418+0000 7fadfc400640 1 -- 172.21.15.43:0/491071033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf4103260 msgr2=0x7fadf41a1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:26.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.418+0000 7fadfc400640 1 --2- 172.21.15.43:0/491071033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf4103260 0x7fadf41a1030 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fadec036040 tx=0x7fadec005dd0 comp rx=0 tx=0).stop 2024-03-26T08:31:26.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.418+0000 7fadfc400640 1 -- 172.21.15.43:0/491071033 shutdown_connections 2024-03-26T08:31:26.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.418+0000 7fadfc400640 1 --2- 172.21.15.43:0/491071033 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fadd80a2800 0x7fadd80a4cc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:26.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.418+0000 7fadfc400640 1 --2- 172.21.15.43:0/491071033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf4103260 0x7fadf41a1030 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:26.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.418+0000 7fadfc400640 1 --2- 172.21.15.43:0/491071033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadf41028a0 0x7fadf4199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:26.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.418+0000 7fadfc400640 1 --2- 172.21.15.43:0/491071033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadf41016b0 0x7fadf4199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:26.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.418+0000 7fadfc400640 1 -- 172.21.15.43:0/491071033 >> 172.21.15.43:0/491071033 conn(0x7fadf40fcc40 msgr2=0x7fadf40738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:26.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.419+0000 7fadfc400640 1 -- 172.21.15.43:0/491071033 shutdown_connections 2024-03-26T08:31:26.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.419+0000 7fadfc400640 1 -- 172.21.15.43:0/491071033 wait complete. 2024-03-26T08:31:26.446 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-26T08:31:26.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:26 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2583336604' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-26T08:31:26.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:26 smithi183 ceph-mon[35818]: osdmap e88: 12 total, 12 up, 12 in 2024-03-26T08:31:26.549 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:26 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/491071033' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-26T08:31:26.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.644+0000 7fa8f1c00640 1 -- 172.21.15.43:0/2464931102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8ec1038e0 msgr2=0x7fa8ec103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:26.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.644+0000 7fa8f1c00640 1 --2- 172.21.15.43:0/2464931102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8ec1038e0 0x7fa8ec103d00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc009900 tx=0x7fa8dc02bc30 comp rx=0 tx=0).stop 2024-03-26T08:31:26.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.644+0000 7fa8f1c00640 1 -- 172.21.15.43:0/2464931102 shutdown_connections 2024-03-26T08:31:26.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.644+0000 7fa8f1c00640 1 --2- 172.21.15.43:0/2464931102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8ec105490 0x7fa8ec10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:26.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.644+0000 7fa8f1c00640 1 --2- 172.21.15.43:0/2464931102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8ec104ad0 0x7fa8ec104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:26.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.644+0000 7fa8f1c00640 1 --2- 172.21.15.43:0/2464931102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8ec1038e0 0x7fa8ec103d00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:26.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.644+0000 7fa8f1c00640 1 -- 172.21.15.43:0/2464931102 >> 172.21.15.43:0/2464931102 conn(0x7fa8ec0fee50 msgr2=0x7fa8ec101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:26.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.644+0000 7fa8f1c00640 1 -- 172.21.15.43:0/2464931102 shutdown_connections 2024-03-26T08:31:26.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.644+0000 7fa8f1c00640 1 -- 172.21.15.43:0/2464931102 wait complete. 2024-03-26T08:31:26.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.645+0000 7fa8f1c00640 1 Processor -- start 2024-03-26T08:31:26.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.645+0000 7fa8f1c00640 1 -- start start 2024-03-26T08:31:26.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.645+0000 7fa8f1c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8ec1038e0 0x7fa8ec199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:26.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.645+0000 7fa8f1c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8ec104ad0 0x7fa8ec19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:26.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.645+0000 7fa8f1c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8ec105490 0x7fa8ec1a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:26.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.645+0000 7fa8f1c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ec10f460 con 0x7fa8ec105490 2024-03-26T08:31:26.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.645+0000 7fa8f1c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ec10f2e0 con 0x7fa8ec1038e0 2024-03-26T08:31:26.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.645+0000 7fa8f1c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ec10f5e0 con 0x7fa8ec104ad0 2024-03-26T08:31:26.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.646+0000 7fa8eb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8ec105490 0x7fa8ec1a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:26.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.646+0000 7fa8eb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8ec105490 0x7fa8ec1a10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53580/0 (socket says 172.21.15.43:53580) 2024-03-26T08:31:26.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.646+0000 7fa8eb400640 1 -- 172.21.15.43:0/2632881647 learned_addr learned my addr 172.21.15.43:0/2632881647 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:26.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.646+0000 7fa8eaa00640 1 --2- 172.21.15.43:0/2632881647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8ec1038e0 0x7fa8ec199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:26.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.646+0000 7fa8ea000640 1 --2- 172.21.15.43:0/2632881647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8ec104ad0 0x7fa8ec19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:26.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.646+0000 7fa8eb400640 1 -- 172.21.15.43:0/2632881647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8ec104ad0 msgr2=0x7fa8ec19a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:26.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.646+0000 7fa8eb400640 1 --2- 172.21.15.43:0/2632881647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8ec104ad0 0x7fa8ec19a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:26.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.646+0000 7fa8eb400640 1 -- 172.21.15.43:0/2632881647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8ec1038e0 msgr2=0x7fa8ec199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:26.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.646+0000 7fa8eb400640 1 --2- 172.21.15.43:0/2632881647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8ec1038e0 0x7fa8ec199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:26.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.646+0000 7fa8eb400640 1 -- 172.21.15.43:0/2632881647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8ec1a1750 con 0x7fa8ec105490 2024-03-26T08:31:26.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.646+0000 7fa8eaa00640 1 --2- 172.21.15.43:0/2632881647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8ec1038e0 0x7fa8ec199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:31:26.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.647+0000 7fa8eb400640 1 --2- 172.21.15.43:0/2632881647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8ec105490 0x7fa8ec1a10e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa8e400eaa0 tx=0x7fa8e400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:26.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.647+0000 7fa8e3400640 1 -- 172.21.15.43:0/2632881647 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e400cdb0 con 0x7fa8ec105490 2024-03-26T08:31:26.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.647+0000 7fa8e3400640 1 -- 172.21.15.43:0/2632881647 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa8e4005020 con 0x7fa8ec105490 2024-03-26T08:31:26.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.647+0000 7fa8e3400640 1 -- 172.21.15.43:0/2632881647 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e40116d0 con 0x7fa8ec105490 2024-03-26T08:31:26.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.647+0000 7fa8f1c00640 1 -- 172.21.15.43:0/2632881647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8ec1a1a40 con 0x7fa8ec105490 2024-03-26T08:31:26.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.647+0000 7fa8f1c00640 1 -- 172.21.15.43:0/2632881647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8ec1a1f40 con 0x7fa8ec105490 2024-03-26T08:31:26.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.649+0000 7fa8f1c00640 1 -- 172.21.15.43:0/2632881647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8ec10ac50 con 0x7fa8ec105490 2024-03-26T08:31:26.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.651+0000 7fa8e3400640 1 -- 172.21.15.43:0/2632881647 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa8e40051d0 con 0x7fa8ec105490 2024-03-26T08:31:26.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.652+0000 7fa8e3400640 1 --2- 172.21.15.43:0/2632881647 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa8d00a2800 0x7fa8d00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:26.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.652+0000 7fa8e3400640 1 -- 172.21.15.43:0/2632881647 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8770+0+0 (secure 0 0 0) 0x7fa8e40cef10 con 0x7fa8ec105490 2024-03-26T08:31:26.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.652+0000 7fa8eaa00640 1 --2- 172.21.15.43:0/2632881647 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa8d00a2800 0x7fa8d00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:26.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.653+0000 7fa8eaa00640 1 --2- 172.21.15.43:0/2632881647 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa8d00a2800 0x7fa8d00a4cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc02c240 tx=0x7fa8dc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:26.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.657+0000 7fa8e3400640 1 -- 172.21.15.43:0/2632881647 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa8e4016030 con 0x7fa8ec105490 2024-03-26T08:31:26.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:26 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2583336604' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-26T08:31:26.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:26 smithi043 ceph-mon[31929]: osdmap e88: 12 total, 12 up, 12 in 2024-03-26T08:31:26.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:26 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/491071033' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-26T08:31:26.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:26 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2583336604' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-26T08:31:26.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:26 smithi052 ceph-mon[35071]: osdmap e88: 12 total, 12 up, 12 in 2024-03-26T08:31:26.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:26 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/491071033' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-26T08:31:26.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:26.905+0000 7fa8f1c00640 1 -- 172.21.15.43:0/2632881647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fa8ec0622d0 con 0x7fa8ec105490 2024-03-26T08:31:27.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.419+0000 7fa8e3400640 1 -- 172.21.15.43:0/2632881647 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7fa8e409bd30 con 0x7fa8ec105490 2024-03-26T08:31:27.413 INFO:teuthology.orchestra.run.smithi043.stderr:added data pool 4 to fsmap 2024-03-26T08:31:27.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.423+0000 7fa8f1c00640 1 -- 172.21.15.43:0/2632881647 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa8d00a2800 msgr2=0x7fa8d00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:27.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.423+0000 7fa8f1c00640 1 --2- 172.21.15.43:0/2632881647 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa8d00a2800 0x7fa8d00a4cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc02c240 tx=0x7fa8dc005fd0 comp rx=0 tx=0).stop 2024-03-26T08:31:27.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.423+0000 7fa8f1c00640 1 -- 172.21.15.43:0/2632881647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8ec105490 msgr2=0x7fa8ec1a10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:27.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.423+0000 7fa8f1c00640 1 --2- 172.21.15.43:0/2632881647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8ec105490 0x7fa8ec1a10e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa8e400eaa0 tx=0x7fa8e400ef60 comp rx=0 tx=0).stop 2024-03-26T08:31:27.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.423+0000 7fa8f1c00640 1 -- 172.21.15.43:0/2632881647 shutdown_connections 2024-03-26T08:31:27.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.423+0000 7fa8f1c00640 1 --2- 172.21.15.43:0/2632881647 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa8d00a2800 0x7fa8d00a4cc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.423+0000 7fa8f1c00640 1 --2- 172.21.15.43:0/2632881647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8ec105490 0x7fa8ec1a10e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.423+0000 7fa8f1c00640 1 --2- 172.21.15.43:0/2632881647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8ec104ad0 0x7fa8ec19a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.423+0000 7fa8f1c00640 1 --2- 172.21.15.43:0/2632881647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8ec1038e0 0x7fa8ec199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.423+0000 7fa8f1c00640 1 -- 172.21.15.43:0/2632881647 >> 172.21.15.43:0/2632881647 conn(0x7fa8ec0fee50 msgr2=0x7fa8ec107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.424+0000 7fa8f1c00640 1 -- 172.21.15.43:0/2632881647 shutdown_connections 2024-03-26T08:31:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.424+0000 7fa8f1c00640 1 -- 172.21.15.43:0/2632881647 wait complete. 2024-03-26T08:31:27.437 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:31:27.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:27 smithi043 ceph-mon[31929]: pgmap v373: 129 pgs: 10 creating+peering, 97 active+clean, 22 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 11 op/s 2024-03-26T08:31:27.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:27 smithi043 ceph-mon[31929]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T08:31:27.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:27 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/491071033' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-26T08:31:27.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:27 smithi043 ceph-mon[31929]: osdmap e89: 12 total, 12 up, 12 in 2024-03-26T08:31:27.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:27 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2632881647' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-26T08:31:27.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.663+0000 7f190f200640 1 -- 172.21.15.43:0/996513254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1910100740 msgr2=0x7f1910100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:27.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.663+0000 7f190f200640 1 --2- 172.21.15.43:0/996513254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1910100740 0x7f1910100ba0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f18f8009900 tx=0x7f18f802bcd0 comp rx=0 tx=0).stop 2024-03-26T08:31:27.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.663+0000 7f190f200640 1 -- 172.21.15.43:0/996513254 shutdown_connections 2024-03-26T08:31:27.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.663+0000 7f190f200640 1 --2- 172.21.15.43:0/996513254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19101010e0 0x7f1910101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:27.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.663+0000 7f190f200640 1 --2- 172.21.15.43:0/996513254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1910100740 0x7f1910100ba0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:27.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.663+0000 7f190f200640 1 --2- 172.21.15.43:0/996513254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19100ffde0 0x7f1910100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:27.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.663+0000 7f190f200640 1 -- 172.21.15.43:0/996513254 >> 172.21.15.43:0/996513254 conn(0x7f19100fb980 msgr2=0x7f19100fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:27.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.663+0000 7f190f200640 1 -- 172.21.15.43:0/996513254 shutdown_connections 2024-03-26T08:31:27.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.663+0000 7f190f200640 1 -- 172.21.15.43:0/996513254 wait complete. 2024-03-26T08:31:27.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.664+0000 7f190f200640 1 Processor -- start 2024-03-26T08:31:27.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.664+0000 7f190f200640 1 -- start start 2024-03-26T08:31:27.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.665+0000 7f190f200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19100ffde0 0x7f19101998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:27.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.665+0000 7f190f200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1910100740 0x7f1910199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:27.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.665+0000 7f190f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19101010e0 0x7f19101a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:27.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.665+0000 7f190f200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191010f250 con 0x7f19101010e0 2024-03-26T08:31:27.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.665+0000 7f190f200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191010f0d0 con 0x7f1910100740 2024-03-26T08:31:27.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.665+0000 7f190f200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191010f3d0 con 0x7f19100ffde0 2024-03-26T08:31:27.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.665+0000 7f190e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19101010e0 0x7f19101a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:27.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.665+0000 7f190e800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19101010e0 0x7f19101a0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53588/0 (socket says 172.21.15.43:53588) 2024-03-26T08:31:27.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.665+0000 7f190e800640 1 -- 172.21.15.43:0/152586734 learned_addr learned my addr 172.21.15.43:0/152586734 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:27.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.665+0000 7f1905400640 1 --2- 172.21.15.43:0/152586734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1910100740 0x7f1910199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:27.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.665+0000 7f190de00640 1 --2- 172.21.15.43:0/152586734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19100ffde0 0x7f19101998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:27.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.665+0000 7f190e800640 1 -- 172.21.15.43:0/152586734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19100ffde0 msgr2=0x7f19101998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:27.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.665+0000 7f190e800640 1 --2- 172.21.15.43:0/152586734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19100ffde0 0x7f19101998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:27.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.665+0000 7f190e800640 1 -- 172.21.15.43:0/152586734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1910100740 msgr2=0x7f1910199e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:27.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.665+0000 7f190e800640 1 --2- 172.21.15.43:0/152586734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1910100740 0x7f1910199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:27.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.665+0000 7f190e800640 1 -- 172.21.15.43:0/152586734 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19101a1510 con 0x7f19101010e0 2024-03-26T08:31:27.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.665+0000 7f190de00640 1 --2- 172.21.15.43:0/152586734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19100ffde0 0x7f19101998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:31:27.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.666+0000 7f190e800640 1 --2- 172.21.15.43:0/152586734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19101010e0 0x7f19101a0ea0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f190800c970 tx=0x7f190800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:27.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.666+0000 7f1907400640 1 -- 172.21.15.43:0/152586734 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1908007bb0 con 0x7f19101010e0 2024-03-26T08:31:27.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.666+0000 7f1907400640 1 -- 172.21.15.43:0/152586734 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1908007d50 con 0x7f19101010e0 2024-03-26T08:31:27.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.666+0000 7f190f200640 1 -- 172.21.15.43:0/152586734 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19101a1800 con 0x7f19101010e0 2024-03-26T08:31:27.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.666+0000 7f1907400640 1 -- 172.21.15.43:0/152586734 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1908018700 con 0x7f19101010e0 2024-03-26T08:31:27.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.666+0000 7f190f200640 1 -- 172.21.15.43:0/152586734 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19101a1d00 con 0x7f19101010e0 2024-03-26T08:31:27.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.667+0000 7f190f200640 1 -- 172.21.15.43:0/152586734 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f191010ab30 con 0x7f19101010e0 2024-03-26T08:31:27.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.669+0000 7f1907400640 1 -- 172.21.15.43:0/152586734 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f190800fa20 con 0x7f19101010e0 2024-03-26T08:31:27.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.671+0000 7f1907400640 1 --2- 172.21.15.43:0/152586734 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f18f40a2800 0x7f18f40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:27.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.671+0000 7f1907400640 1 -- 172.21.15.43:0/152586734 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f19080ceeb0 con 0x7f19101010e0 2024-03-26T08:31:27.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.671+0000 7f190de00640 1 --2- 172.21.15.43:0/152586734 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f18f40a2800 0x7f18f40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:27.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.671+0000 7f190de00640 1 --2- 172.21.15.43:0/152586734 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f18f40a2800 0x7f18f40a4cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1900005fd0 tx=0x7f19000093e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:27.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.675+0000 7f1907400640 1 -- 172.21.15.43:0/152586734 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1908014030 con 0x7f19101010e0 2024-03-26T08:31:27.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:27 smithi183 ceph-mon[35818]: pgmap v373: 129 pgs: 10 creating+peering, 97 active+clean, 22 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 11 op/s 2024-03-26T08:31:27.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:27 smithi183 ceph-mon[35818]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T08:31:27.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:27 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/491071033' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-26T08:31:27.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:27 smithi183 ceph-mon[35818]: osdmap e89: 12 total, 12 up, 12 in 2024-03-26T08:31:27.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:27 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2632881647' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-26T08:31:27.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:27 smithi052 ceph-mon[35071]: pgmap v373: 129 pgs: 10 creating+peering, 97 active+clean, 22 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 11 op/s 2024-03-26T08:31:27.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:27 smithi052 ceph-mon[35071]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T08:31:27.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:27 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/491071033' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-26T08:31:27.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:27 smithi052 ceph-mon[35071]: osdmap e89: 12 total, 12 up, 12 in 2024-03-26T08:31:27.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:27 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2632881647' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-26T08:31:27.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.922+0000 7f190f200640 1 -- 172.21.15.43:0/152586734 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f191002d070 con 0x7f19101010e0 2024-03-26T08:31:27.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.923+0000 7f1907400640 1 -- 172.21.15.43:0/152586734 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11321 (secure 0 0 0) 0x7f190809bcb0 con 0x7f19101010e0 2024-03-26T08:31:27.916 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:31:27.916 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1452206807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1452206807},{"type":"v1","addr":"172.21.15.43:6835","nonce":1452206807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3151775467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3151775467},{"type":"v1","addr":"172.21.15.43:6837","nonce":3151775467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/1284655682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":1284655682},{"type":"v1","addr":"172.21.15.43:6839","nonce":1284655682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3081085787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3081085787},{"type":"v1","addr":"172.21.15.43:6841","nonce":3081085787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1842295306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1842295306},{"type":"v1","addr":"172.21.15.183:6835","nonce":1842295306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2267781666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2267781666},{"type":"v1","addr":"172.21.15.183:6837","nonce":2267781666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3572429948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3572429948},{"type":"v1","addr":"172.21.15.183:6839","nonce":3572429948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34448,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/1108327179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108327179},{"type":"v1","addr":"172.21.15.52:6833","nonce":1108327179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34487,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/742057410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":742057410},{"type":"v1","addr":"172.21.15.52:6835","nonce":742057410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34517,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/748239918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":748239918},{"type":"v1","addr":"172.21.15.52:6837","nonce":748239918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-26T08:31:21.819689+0000","modified":"2024-03-26T08:31:26.907931+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34559":{"gid":34559,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.52:6839/2551965101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2551965101},{"type":"v1","addr":"172.21.15.52:6839","nonce":2551965101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34559,"qdb_cluster":[34559]},"id":1}]} 2024-03-26T08:31:27.917 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 17 2024-03-26T08:31:27.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.926+0000 7f190f200640 1 -- 172.21.15.43:0/152586734 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f18f40a2800 msgr2=0x7f18f40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:27.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.926+0000 7f190f200640 1 --2- 172.21.15.43:0/152586734 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f18f40a2800 0x7f18f40a4cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1900005fd0 tx=0x7f19000093e0 comp rx=0 tx=0).stop 2024-03-26T08:31:27.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.926+0000 7f190f200640 1 -- 172.21.15.43:0/152586734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19101010e0 msgr2=0x7f19101a0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:27.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.926+0000 7f190f200640 1 --2- 172.21.15.43:0/152586734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19101010e0 0x7f19101a0ea0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f190800c970 tx=0x7f190800ce30 comp rx=0 tx=0).stop 2024-03-26T08:31:27.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.926+0000 7f190f200640 1 -- 172.21.15.43:0/152586734 shutdown_connections 2024-03-26T08:31:27.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.926+0000 7f190f200640 1 --2- 172.21.15.43:0/152586734 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f18f40a2800 0x7f18f40a4cc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:27.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.926+0000 7f190f200640 1 --2- 172.21.15.43:0/152586734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19101010e0 0x7f19101a0ea0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:27.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.926+0000 7f190f200640 1 --2- 172.21.15.43:0/152586734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1910100740 0x7f1910199e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:27.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.926+0000 7f190f200640 1 --2- 172.21.15.43:0/152586734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19100ffde0 0x7f19101998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:27.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.926+0000 7f190f200640 1 -- 172.21.15.43:0/152586734 >> 172.21.15.43:0/152586734 conn(0x7f19100fb980 msgr2=0x7f19100fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:27.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.927+0000 7f190f200640 1 -- 172.21.15.43:0/152586734 shutdown_connections 2024-03-26T08:31:27.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:27.927+0000 7f190f200640 1 -- 172.21.15.43:0/152586734 wait complete. 2024-03-26T08:31:27.939 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:31:28.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.172+0000 7f2973000640 1 -- 172.21.15.43:0/4068688853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f296c107ce0 msgr2=0x7f296c108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:28.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.172+0000 7f2973000640 1 --2- 172.21.15.43:0/4068688853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f296c107ce0 0x7f296c108100 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f295c009900 tx=0x7f295c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:31:28.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.172+0000 7f2973000640 1 -- 172.21.15.43:0/4068688853 shutdown_connections 2024-03-26T08:31:28.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.172+0000 7f2973000640 1 --2- 172.21.15.43:0/4068688853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f296c109890 0x7f296c110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:28.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.172+0000 7f2973000640 1 --2- 172.21.15.43:0/4068688853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f296c108ed0 0x7f296c109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:28.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.172+0000 7f2973000640 1 --2- 172.21.15.43:0/4068688853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f296c107ce0 0x7f296c108100 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:28.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.172+0000 7f2973000640 1 -- 172.21.15.43:0/4068688853 >> 172.21.15.43:0/4068688853 conn(0x7f296c076cb0 msgr2=0x7f296c0790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:28.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.172+0000 7f2973000640 1 -- 172.21.15.43:0/4068688853 shutdown_connections 2024-03-26T08:31:28.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.172+0000 7f2973000640 1 -- 172.21.15.43:0/4068688853 wait complete. 2024-03-26T08:31:28.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.173+0000 7f2973000640 1 Processor -- start 2024-03-26T08:31:28.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.173+0000 7f2973000640 1 -- start start 2024-03-26T08:31:28.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.173+0000 7f2973000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f296c107ce0 0x7f296c19df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:28.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.173+0000 7f2973000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f296c108ed0 0x7f296c19e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:28.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.173+0000 7f2973000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f296c109890 0x7f296c1a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:28.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.173+0000 7f2973000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c113850 con 0x7f296c108ed0 2024-03-26T08:31:28.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.173+0000 7f2973000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c1136d0 con 0x7f296c109890 2024-03-26T08:31:28.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.173+0000 7f2973000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c1139d0 con 0x7f296c107ce0 2024-03-26T08:31:28.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.174+0000 7f296b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f296c108ed0 0x7f296c19e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:28.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.174+0000 7f296b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f296c108ed0 0x7f296c19e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53608/0 (socket says 172.21.15.43:53608) 2024-03-26T08:31:28.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.174+0000 7f296b400640 1 -- 172.21.15.43:0/1417304632 learned_addr learned my addr 172.21.15.43:0/1417304632 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:28.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.174+0000 7f296be00640 1 --2- 172.21.15.43:0/1417304632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f296c107ce0 0x7f296c19df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:28.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.174+0000 7f2971000640 1 --2- 172.21.15.43:0/1417304632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f296c109890 0x7f296c1a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:28.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.174+0000 7f296b400640 1 -- 172.21.15.43:0/1417304632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f296c107ce0 msgr2=0x7f296c19df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:28.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.174+0000 7f296b400640 1 --2- 172.21.15.43:0/1417304632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f296c107ce0 0x7f296c19df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:28.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.174+0000 7f296b400640 1 -- 172.21.15.43:0/1417304632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f296c109890 msgr2=0x7f296c1a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:28.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.174+0000 7f296b400640 1 --2- 172.21.15.43:0/1417304632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f296c109890 0x7f296c1a5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:28.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.174+0000 7f296b400640 1 -- 172.21.15.43:0/1417304632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296c1a5b70 con 0x7f296c108ed0 2024-03-26T08:31:28.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.174+0000 7f296b400640 1 --2- 172.21.15.43:0/1417304632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f296c108ed0 0x7f296c19e490 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f295800d770 tx=0x7f295800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:28.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.175+0000 7f2968c00640 1 -- 172.21.15.43:0/1417304632 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958017070 con 0x7f296c108ed0 2024-03-26T08:31:28.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.175+0000 7f2968c00640 1 -- 172.21.15.43:0/1417304632 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f29580054d0 con 0x7f296c108ed0 2024-03-26T08:31:28.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.175+0000 7f2973000640 1 -- 172.21.15.43:0/1417304632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f296c1a5e60 con 0x7f296c108ed0 2024-03-26T08:31:28.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.175+0000 7f2968c00640 1 -- 172.21.15.43:0/1417304632 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958004cf0 con 0x7f296c108ed0 2024-03-26T08:31:28.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.175+0000 7f2973000640 1 -- 172.21.15.43:0/1417304632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296c1a6320 con 0x7f296c108ed0 2024-03-26T08:31:28.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.176+0000 7f2973000640 1 -- 172.21.15.43:0/1417304632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f296c10f050 con 0x7f296c108ed0 2024-03-26T08:31:28.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.179+0000 7f2968c00640 1 -- 172.21.15.43:0/1417304632 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f2958004e90 con 0x7f296c108ed0 2024-03-26T08:31:28.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.180+0000 7f2968c00640 1 --2- 172.21.15.43:0/1417304632 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f29500a2880 0x7f29500a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:28.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.181+0000 7f2968c00640 1 -- 172.21.15.43:0/1417304632 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f29580ceb00 con 0x7f296c108ed0 2024-03-26T08:31:28.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.181+0000 7f296be00640 1 --2- 172.21.15.43:0/1417304632 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f29500a2880 0x7f29500a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:28.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.182+0000 7f296be00640 1 --2- 172.21.15.43:0/1417304632 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f29500a2880 0x7f29500a4d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f295c02c240 tx=0x7f295c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:28.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.182+0000 7f2968c00640 1 -- 172.21.15.43:0/1417304632 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f295809b980 con 0x7f296c108ed0 2024-03-26T08:31:28.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.386+0000 7f2973000640 1 -- 172.21.15.43:0/1417304632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f296c02d2c0 con 0x7f296c108ed0 2024-03-26T08:31:28.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.388+0000 7f2968c00640 1 -- 172.21.15.43:0/1417304632 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24080 (secure 0 0 0) 0x7f295809b280 con 0x7f296c108ed0 2024-03-26T08:31:28.381 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:31:28.382 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":90,"fsid":"88759420-eb49-11ee-95d0-87774f69a715","created":"2024-03-26T08:20:27.817107+0000","modified":"2024-03-26T08:31:26.907701+0000","last_up_change":"2024-03-26T08:30:10.520976+0000","last_in_change":"2024-03-26T08:29:46.394360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:24:54.852427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:31:19.924902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:31:21.323881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T08:31:24.951015+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"28670673-9193-4fa3-9cc0-c1a5a6b27284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6803","nonce":936601146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6805","nonce":936601146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6809","nonce":936601146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6807","nonce":936601146}]},"public_addr":"172.21.15.43:6803/936601146","cluster_addr":"172.21.15.43:6805/936601146","heartbeat_back_addr":"172.21.15.43:6809/936601146","heartbeat_front_addr":"172.21.15.43:6807/936601146","state":["exists","up"]},{"osd":1,"uuid":"fa8ad6b6-277a-42b8-b0f8-b36fc219de2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6801","nonce":3268412328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6803","nonce":3268412328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6807","nonce":3268412328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6805","nonce":3268412328}]},"public_addr":"172.21.15.52:6801/3268412328","cluster_addr":"172.21.15.52:6803/3268412328","heartbeat_back_addr":"172.21.15.52:6807/3268412328","heartbeat_front_addr":"172.21.15.52:6805/3268412328","state":["exists","up"]},{"osd":2,"uuid":"ec0039c3-f9bd-419a-81de-fa4418359808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6801","nonce":2103180908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6803","nonce":2103180908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6807","nonce":2103180908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6805","nonce":2103180908}]},"public_addr":"172.21.15.183:6801/2103180908","cluster_addr":"172.21.15.183:6803/2103180908","heartbeat_back_addr":"172.21.15.183:6807/2103180908","heartbeat_front_addr":"172.21.15.183:6805/2103180908","state":["exists","up"]},{"osd":3,"uuid":"e7dcbf00-6586-4664-a69e-3cb0df8dfd30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6811","nonce":3722846033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6813","nonce":3722846033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6817","nonce":3722846033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6815","nonce":3722846033}]},"public_addr":"172.21.15.43:6811/3722846033","cluster_addr":"172.21.15.43:6813/3722846033","heartbeat_back_addr":"172.21.15.43:6817/3722846033","heartbeat_front_addr":"172.21.15.43:6815/3722846033","state":["exists","up"]},{"osd":4,"uuid":"db089a49-5e98-432d-9687-cdc044114c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6809","nonce":4028436523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6811","nonce":4028436523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6815","nonce":4028436523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6813","nonce":4028436523}]},"public_addr":"172.21.15.52:6809/4028436523","cluster_addr":"172.21.15.52:6811/4028436523","heartbeat_back_addr":"172.21.15.52:6815/4028436523","heartbeat_front_addr":"172.21.15.52:6813/4028436523","state":["exists","up"]},{"osd":5,"uuid":"556ac804-02ea-423b-a599-55baf5be4ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6809","nonce":787132252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6811","nonce":787132252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6815","nonce":787132252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6813","nonce":787132252}]},"public_addr":"172.21.15.183:6809/787132252","cluster_addr":"172.21.15.183:6811/787132252","heartbeat_back_addr":"172.21.15.183:6815/787132252","heartbeat_front_addr":"172.21.15.183:6813/787132252","state":["exists","up"]},{"osd":6,"uuid":"3de2a49d-6cef-4580-9d8a-f90cdfcecd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6819","nonce":4206861158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6821","nonce":4206861158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6825","nonce":4206861158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6823","nonce":4206861158}]},"public_addr":"172.21.15.43:6819/4206861158","cluster_addr":"172.21.15.43:6821/4206861158","heartbeat_back_addr":"172.21.15.43:6825/4206861158","heartbeat_front_addr":"172.21.15.43:6823/4206861158","state":["exists","up"]},{"osd":7,"uuid":"a3b832f7-7d7d-4a62-b679-db6da98275d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6817","nonce":3452064146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6819","nonce":3452064146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6823","nonce":3452064146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6821","nonce":3452064146}]},"public_addr":"172.21.15.52:6817/3452064146","cluster_addr":"172.21.15.52:6819/3452064146","heartbeat_back_addr":"172.21.15.52:6823/3452064146","heartbeat_front_addr":"172.21.15.52:6821/3452064146","state":["exists","up"]},{"osd":8,"uuid":"ddc6b07f-d703-414d-a59a-02316376b13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6817","nonce":1023131725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6819","nonce":1023131725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6823","nonce":1023131725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6821","nonce":1023131725}]},"public_addr":"172.21.15.183:6817/1023131725","cluster_addr":"172.21.15.183:6819/1023131725","heartbeat_back_addr":"172.21.15.183:6823/1023131725","heartbeat_front_addr":"172.21.15.183:6821/1023131725","state":["exists","up"]},{"osd":9,"uuid":"ac16bb0e-6232-45c8-b16a-4d878874c588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6827","nonce":3935195657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6829","nonce":3935195657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6833","nonce":3935195657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6831","nonce":3935195657}]},"public_addr":"172.21.15.43:6827/3935195657","cluster_addr":"172.21.15.43:6829/3935195657","heartbeat_back_addr":"172.21.15.43:6833/3935195657","heartbeat_front_addr":"172.21.15.43:6831/3935195657","state":["exists","up"]},{"osd":10,"uuid":"fd839188-f398-4cff-aa7b-36522ff1a8a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6825","nonce":377628320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6827","nonce":377628320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6831","nonce":377628320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6829","nonce":377628320}]},"public_addr":"172.21.15.52:6825/377628320","cluster_addr":"172.21.15.52:6827/377628320","heartbeat_back_addr":"172.21.15.52:6831/377628320","heartbeat_front_addr":"172.21.15.52:6829/377628320","state":["exists","up"]},{"osd":11,"uuid":"a40282a0-0c64-49ee-b64e-6f3f8119e827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6825","nonce":1373086433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6827","nonce":1373086433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6831","nonce":1373086433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6829","nonce":1373086433}]},"public_addr":"172.21.15.183:6825/1373086433","cluster_addr":"172.21.15.183:6827/1373086433","heartbeat_back_addr":"172.21.15.183:6831/1373086433","heartbeat_front_addr":"172.21.15.183:6829/1373086433","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:23:43.478905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:17.573532+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:50.286610+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:25:27.407663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:03.081294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:36.057416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:12.062013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:47.032271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:21.023201+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:57.364739+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:29:34.039995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:30:08.827577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6801/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/4240775295":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/510203140":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/4251957687":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:6801/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1885244665":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/298320867":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1301092625":"2024-03-27T08:21:36.149092+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:31:28.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.391+0000 7f2973000640 1 -- 172.21.15.43:0/1417304632 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f29500a2880 msgr2=0x7f29500a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:28.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.391+0000 7f2973000640 1 --2- 172.21.15.43:0/1417304632 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f29500a2880 0x7f29500a4d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f295c02c240 tx=0x7f295c005fd0 comp rx=0 tx=0).stop 2024-03-26T08:31:28.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.391+0000 7f2973000640 1 -- 172.21.15.43:0/1417304632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f296c108ed0 msgr2=0x7f296c19e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:28.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.391+0000 7f2973000640 1 --2- 172.21.15.43:0/1417304632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f296c108ed0 0x7f296c19e490 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f295800d770 tx=0x7f295800dc30 comp rx=0 tx=0).stop 2024-03-26T08:31:28.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.391+0000 7f2973000640 1 -- 172.21.15.43:0/1417304632 shutdown_connections 2024-03-26T08:31:28.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.391+0000 7f2973000640 1 --2- 172.21.15.43:0/1417304632 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f29500a2880 0x7f29500a4d40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:28.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.391+0000 7f2973000640 1 --2- 172.21.15.43:0/1417304632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f296c109890 0x7f296c1a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:28.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.391+0000 7f2973000640 1 --2- 172.21.15.43:0/1417304632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f296c108ed0 0x7f296c19e490 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:28.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.391+0000 7f2973000640 1 --2- 172.21.15.43:0/1417304632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f296c107ce0 0x7f296c19df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:28.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.391+0000 7f2973000640 1 -- 172.21.15.43:0/1417304632 >> 172.21.15.43:0/1417304632 conn(0x7f296c076cb0 msgr2=0x7f296c10c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:28.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.391+0000 7f2973000640 1 -- 172.21.15.43:0/1417304632 shutdown_connections 2024-03-26T08:31:28.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.392+0000 7f2973000640 1 -- 172.21.15.43:0/1417304632 wait complete. 2024-03-26T08:31:28.407 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:31:28.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.649+0000 7f5f97400640 1 -- 172.21.15.43:0/2969731069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f90108ed0 msgr2=0x7f5f90109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:28.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.649+0000 7f5f97400640 1 --2- 172.21.15.43:0/2969731069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f90108ed0 0x7f5f90109350 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5f78009900 tx=0x7f5f7802bc30 comp rx=0 tx=0).stop 2024-03-26T08:31:28.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.649+0000 7f5f97400640 1 -- 172.21.15.43:0/2969731069 shutdown_connections 2024-03-26T08:31:28.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.649+0000 7f5f97400640 1 --2- 172.21.15.43:0/2969731069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f90109890 0x7f5f90110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:28.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.649+0000 7f5f97400640 1 --2- 172.21.15.43:0/2969731069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f90108ed0 0x7f5f90109350 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:28.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.649+0000 7f5f97400640 1 --2- 172.21.15.43:0/2969731069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f90107ce0 0x7f5f90108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:28.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.649+0000 7f5f97400640 1 -- 172.21.15.43:0/2969731069 >> 172.21.15.43:0/2969731069 conn(0x7f5f90076cb0 msgr2=0x7f5f900790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:28.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.649+0000 7f5f97400640 1 -- 172.21.15.43:0/2969731069 shutdown_connections 2024-03-26T08:31:28.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.649+0000 7f5f97400640 1 -- 172.21.15.43:0/2969731069 wait complete. 2024-03-26T08:31:28.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.650+0000 7f5f97400640 1 Processor -- start 2024-03-26T08:31:28.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.650+0000 7f5f97400640 1 -- start start 2024-03-26T08:31:28.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.651+0000 7f5f97400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f90107ce0 0x7f5f9019df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:28.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.651+0000 7f5f97400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f90108ed0 0x7f5f9019e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:28.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.651+0000 7f5f97400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f90109890 0x7f5f901a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:28.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.651+0000 7f5f97400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f90113810 con 0x7f5f90108ed0 2024-03-26T08:31:28.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.651+0000 7f5f97400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f90113690 con 0x7f5f90109890 2024-03-26T08:31:28.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.651+0000 7f5f97400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f90113990 con 0x7f5f90107ce0 2024-03-26T08:31:28.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.651+0000 7f5f8fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f90108ed0 0x7f5f9019e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:28.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.651+0000 7f5f8fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f90108ed0 0x7f5f9019e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53630/0 (socket says 172.21.15.43:53630) 2024-03-26T08:31:28.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.651+0000 7f5f8fe00640 1 -- 172.21.15.43:0/63300886 learned_addr learned my addr 172.21.15.43:0/63300886 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:28.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.652+0000 7f5f94a00640 1 --2- 172.21.15.43:0/63300886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f90107ce0 0x7f5f9019df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:28.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.652+0000 7f5f95400640 1 --2- 172.21.15.43:0/63300886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f90109890 0x7f5f901a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:28.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.652+0000 7f5f8fe00640 1 -- 172.21.15.43:0/63300886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f90107ce0 msgr2=0x7f5f9019df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:28.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.652+0000 7f5f8fe00640 1 --2- 172.21.15.43:0/63300886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f90107ce0 0x7f5f9019df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:28.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.652+0000 7f5f8fe00640 1 -- 172.21.15.43:0/63300886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f90109890 msgr2=0x7f5f901a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:28.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.652+0000 7f5f8fe00640 1 --2- 172.21.15.43:0/63300886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f90109890 0x7f5f901a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:28.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.652+0000 7f5f8fe00640 1 -- 172.21.15.43:0/63300886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f901a5b30 con 0x7f5f90108ed0 2024-03-26T08:31:28.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.652+0000 7f5f95400640 1 --2- 172.21.15.43:0/63300886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f90109890 0x7f5f901a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:31:28.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.652+0000 7f5f8fe00640 1 --2- 172.21.15.43:0/63300886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f90108ed0 0x7f5f9019e450 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5f78006010 tx=0x7f5f780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:28.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.652+0000 7f5f8d600640 1 -- 172.21.15.43:0/63300886 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f78043020 con 0x7f5f90108ed0 2024-03-26T08:31:28.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.652+0000 7f5f8d600640 1 -- 172.21.15.43:0/63300886 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5f78004aa0 con 0x7f5f90108ed0 2024-03-26T08:31:28.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.652+0000 7f5f8d600640 1 -- 172.21.15.43:0/63300886 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f78003c60 con 0x7f5f90108ed0 2024-03-26T08:31:28.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.652+0000 7f5f97400640 1 -- 172.21.15.43:0/63300886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f901a5dc0 con 0x7f5f90108ed0 2024-03-26T08:31:28.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.653+0000 7f5f97400640 1 -- 172.21.15.43:0/63300886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f901a6280 con 0x7f5f90108ed0 2024-03-26T08:31:28.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.654+0000 7f5f97400640 1 -- 172.21.15.43:0/63300886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f9010f080 con 0x7f5f90108ed0 2024-03-26T08:31:28.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.656+0000 7f5f8d600640 1 -- 172.21.15.43:0/63300886 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5f78046020 con 0x7f5f90108ed0 2024-03-26T08:31:28.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.658+0000 7f5f8d600640 1 --2- 172.21.15.43:0/63300886 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5f700a2800 0x7f5f700a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:28.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:28 smithi043 ceph-mon[31929]: osdmap e90: 12 total, 12 up, 12 in 2024-03-26T08:31:28.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:28 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2632881647' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-26T08:31:28.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:28 smithi043 ceph-mon[31929]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T08:31:28.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:28 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/152586734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:28.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:28 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1417304632' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:31:28.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.659+0000 7f5f8d600640 1 -- 172.21.15.43:0/63300886 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f5f780f1190 con 0x7f5f90108ed0 2024-03-26T08:31:28.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.659+0000 7f5f94a00640 1 --2- 172.21.15.43:0/63300886 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5f700a2800 0x7f5f700a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:28.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.659+0000 7f5f94a00640 1 --2- 172.21.15.43:0/63300886 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5f700a2800 0x7f5f700a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5f80005fd0 tx=0x7f5f80005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:28.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.663+0000 7f5f8d600640 1 -- 172.21.15.43:0/63300886 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5f7804d020 con 0x7f5f90108ed0 2024-03-26T08:31:28.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:28 smithi183 ceph-mon[35818]: osdmap e90: 12 total, 12 up, 12 in 2024-03-26T08:31:28.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:28 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2632881647' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-26T08:31:28.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:28 smithi183 ceph-mon[35818]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T08:31:28.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:28 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/152586734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:28.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:28 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1417304632' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:31:28.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:28 smithi052 ceph-mon[35071]: osdmap e90: 12 total, 12 up, 12 in 2024-03-26T08:31:28.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:28 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2632881647' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-26T08:31:28.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:28 smithi052 ceph-mon[35071]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T08:31:28.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:28 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/152586734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:28.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:28 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1417304632' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:31:28.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.870+0000 7f5f97400640 1 -- 172.21.15.43:0/63300886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f90062340 con 0x7f5f90108ed0 2024-03-26T08:31:28.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.874+0000 7f5f8d600640 1 -- 172.21.15.43:0/63300886 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24080 (secure 0 0 0) 0x7f5f780bdf90 con 0x7f5f90108ed0 2024-03-26T08:31:28.868 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:31:28.868 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":90,"fsid":"88759420-eb49-11ee-95d0-87774f69a715","created":"2024-03-26T08:20:27.817107+0000","modified":"2024-03-26T08:31:26.907701+0000","last_up_change":"2024-03-26T08:30:10.520976+0000","last_in_change":"2024-03-26T08:29:46.394360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:24:54.852427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:31:19.924902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:31:21.323881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T08:31:24.951015+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"28670673-9193-4fa3-9cc0-c1a5a6b27284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6803","nonce":936601146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6805","nonce":936601146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6809","nonce":936601146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6807","nonce":936601146}]},"public_addr":"172.21.15.43:6803/936601146","cluster_addr":"172.21.15.43:6805/936601146","heartbeat_back_addr":"172.21.15.43:6809/936601146","heartbeat_front_addr":"172.21.15.43:6807/936601146","state":["exists","up"]},{"osd":1,"uuid":"fa8ad6b6-277a-42b8-b0f8-b36fc219de2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6801","nonce":3268412328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6803","nonce":3268412328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6807","nonce":3268412328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6805","nonce":3268412328}]},"public_addr":"172.21.15.52:6801/3268412328","cluster_addr":"172.21.15.52:6803/3268412328","heartbeat_back_addr":"172.21.15.52:6807/3268412328","heartbeat_front_addr":"172.21.15.52:6805/3268412328","state":["exists","up"]},{"osd":2,"uuid":"ec0039c3-f9bd-419a-81de-fa4418359808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6801","nonce":2103180908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6803","nonce":2103180908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6807","nonce":2103180908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6805","nonce":2103180908}]},"public_addr":"172.21.15.183:6801/2103180908","cluster_addr":"172.21.15.183:6803/2103180908","heartbeat_back_addr":"172.21.15.183:6807/2103180908","heartbeat_front_addr":"172.21.15.183:6805/2103180908","state":["exists","up"]},{"osd":3,"uuid":"e7dcbf00-6586-4664-a69e-3cb0df8dfd30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6811","nonce":3722846033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6813","nonce":3722846033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6817","nonce":3722846033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6815","nonce":3722846033}]},"public_addr":"172.21.15.43:6811/3722846033","cluster_addr":"172.21.15.43:6813/3722846033","heartbeat_back_addr":"172.21.15.43:6817/3722846033","heartbeat_front_addr":"172.21.15.43:6815/3722846033","state":["exists","up"]},{"osd":4,"uuid":"db089a49-5e98-432d-9687-cdc044114c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6809","nonce":4028436523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6811","nonce":4028436523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6815","nonce":4028436523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6813","nonce":4028436523}]},"public_addr":"172.21.15.52:6809/4028436523","cluster_addr":"172.21.15.52:6811/4028436523","heartbeat_back_addr":"172.21.15.52:6815/4028436523","heartbeat_front_addr":"172.21.15.52:6813/4028436523","state":["exists","up"]},{"osd":5,"uuid":"556ac804-02ea-423b-a599-55baf5be4ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6809","nonce":787132252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6811","nonce":787132252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6815","nonce":787132252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6813","nonce":787132252}]},"public_addr":"172.21.15.183:6809/787132252","cluster_addr":"172.21.15.183:6811/787132252","heartbeat_back_addr":"172.21.15.183:6815/787132252","heartbeat_front_addr":"172.21.15.183:6813/787132252","state":["exists","up"]},{"osd":6,"uuid":"3de2a49d-6cef-4580-9d8a-f90cdfcecd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6819","nonce":4206861158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6821","nonce":4206861158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6825","nonce":4206861158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6823","nonce":4206861158}]},"public_addr":"172.21.15.43:6819/4206861158","cluster_addr":"172.21.15.43:6821/4206861158","heartbeat_back_addr":"172.21.15.43:6825/4206861158","heartbeat_front_addr":"172.21.15.43:6823/4206861158","state":["exists","up"]},{"osd":7,"uuid":"a3b832f7-7d7d-4a62-b679-db6da98275d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6817","nonce":3452064146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6819","nonce":3452064146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6823","nonce":3452064146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6821","nonce":3452064146}]},"public_addr":"172.21.15.52:6817/3452064146","cluster_addr":"172.21.15.52:6819/3452064146","heartbeat_back_addr":"172.21.15.52:6823/3452064146","heartbeat_front_addr":"172.21.15.52:6821/3452064146","state":["exists","up"]},{"osd":8,"uuid":"ddc6b07f-d703-414d-a59a-02316376b13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6817","nonce":1023131725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6819","nonce":1023131725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6823","nonce":1023131725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6821","nonce":1023131725}]},"public_addr":"172.21.15.183:6817/1023131725","cluster_addr":"172.21.15.183:6819/1023131725","heartbeat_back_addr":"172.21.15.183:6823/1023131725","heartbeat_front_addr":"172.21.15.183:6821/1023131725","state":["exists","up"]},{"osd":9,"uuid":"ac16bb0e-6232-45c8-b16a-4d878874c588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6827","nonce":3935195657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6829","nonce":3935195657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6833","nonce":3935195657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6831","nonce":3935195657}]},"public_addr":"172.21.15.43:6827/3935195657","cluster_addr":"172.21.15.43:6829/3935195657","heartbeat_back_addr":"172.21.15.43:6833/3935195657","heartbeat_front_addr":"172.21.15.43:6831/3935195657","state":["exists","up"]},{"osd":10,"uuid":"fd839188-f398-4cff-aa7b-36522ff1a8a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6825","nonce":377628320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6827","nonce":377628320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6831","nonce":377628320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6829","nonce":377628320}]},"public_addr":"172.21.15.52:6825/377628320","cluster_addr":"172.21.15.52:6827/377628320","heartbeat_back_addr":"172.21.15.52:6831/377628320","heartbeat_front_addr":"172.21.15.52:6829/377628320","state":["exists","up"]},{"osd":11,"uuid":"a40282a0-0c64-49ee-b64e-6f3f8119e827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6825","nonce":1373086433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6827","nonce":1373086433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6831","nonce":1373086433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6829","nonce":1373086433}]},"public_addr":"172.21.15.183:6825/1373086433","cluster_addr":"172.21.15.183:6827/1373086433","heartbeat_back_addr":"172.21.15.183:6831/1373086433","heartbeat_front_addr":"172.21.15.183:6829/1373086433","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:23:43.478905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:17.573532+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:50.286610+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:25:27.407663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:03.081294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:36.057416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:12.062013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:47.032271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:21.023201+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:57.364739+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:29:34.039995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:30:08.827577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6801/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/4240775295":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/510203140":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/4251957687":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:6801/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1885244665":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/298320867":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1301092625":"2024-03-27T08:21:36.149092+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:31:28.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.877+0000 7f5f97400640 1 -- 172.21.15.43:0/63300886 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5f700a2800 msgr2=0x7f5f700a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:28.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.877+0000 7f5f97400640 1 --2- 172.21.15.43:0/63300886 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5f700a2800 0x7f5f700a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5f80005fd0 tx=0x7f5f80005850 comp rx=0 tx=0).stop 2024-03-26T08:31:28.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.877+0000 7f5f97400640 1 -- 172.21.15.43:0/63300886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f90108ed0 msgr2=0x7f5f9019e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:28.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.877+0000 7f5f97400640 1 --2- 172.21.15.43:0/63300886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f90108ed0 0x7f5f9019e450 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5f78006010 tx=0x7f5f780042f0 comp rx=0 tx=0).stop 2024-03-26T08:31:28.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.877+0000 7f5f97400640 1 -- 172.21.15.43:0/63300886 shutdown_connections 2024-03-26T08:31:28.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.877+0000 7f5f97400640 1 --2- 172.21.15.43:0/63300886 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5f700a2800 0x7f5f700a4cc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:28.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.877+0000 7f5f97400640 1 --2- 172.21.15.43:0/63300886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f90109890 0x7f5f901a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:28.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.877+0000 7f5f97400640 1 --2- 172.21.15.43:0/63300886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f90108ed0 0x7f5f9019e450 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:28.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.877+0000 7f5f97400640 1 --2- 172.21.15.43:0/63300886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f90107ce0 0x7f5f9019df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:28.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.877+0000 7f5f97400640 1 -- 172.21.15.43:0/63300886 >> 172.21.15.43:0/63300886 conn(0x7f5f90076cb0 msgr2=0x7f5f9010c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:28.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.878+0000 7f5f97400640 1 -- 172.21.15.43:0/63300886 shutdown_connections 2024-03-26T08:31:28.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:28.878+0000 7f5f97400640 1 -- 172.21.15.43:0/63300886 wait complete. 2024-03-26T08:31:28.907 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:31:29.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.140+0000 7f0803200640 1 -- 172.21.15.43:0/544178755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08041026d0 msgr2=0x7f0804102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:29.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.140+0000 7f0803200640 1 --2- 172.21.15.43:0/544178755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08041026d0 0x7f0804102b50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009ca0 tx=0x7f07ec02c0f0 comp rx=0 tx=0).stop 2024-03-26T08:31:29.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.140+0000 7f0803200640 1 -- 172.21.15.43:0/544178755 shutdown_connections 2024-03-26T08:31:29.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.140+0000 7f0803200640 1 --2- 172.21.15.43:0/544178755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0804103090 0x7f0804109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:29.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.140+0000 7f0803200640 1 --2- 172.21.15.43:0/544178755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08041026d0 0x7f0804102b50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:29.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.140+0000 7f0803200640 1 --2- 172.21.15.43:0/544178755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08041014e0 0x7f0804101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:29.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.140+0000 7f0803200640 1 -- 172.21.15.43:0/544178755 >> 172.21.15.43:0/544178755 conn(0x7f08040fca30 msgr2=0x7f08040fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:29.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.140+0000 7f0803200640 1 -- 172.21.15.43:0/544178755 shutdown_connections 2024-03-26T08:31:29.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.141+0000 7f0803200640 1 -- 172.21.15.43:0/544178755 wait complete. 2024-03-26T08:31:29.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.141+0000 7f0803200640 1 Processor -- start 2024-03-26T08:31:29.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.142+0000 7f0803200640 1 -- start start 2024-03-26T08:31:29.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.142+0000 7f0803200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08041014e0 0x7f0804199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:29.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.142+0000 7f0803200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0804103090 0x7f080419a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:29.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.142+0000 7f0803200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f080419f100 0x7f08041a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:29.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.142+0000 7f0803200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080410d020 con 0x7f080419f100 2024-03-26T08:31:29.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.142+0000 7f0803200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080410cea0 con 0x7f08041014e0 2024-03-26T08:31:29.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.142+0000 7f0803200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080410d1a0 con 0x7f0804103090 2024-03-26T08:31:29.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.143+0000 7f0801e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08041014e0 0x7f0804199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:29.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.143+0000 7f0802800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f080419f100 0x7f08041a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:29.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.143+0000 7f0802800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f080419f100 0x7f08041a14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53648/0 (socket says 172.21.15.43:53648) 2024-03-26T08:31:29.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.143+0000 7f0802800640 1 -- 172.21.15.43:0/1075894352 learned_addr learned my addr 172.21.15.43:0/1075894352 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:29.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.143+0000 7f0801400640 1 --2- 172.21.15.43:0/1075894352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0804103090 0x7f080419a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:29.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.143+0000 7f0802800640 1 -- 172.21.15.43:0/1075894352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0804103090 msgr2=0x7f080419a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:29.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.143+0000 7f0802800640 1 --2- 172.21.15.43:0/1075894352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0804103090 0x7f080419a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:29.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.143+0000 7f0802800640 1 -- 172.21.15.43:0/1075894352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08041014e0 msgr2=0x7f0804199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:29.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.143+0000 7f0802800640 1 --2- 172.21.15.43:0/1075894352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08041014e0 0x7f0804199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:29.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.143+0000 7f0802800640 1 -- 172.21.15.43:0/1075894352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08041a1b60 con 0x7f080419f100 2024-03-26T08:31:29.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.143+0000 7f0802800640 1 --2- 172.21.15.43:0/1075894352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f080419f100 0x7f08041a14f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00d750 tx=0x7f07fc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:29.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.144+0000 7f07faa00640 1 -- 172.21.15.43:0/1075894352 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc017070 con 0x7f080419f100 2024-03-26T08:31:29.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.144+0000 7f07faa00640 1 -- 172.21.15.43:0/1075894352 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f07fc0054d0 con 0x7f080419f100 2024-03-26T08:31:29.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.144+0000 7f07faa00640 1 -- 172.21.15.43:0/1075894352 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc004c60 con 0x7f080419f100 2024-03-26T08:31:29.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.144+0000 7f0803200640 1 -- 172.21.15.43:0/1075894352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08041a1e50 con 0x7f080419f100 2024-03-26T08:31:29.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.144+0000 7f0803200640 1 -- 172.21.15.43:0/1075894352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08041a2360 con 0x7f080419f100 2024-03-26T08:31:29.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.145+0000 7f0803200640 1 -- 172.21.15.43:0/1075894352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08041a2590 con 0x7f080419f100 2024-03-26T08:31:29.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.147+0000 7f07faa00640 1 -- 172.21.15.43:0/1075894352 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f07fc004e00 con 0x7f080419f100 2024-03-26T08:31:29.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.148+0000 7f07faa00640 1 --2- 172.21.15.43:0/1075894352 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f07e40a2880 0x7f07e40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:29.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.149+0000 7f07faa00640 1 -- 172.21.15.43:0/1075894352 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f07fc0ce550 con 0x7f080419f100 2024-03-26T08:31:29.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.149+0000 7f0801e00640 1 --2- 172.21.15.43:0/1075894352 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f07e40a2880 0x7f07e40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:29.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.149+0000 7f0801e00640 1 --2- 172.21.15.43:0/1075894352 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f07e40a2880 0x7f07e40a4d40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0804102530 tx=0x7f07f400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:29.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.153+0000 7f07faa00640 1 -- 172.21.15.43:0/1075894352 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f07fc09ad00 con 0x7f080419f100 2024-03-26T08:31:29.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.401+0000 7f0803200640 1 -- 172.21.15.43:0/1075894352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08041a2800 con 0x7f080419f100 2024-03-26T08:31:29.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.403+0000 7f07faa00640 1 -- 172.21.15.43:0/1075894352 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11321 (secure 0 0 0) 0x7f07fc09eda0 con 0x7f080419f100 2024-03-26T08:31:29.396 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:31:29.396 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1452206807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1452206807},{"type":"v1","addr":"172.21.15.43:6835","nonce":1452206807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3151775467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3151775467},{"type":"v1","addr":"172.21.15.43:6837","nonce":3151775467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/1284655682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":1284655682},{"type":"v1","addr":"172.21.15.43:6839","nonce":1284655682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3081085787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3081085787},{"type":"v1","addr":"172.21.15.43:6841","nonce":3081085787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1842295306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1842295306},{"type":"v1","addr":"172.21.15.183:6835","nonce":1842295306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2267781666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2267781666},{"type":"v1","addr":"172.21.15.183:6837","nonce":2267781666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3572429948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3572429948},{"type":"v1","addr":"172.21.15.183:6839","nonce":3572429948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34448,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/1108327179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108327179},{"type":"v1","addr":"172.21.15.52:6833","nonce":1108327179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34487,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/742057410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":742057410},{"type":"v1","addr":"172.21.15.52:6835","nonce":742057410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34517,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/748239918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":748239918},{"type":"v1","addr":"172.21.15.52:6837","nonce":748239918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-26T08:31:21.819689+0000","modified":"2024-03-26T08:31:26.907931+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34559":{"gid":34559,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.52:6839/2551965101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2551965101},{"type":"v1","addr":"172.21.15.52:6839","nonce":2551965101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34559,"qdb_cluster":[34559]},"id":1}]} 2024-03-26T08:31:29.396 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 17 2024-03-26T08:31:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.405+0000 7f0803200640 1 -- 172.21.15.43:0/1075894352 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f07e40a2880 msgr2=0x7f07e40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.405+0000 7f0803200640 1 --2- 172.21.15.43:0/1075894352 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f07e40a2880 0x7f07e40a4d40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0804102530 tx=0x7f07f400b390 comp rx=0 tx=0).stop 2024-03-26T08:31:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.405+0000 7f0803200640 1 -- 172.21.15.43:0/1075894352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f080419f100 msgr2=0x7f08041a14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.405+0000 7f0803200640 1 --2- 172.21.15.43:0/1075894352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f080419f100 0x7f08041a14f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00d750 tx=0x7f07fc00dc10 comp rx=0 tx=0).stop 2024-03-26T08:31:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.406+0000 7f0803200640 1 -- 172.21.15.43:0/1075894352 shutdown_connections 2024-03-26T08:31:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.406+0000 7f0803200640 1 --2- 172.21.15.43:0/1075894352 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f07e40a2880 0x7f07e40a4d40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.406+0000 7f0803200640 1 --2- 172.21.15.43:0/1075894352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f080419f100 0x7f08041a14f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.406+0000 7f0803200640 1 --2- 172.21.15.43:0/1075894352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0804103090 0x7f080419a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.406+0000 7f0803200640 1 --2- 172.21.15.43:0/1075894352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08041014e0 0x7f0804199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.406+0000 7f0803200640 1 -- 172.21.15.43:0/1075894352 >> 172.21.15.43:0/1075894352 conn(0x7f08040fca30 msgr2=0x7f0804107f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.406+0000 7f0803200640 1 -- 172.21.15.43:0/1075894352 shutdown_connections 2024-03-26T08:31:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.406+0000 7f0803200640 1 -- 172.21.15.43:0/1075894352 wait complete. 2024-03-26T08:31:29.420 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:31:29.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.645+0000 7f758fe00640 1 -- 172.21.15.43:0/2239173534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75901036f0 msgr2=0x7f7590103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:29.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.645+0000 7f758fe00640 1 --2- 172.21.15.43:0/2239173534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75901036f0 0x7f7590103b10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7580009900 tx=0x7f758002bc40 comp rx=0 tx=0).stop 2024-03-26T08:31:29.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.646+0000 7f758fe00640 1 -- 172.21.15.43:0/2239173534 shutdown_connections 2024-03-26T08:31:29.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.646+0000 7f758fe00640 1 --2- 172.21.15.43:0/2239173534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75901052a0 0x7f759010bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:29.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.646+0000 7f758fe00640 1 --2- 172.21.15.43:0/2239173534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75901048e0 0x7f7590104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:29.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.646+0000 7f758fe00640 1 --2- 172.21.15.43:0/2239173534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75901036f0 0x7f7590103b10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:29.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.646+0000 7f758fe00640 1 -- 172.21.15.43:0/2239173534 >> 172.21.15.43:0/2239173534 conn(0x7f75900fec80 msgr2=0x7f75901010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:29.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.646+0000 7f758fe00640 1 -- 172.21.15.43:0/2239173534 shutdown_connections 2024-03-26T08:31:29.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.646+0000 7f758fe00640 1 -- 172.21.15.43:0/2239173534 wait complete. 2024-03-26T08:31:29.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.647+0000 7f758fe00640 1 Processor -- start 2024-03-26T08:31:29.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.647+0000 7f758fe00640 1 -- start start 2024-03-26T08:31:29.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.647+0000 7f758fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75901036f0 0x7f75901999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:29.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.647+0000 7f758fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75901048e0 0x7f7590199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:29.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.647+0000 7f758fe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75901052a0 0x7f75901a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:29.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.647+0000 7f758fe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759010f2c0 con 0x7f75901048e0 2024-03-26T08:31:29.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.647+0000 7f758fe00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759010f140 con 0x7f75901052a0 2024-03-26T08:31:29.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.647+0000 7f758fe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759010f440 con 0x7f75901036f0 2024-03-26T08:31:29.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.647+0000 7f7587e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75901048e0 0x7f7590199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:29.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.648+0000 7f7587e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75901048e0 0x7f7590199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53676/0 (socket says 172.21.15.43:53676) 2024-03-26T08:31:29.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.648+0000 7f7587e00640 1 -- 172.21.15.43:0/685957686 learned_addr learned my addr 172.21.15.43:0/685957686 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:29.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.648+0000 7f758ea00640 1 --2- 172.21.15.43:0/685957686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75901036f0 0x7f75901999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:29.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.648+0000 7f758f400640 1 --2- 172.21.15.43:0/685957686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75901052a0 0x7f75901a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:29.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.648+0000 7f7587e00640 1 -- 172.21.15.43:0/685957686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75901036f0 msgr2=0x7f75901999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:29.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.648+0000 7f7587e00640 1 --2- 172.21.15.43:0/685957686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75901036f0 0x7f75901999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:29.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.648+0000 7f7587e00640 1 -- 172.21.15.43:0/685957686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75901052a0 msgr2=0x7f75901a0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:29.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.648+0000 7f7587e00640 1 --2- 172.21.15.43:0/685957686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75901052a0 0x7f75901a0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:29.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.648+0000 7f7587e00640 1 -- 172.21.15.43:0/685957686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75901a15c0 con 0x7f75901048e0 2024-03-26T08:31:29.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.648+0000 7f7587e00640 1 --2- 172.21.15.43:0/685957686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75901048e0 0x7f7590199ee0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f757800b4a0 tx=0x7f757800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:29.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.649+0000 7f7587400640 1 -- 172.21.15.43:0/685957686 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7578014070 con 0x7f75901048e0 2024-03-26T08:31:29.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.649+0000 7f7587400640 1 -- 172.21.15.43:0/685957686 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7578010ac0 con 0x7f75901048e0 2024-03-26T08:31:29.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.649+0000 7f7587400640 1 -- 172.21.15.43:0/685957686 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7578017c30 con 0x7f75901048e0 2024-03-26T08:31:29.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.649+0000 7f758fe00640 1 -- 172.21.15.43:0/685957686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75901a18b0 con 0x7f75901048e0 2024-03-26T08:31:29.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.649+0000 7f758fe00640 1 -- 172.21.15.43:0/685957686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75901a1e30 con 0x7f75901048e0 2024-03-26T08:31:29.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.650+0000 7f758fe00640 1 -- 172.21.15.43:0/685957686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f759010aa90 con 0x7f75901048e0 2024-03-26T08:31:29.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.652+0000 7f7587400640 1 -- 172.21.15.43:0/685957686 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f7578017070 con 0x7f75901048e0 2024-03-26T08:31:29.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.653+0000 7f7587400640 1 --2- 172.21.15.43:0/685957686 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f75740a2880 0x7f75740a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:29.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.653+0000 7f7587400640 1 -- 172.21.15.43:0/685957686 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f75780ce430 con 0x7f75901048e0 2024-03-26T08:31:29.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.654+0000 7f758ea00640 1 --2- 172.21.15.43:0/685957686 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f75740a2880 0x7f75740a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:29.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.654+0000 7f758ea00640 1 --2- 172.21.15.43:0/685957686 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f75740a2880 0x7f75740a4d40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f758002c250 tx=0x7f7580000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:29.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.658+0000 7f7587400640 1 -- 172.21.15.43:0/685957686 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f757809b230 con 0x7f75901048e0 2024-03-26T08:31:29.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:29 smithi183 ceph-mon[35818]: pgmap v376: 129 pgs: 10 creating+peering, 101 active+clean, 18 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 4 op/s 2024-03-26T08:31:29.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:29 smithi183 ceph-mon[35818]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-26T08:31:29.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:29 smithi183 ceph-mon[35818]: Cluster is now healthy 2024-03-26T08:31:29.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:29 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/63300886' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:31:29.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:29 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1075894352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:29.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:29 smithi052 ceph-mon[35071]: pgmap v376: 129 pgs: 10 creating+peering, 101 active+clean, 18 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 4 op/s 2024-03-26T08:31:29.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:29 smithi052 ceph-mon[35071]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-26T08:31:29.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:29 smithi052 ceph-mon[35071]: Cluster is now healthy 2024-03-26T08:31:29.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:29 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/63300886' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:31:29.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:29 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1075894352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:29.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.864+0000 7f758fe00640 1 -- 172.21.15.43:0/685957686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7590062340 con 0x7f75901048e0 2024-03-26T08:31:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.868+0000 7f7587400640 1 -- 172.21.15.43:0/685957686 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24080 (secure 0 0 0) 0x7f757809ab30 con 0x7f75901048e0 2024-03-26T08:31:29.862 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:31:29.862 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":90,"fsid":"88759420-eb49-11ee-95d0-87774f69a715","created":"2024-03-26T08:20:27.817107+0000","modified":"2024-03-26T08:31:26.907701+0000","last_up_change":"2024-03-26T08:30:10.520976+0000","last_in_change":"2024-03-26T08:29:46.394360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:24:54.852427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:31:19.924902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:31:21.323881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T08:31:24.951015+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"28670673-9193-4fa3-9cc0-c1a5a6b27284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6803","nonce":936601146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6805","nonce":936601146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6809","nonce":936601146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6807","nonce":936601146}]},"public_addr":"172.21.15.43:6803/936601146","cluster_addr":"172.21.15.43:6805/936601146","heartbeat_back_addr":"172.21.15.43:6809/936601146","heartbeat_front_addr":"172.21.15.43:6807/936601146","state":["exists","up"]},{"osd":1,"uuid":"fa8ad6b6-277a-42b8-b0f8-b36fc219de2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6801","nonce":3268412328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6803","nonce":3268412328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6807","nonce":3268412328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6805","nonce":3268412328}]},"public_addr":"172.21.15.52:6801/3268412328","cluster_addr":"172.21.15.52:6803/3268412328","heartbeat_back_addr":"172.21.15.52:6807/3268412328","heartbeat_front_addr":"172.21.15.52:6805/3268412328","state":["exists","up"]},{"osd":2,"uuid":"ec0039c3-f9bd-419a-81de-fa4418359808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6801","nonce":2103180908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6803","nonce":2103180908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6807","nonce":2103180908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6805","nonce":2103180908}]},"public_addr":"172.21.15.183:6801/2103180908","cluster_addr":"172.21.15.183:6803/2103180908","heartbeat_back_addr":"172.21.15.183:6807/2103180908","heartbeat_front_addr":"172.21.15.183:6805/2103180908","state":["exists","up"]},{"osd":3,"uuid":"e7dcbf00-6586-4664-a69e-3cb0df8dfd30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6811","nonce":3722846033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6813","nonce":3722846033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6817","nonce":3722846033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6815","nonce":3722846033}]},"public_addr":"172.21.15.43:6811/3722846033","cluster_addr":"172.21.15.43:6813/3722846033","heartbeat_back_addr":"172.21.15.43:6817/3722846033","heartbeat_front_addr":"172.21.15.43:6815/3722846033","state":["exists","up"]},{"osd":4,"uuid":"db089a49-5e98-432d-9687-cdc044114c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6809","nonce":4028436523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6811","nonce":4028436523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6815","nonce":4028436523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6813","nonce":4028436523}]},"public_addr":"172.21.15.52:6809/4028436523","cluster_addr":"172.21.15.52:6811/4028436523","heartbeat_back_addr":"172.21.15.52:6815/4028436523","heartbeat_front_addr":"172.21.15.52:6813/4028436523","state":["exists","up"]},{"osd":5,"uuid":"556ac804-02ea-423b-a599-55baf5be4ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6809","nonce":787132252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6811","nonce":787132252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6815","nonce":787132252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6813","nonce":787132252}]},"public_addr":"172.21.15.183:6809/787132252","cluster_addr":"172.21.15.183:6811/787132252","heartbeat_back_addr":"172.21.15.183:6815/787132252","heartbeat_front_addr":"172.21.15.183:6813/787132252","state":["exists","up"]},{"osd":6,"uuid":"3de2a49d-6cef-4580-9d8a-f90cdfcecd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6819","nonce":4206861158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6821","nonce":4206861158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6825","nonce":4206861158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6823","nonce":4206861158}]},"public_addr":"172.21.15.43:6819/4206861158","cluster_addr":"172.21.15.43:6821/4206861158","heartbeat_back_addr":"172.21.15.43:6825/4206861158","heartbeat_front_addr":"172.21.15.43:6823/4206861158","state":["exists","up"]},{"osd":7,"uuid":"a3b832f7-7d7d-4a62-b679-db6da98275d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6817","nonce":3452064146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6819","nonce":3452064146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6823","nonce":3452064146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6821","nonce":3452064146}]},"public_addr":"172.21.15.52:6817/3452064146","cluster_addr":"172.21.15.52:6819/3452064146","heartbeat_back_addr":"172.21.15.52:6823/3452064146","heartbeat_front_addr":"172.21.15.52:6821/3452064146","state":["exists","up"]},{"osd":8,"uuid":"ddc6b07f-d703-414d-a59a-02316376b13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6817","nonce":1023131725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6819","nonce":1023131725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6823","nonce":1023131725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6821","nonce":1023131725}]},"public_addr":"172.21.15.183:6817/1023131725","cluster_addr":"172.21.15.183:6819/1023131725","heartbeat_back_addr":"172.21.15.183:6823/1023131725","heartbeat_front_addr":"172.21.15.183:6821/1023131725","state":["exists","up"]},{"osd":9,"uuid":"ac16bb0e-6232-45c8-b16a-4d878874c588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6827","nonce":3935195657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6829","nonce":3935195657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6833","nonce":3935195657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6831","nonce":3935195657}]},"public_addr":"172.21.15.43:6827/3935195657","cluster_addr":"172.21.15.43:6829/3935195657","heartbeat_back_addr":"172.21.15.43:6833/3935195657","heartbeat_front_addr":"172.21.15.43:6831/3935195657","state":["exists","up"]},{"osd":10,"uuid":"fd839188-f398-4cff-aa7b-36522ff1a8a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6825","nonce":377628320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6827","nonce":377628320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6831","nonce":377628320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6829","nonce":377628320}]},"public_addr":"172.21.15.52:6825/377628320","cluster_addr":"172.21.15.52:6827/377628320","heartbeat_back_addr":"172.21.15.52:6831/377628320","heartbeat_front_addr":"172.21.15.52:6829/377628320","state":["exists","up"]},{"osd":11,"uuid":"a40282a0-0c64-49ee-b64e-6f3f8119e827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6825","nonce":1373086433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6827","nonce":1373086433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6831","nonce":1373086433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6829","nonce":1373086433}]},"public_addr":"172.21.15.183:6825/1373086433","cluster_addr":"172.21.15.183:6827/1373086433","heartbeat_back_addr":"172.21.15.183:6831/1373086433","heartbeat_front_addr":"172.21.15.183:6829/1373086433","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:23:43.478905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:17.573532+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:50.286610+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:25:27.407663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:03.081294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:36.057416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:12.062013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:47.032271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:21.023201+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:57.364739+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:29:34.039995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:30:08.827577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6801/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/4240775295":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/510203140":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/4251957687":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:6801/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1885244665":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/298320867":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1301092625":"2024-03-27T08:21:36.149092+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:31:29.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.871+0000 7f758fe00640 1 -- 172.21.15.43:0/685957686 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f75740a2880 msgr2=0x7f75740a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:29.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.871+0000 7f758fe00640 1 --2- 172.21.15.43:0/685957686 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f75740a2880 0x7f75740a4d40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f758002c250 tx=0x7f7580000bc0 comp rx=0 tx=0).stop 2024-03-26T08:31:29.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.871+0000 7f758fe00640 1 -- 172.21.15.43:0/685957686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75901048e0 msgr2=0x7f7590199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:29.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.871+0000 7f758fe00640 1 --2- 172.21.15.43:0/685957686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75901048e0 0x7f7590199ee0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f757800b4a0 tx=0x7f757800b960 comp rx=0 tx=0).stop 2024-03-26T08:31:29.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.871+0000 7f758fe00640 1 -- 172.21.15.43:0/685957686 shutdown_connections 2024-03-26T08:31:29.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.871+0000 7f758fe00640 1 --2- 172.21.15.43:0/685957686 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f75740a2880 0x7f75740a4d40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:29.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.871+0000 7f758fe00640 1 --2- 172.21.15.43:0/685957686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75901052a0 0x7f75901a0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:29.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.871+0000 7f758fe00640 1 --2- 172.21.15.43:0/685957686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75901048e0 0x7f7590199ee0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:29.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.871+0000 7f758fe00640 1 --2- 172.21.15.43:0/685957686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75901036f0 0x7f75901999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:29.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.871+0000 7f758fe00640 1 -- 172.21.15.43:0/685957686 >> 172.21.15.43:0/685957686 conn(0x7f75900fec80 msgr2=0x7f7590107c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:29.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.871+0000 7f758fe00640 1 -- 172.21.15.43:0/685957686 shutdown_connections 2024-03-26T08:31:29.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:29.872+0000 7f758fe00640 1 -- 172.21.15.43:0/685957686 wait complete. 2024-03-26T08:31:29.880 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:29 smithi043 ceph-mon[31929]: pgmap v376: 129 pgs: 10 creating+peering, 101 active+clean, 18 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 4 op/s 2024-03-26T08:31:29.880 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:29 smithi043 ceph-mon[31929]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-26T08:31:29.880 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:29 smithi043 ceph-mon[31929]: Cluster is now healthy 2024-03-26T08:31:29.880 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:29 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/63300886' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:31:29.880 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:29 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1075894352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:29.885 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-26T08:31:21.819689+0000', 'modified': '2024-03-26T08:31:26.907931+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34559}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34559': {'gid': 34559, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.52:6839/2551965101', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6838', 'nonce': 2551965101}, {'type': 'v1', 'addr': '172.21.15.52:6839', 'nonce': 2551965101}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 34559, 'qdb_cluster': [34559]} 2024-03-26T08:31:29.885 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-26T08:31:29.885 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-26T08:31:30.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.123+0000 7f2cc2400640 1 -- 172.21.15.43:0/776194802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cbc076060 msgr2=0x7f2cbc076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:30.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.123+0000 7f2cc2400640 1 --2- 172.21.15.43:0/776194802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cbc076060 0x7f2cbc076480 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2cac009900 tx=0x7f2cac02bc30 comp rx=0 tx=0).stop 2024-03-26T08:31:30.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.123+0000 7f2cc2400640 1 -- 172.21.15.43:0/776194802 shutdown_connections 2024-03-26T08:31:30.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.123+0000 7f2cc2400640 1 --2- 172.21.15.43:0/776194802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cbc077c40 0x7f2cbc0784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.123+0000 7f2cc2400640 1 --2- 172.21.15.43:0/776194802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2cbc0772a0 0x7f2cbc077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.123+0000 7f2cc2400640 1 --2- 172.21.15.43:0/776194802 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cbc076060 0x7f2cbc076480 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.123+0000 7f2cc2400640 1 -- 172.21.15.43:0/776194802 >> 172.21.15.43:0/776194802 conn(0x7f2cbc0fee50 msgr2=0x7f2cbc101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:30.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.123+0000 7f2cc2400640 1 -- 172.21.15.43:0/776194802 shutdown_connections 2024-03-26T08:31:30.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.123+0000 7f2cc2400640 1 -- 172.21.15.43:0/776194802 wait complete. 2024-03-26T08:31:30.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.124+0000 7f2cc2400640 1 Processor -- start 2024-03-26T08:31:30.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.124+0000 7f2cc2400640 1 -- start start 2024-03-26T08:31:30.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.125+0000 7f2cc2400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2cbc076060 0x7f2cbc19df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:30.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.125+0000 7f2cc2400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cbc0772a0 0x7f2cbc19e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:30.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.125+0000 7f2cc2400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cbc077c40 0x7f2cbc1a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:30.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.125+0000 7f2cc2400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc113860 con 0x7f2cbc0772a0 2024-03-26T08:31:30.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.125+0000 7f2cc2400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc1136e0 con 0x7f2cbc077c40 2024-03-26T08:31:30.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.125+0000 7f2cc2400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc1139e0 con 0x7f2cbc076060 2024-03-26T08:31:30.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.125+0000 7f2cbaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cbc0772a0 0x7f2cbc19e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:30.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.125+0000 7f2cbaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cbc0772a0 0x7f2cbc19e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53706/0 (socket says 172.21.15.43:53706) 2024-03-26T08:31:30.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.125+0000 7f2cbaa00640 1 -- 172.21.15.43:0/1445886594 learned_addr learned my addr 172.21.15.43:0/1445886594 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:30.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.125+0000 7f2cbb400640 1 --2- 172.21.15.43:0/1445886594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2cbc076060 0x7f2cbc19df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:30.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.125+0000 7f2cbbe00640 1 --2- 172.21.15.43:0/1445886594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cbc077c40 0x7f2cbc1a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:30.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.126+0000 7f2cbaa00640 1 -- 172.21.15.43:0/1445886594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2cbc076060 msgr2=0x7f2cbc19df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:30.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.126+0000 7f2cbaa00640 1 --2- 172.21.15.43:0/1445886594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2cbc076060 0x7f2cbc19df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.126+0000 7f2cbaa00640 1 -- 172.21.15.43:0/1445886594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cbc077c40 msgr2=0x7f2cbc1a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:30.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.126+0000 7f2cbaa00640 1 --2- 172.21.15.43:0/1445886594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cbc077c40 0x7f2cbc1a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.126+0000 7f2cbaa00640 1 -- 172.21.15.43:0/1445886594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc1a5b30 con 0x7f2cbc0772a0 2024-03-26T08:31:30.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.126+0000 7f2cbaa00640 1 --2- 172.21.15.43:0/1445886594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cbc0772a0 0x7f2cbc19e450 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400d770 tx=0x7f2ca400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:30.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.126+0000 7f2cb3e00640 1 -- 172.21.15.43:0/1445886594 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4017070 con 0x7f2cbc0772a0 2024-03-26T08:31:30.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.126+0000 7f2cb3e00640 1 -- 172.21.15.43:0/1445886594 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2ca40054d0 con 0x7f2cbc0772a0 2024-03-26T08:31:30.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.126+0000 7f2cb3e00640 1 -- 172.21.15.43:0/1445886594 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4004cf0 con 0x7f2cbc0772a0 2024-03-26T08:31:30.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.127+0000 7f2cc2400640 1 -- 172.21.15.43:0/1445886594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cbc1a5e20 con 0x7f2cbc0772a0 2024-03-26T08:31:30.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.127+0000 7f2cc2400640 1 -- 172.21.15.43:0/1445886594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cbc1a6290 con 0x7f2cbc0772a0 2024-03-26T08:31:30.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.131+0000 7f2cc519e740 1 Processor -- start 2024-03-26T08:31:30.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.131+0000 7f2cc519e740 1 -- start start 2024-03-26T08:31:30.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.132+0000 7f2cb3e00640 1 -- 172.21.15.43:0/1445886594 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f2ca4004e90 con 0x7f2cbc0772a0 2024-03-26T08:31:30.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.132+0000 7f2cc519e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d35563bba0 0x55d35563bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:30.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.132+0000 7f2cc519e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d35563c590 0x55d355641090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:30.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.132+0000 7f2cc519e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d355641700 0x55d355643b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:30.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.132+0000 7f2cc519e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3555a0c90 con 0x55d35563bba0 2024-03-26T08:31:30.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.132+0000 7f2cc519e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d355549980 con 0x55d35563c590 2024-03-26T08:31:30.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.132+0000 7f2cc519e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35559cd30 con 0x55d355641700 2024-03-26T08:31:30.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.132+0000 7f2cbbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d35563bba0 0x55d35563bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:30.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.132+0000 7f2cbbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d35563bba0 0x55d35563bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53718/0 (socket says 172.21.15.43:53718) 2024-03-26T08:31:30.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.132+0000 7f2cbbe00640 1 -- 172.21.15.43:0/56157882 learned_addr learned my addr 172.21.15.43:0/56157882 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:30.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.132+0000 7f2cbb400640 1 --2- 172.21.15.43:0/56157882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d35563c590 0x55d355641090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:30.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.132+0000 7f2cbaa00640 1 --2- 172.21.15.43:0/56157882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d355641700 0x55d355643b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:30.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.133+0000 7f2cbbe00640 1 -- 172.21.15.43:0/56157882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d355641700 msgr2=0x55d355643b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:31:30.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.133+0000 7f2cbbe00640 1 --2- 172.21.15.43:0/56157882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d355641700 0x55d355643b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.133+0000 7f2cbbe00640 1 -- 172.21.15.43:0/56157882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d35563c590 msgr2=0x55d355641090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:31:30.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.133+0000 7f2cbbe00640 1 --2- 172.21.15.43:0/56157882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d35563c590 0x55d355641090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.133+0000 7f2cbbe00640 1 -- 172.21.15.43:0/56157882 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d355561510 con 0x55d35563bba0 2024-03-26T08:31:30.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.133+0000 7f2cbbe00640 1 --2- 172.21.15.43:0/56157882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d35563bba0 0x55d35563bfc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4010370 tx=0x7f2cb4010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d79627be27db4cb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:30.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.133+0000 7f2cb0c00640 1 -- 172.21.15.43:0/56157882 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4016070 con 0x55d35563bba0 2024-03-26T08:31:30.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.133+0000 7f2cb0c00640 1 -- 172.21.15.43:0/56157882 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2cb4004d20 con 0x55d35563bba0 2024-03-26T08:31:30.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.133+0000 7f2cb0c00640 1 -- 172.21.15.43:0/56157882 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4005310 con 0x55d35563bba0 2024-03-26T08:31:30.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.134+0000 7f2cb3e00640 1 --2- 172.21.15.43:0/1445886594 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2ca00a2880 0x7f2ca00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:30.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.134+0000 7f2cb3e00640 1 -- 172.21.15.43:0/1445886594 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f2ca40ce600 con 0x7f2cbc0772a0 2024-03-26T08:31:30.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.134+0000 7f2cbb400640 1 --2- 172.21.15.43:0/1445886594 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2ca00a2880 0x7f2ca00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:30.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.134+0000 7f2cc519e740 1 -- 172.21.15.43:0/56157882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d35563bba0 msgr2=0x55d35563bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:30.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.134+0000 7f2cc519e740 1 --2- 172.21.15.43:0/56157882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d35563bba0 0x55d35563bfc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4010370 tx=0x7f2cb4010830 comp rx=0 tx=0).stop 2024-03-26T08:31:30.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.134+0000 7f2cc519e740 1 -- 172.21.15.43:0/56157882 shutdown_connections 2024-03-26T08:31:30.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.134+0000 7f2cc519e740 1 --2- 172.21.15.43:0/56157882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d355641700 0x55d355643b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.134+0000 7f2cc519e740 1 --2- 172.21.15.43:0/56157882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d35563c590 0x55d355641090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.134+0000 7f2cc519e740 1 --2- 172.21.15.43:0/56157882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d35563bba0 0x55d35563bfc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.134+0000 7f2cc519e740 1 -- 172.21.15.43:0/56157882 >> 172.21.15.43:0/56157882 conn(0x55d355637110 msgr2=0x55d355637510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:30.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.134+0000 7f2cc519e740 1 -- 172.21.15.43:0/56157882 shutdown_connections 2024-03-26T08:31:30.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.135+0000 7f2cc519e740 1 -- 172.21.15.43:0/56157882 wait complete. 2024-03-26T08:31:30.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.135+0000 7f2cbb400640 1 --2- 172.21.15.43:0/1445886594 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2ca00a2880 0x7f2ca00a4d40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2cac000c00 tx=0x7f2cac02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:30.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.135+0000 7f2cc519e740 1 -- start start 2024-03-26T08:31:30.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.135+0000 7f2cc519e740 1 Processor -- start 2024-03-26T08:31:30.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.136+0000 7f2cc519e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d35563bba0 0x55d355646060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:30.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.136+0000 7f2cc519e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d35563c590 0x55d3556465a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:30.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.136+0000 7f2cc519e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d355641700 0x55d35564d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:30.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.136+0000 7f2cc519e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3556446b0 con 0x55d35563c590 2024-03-26T08:31:30.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.136+0000 7f2cc519e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d35555cdd0 con 0x55d35563bba0 2024-03-26T08:31:30.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.136+0000 7f2cc519e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d355644830 con 0x55d355641700 2024-03-26T08:31:30.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.136+0000 7f2cc519e740 10 client.0 _ll_register_callbacks cb 0x55d3552b4ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:31:30.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.136+0000 7f2cbaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d35563c590 0x55d3556465a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:30.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.136+0000 7f2cbaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d35563c590 0x55d3556465a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53726/0 (socket says 172.21.15.43:53726) 2024-03-26T08:31:30.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.136+0000 7f2cbaa00640 1 -- 172.21.15.43:0/1598531100 learned_addr learned my addr 172.21.15.43:0/1598531100 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:30.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.136+0000 7f2cbb400640 1 --2- 172.21.15.43:0/1598531100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d35563bba0 0x55d355646060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:30.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.136+0000 7f2cbbe00640 1 --2- 172.21.15.43:0/1598531100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d355641700 0x55d35564d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:30.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.137+0000 7f2cbaa00640 1 -- 172.21.15.43:0/1598531100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d355641700 msgr2=0x55d35564d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:31:30.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.137+0000 7f2cbaa00640 1 --2- 172.21.15.43:0/1598531100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d355641700 0x55d35564d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.137+0000 7f2cbaa00640 1 -- 172.21.15.43:0/1598531100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d35563bba0 msgr2=0x55d355646060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:31:30.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.137+0000 7f2cbaa00640 1 --2- 172.21.15.43:0/1598531100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d35563bba0 0x55d355646060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.137+0000 7f2cbaa00640 1 -- 172.21.15.43:0/1598531100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3552b52d0 con 0x55d35563c590 2024-03-26T08:31:30.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.137+0000 7f2cbaa00640 1 --2- 172.21.15.43:0/1598531100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d35563c590 0x55d3556465a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4097210 tx=0x7f2ca40d1f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263d5e1774e3e3d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:30.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.137+0000 7f2ca9600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-26T08:31:30.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.137+0000 7f2ca9600640 1 -- 172.21.15.43:0/1598531100 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4020070 con 0x55d35563c590 2024-03-26T08:31:30.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.137+0000 7f2ca9600640 1 -- 172.21.15.43:0/1598531100 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f2ca409a200 con 0x55d35563c590 2024-03-26T08:31:30.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.137+0000 7f2cc519e740 1 -- 172.21.15.43:0/1598531100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2cb4004d20 con 0x55d35563c590 2024-03-26T08:31:30.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.137+0000 7f2ca9600640 1 -- 172.21.15.43:0/1598531100 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ca409a8f0 con 0x55d35563c590 2024-03-26T08:31:30.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.137+0000 7f2cc519e740 10 client.14790 fetch_fsmap learned FSMap version 17 2024-03-26T08:31:30.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.137+0000 7f2cc519e740 1 -- 172.21.15.43:0/1598531100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55d35537f6b0 con 0x55d35563c590 2024-03-26T08:31:30.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.138+0000 7f2ca9600640 1 -- 172.21.15.43:0/1598531100 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2cb4004d20 con 0x55d35563c590 2024-03-26T08:31:30.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.138+0000 7f2ca9600640 1 -- 172.21.15.43:0/1598531100 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 17) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f2ca40d0ac0 con 0x55d35563c590 2024-03-26T08:31:30.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.138+0000 7f2cc519e740 10 client.14790 fetch_fsmap finished waiting for FSMap version 17 2024-03-26T08:31:30.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.138+0000 7f2cc519e740 10 client.14790 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T08:31:30.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.138+0000 7f2cc519e740 20 client.14790 populate_metadata read hostname 'smithi043' 2024-03-26T08:31:30.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.138+0000 7f2cc519e740 1 --2- 172.21.15.43:0/1598531100 >> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] conn(0x55d355652880 0x55d355651920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:30.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.138+0000 7f2cc519e740 4 client.14790 mds_command: new command op to 34559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:31:30.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.138+0000 7f2cc519e740 1 -- 172.21.15.43:0/1598531100 --> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d355542520 con 0x55d355652880 2024-03-26T08:31:30.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.139+0000 7f2cbb400640 1 --2- 172.21.15.43:0/1598531100 >> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] conn(0x55d355652880 0x55d355651920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:30.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.140+0000 7f2cbb400640 1 --2- 172.21.15.43:0/1598531100 >> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] conn(0x55d355652880 0x55d355651920 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e3778067e7662cd server_cookie=4e65e7e4cf21119 in_seq=0 out_seq=0 2024-03-26T08:31:30.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.140+0000 7f2ca9600640 10 client.14790 ms_handle_connect on v2:172.21.15.52:6838/2551965101 2024-03-26T08:31:30.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.144+0000 7f2ca9600640 1 -- 172.21.15.43:0/1598531100 <== mds.0 v2:172.21.15.52:6838/2551965101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d355542520 con 0x55d355652880 2024-03-26T08:31:30.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.144+0000 7f2ca9600640 10 client.14790 handle_command_reply: tid=0 2024-03-26T08:31:30.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.144+0000 7f2cc519e740 1 client.14790 shutdown 2024-03-26T08:31:30.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.144+0000 7f2cc519e740 1 -- 172.21.15.43:0/1598531100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d35563c590 msgr2=0x55d3556465a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:30.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.144+0000 7f2cc519e740 1 --2- 172.21.15.43:0/1598531100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d35563c590 0x55d3556465a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4097210 tx=0x7f2ca40d1f80 comp rx=0 tx=0).stop 2024-03-26T08:31:30.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.144+0000 7f2cc519e740 1 -- 172.21.15.43:0/1598531100 shutdown_connections 2024-03-26T08:31:30.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.144+0000 7f2cc519e740 1 --2- 172.21.15.43:0/1598531100 >> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] conn(0x55d355652880 0x55d355651920 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.144+0000 7f2cc519e740 1 --2- 172.21.15.43:0/1598531100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d355641700 0x55d35564d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.144+0000 7f2cc519e740 1 --2- 172.21.15.43:0/1598531100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d35563c590 0x55d3556465a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.144+0000 7f2ca9600640 0 client.14790 ms_handle_reset on v2:172.21.15.52:6838/2551965101 2024-03-26T08:31:30.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.144+0000 7f2cc519e740 1 --2- 172.21.15.43:0/1598531100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d35563bba0 0x55d355646060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.144+0000 7f2cc519e740 1 -- 172.21.15.43:0/1598531100 >> 172.21.15.43:0/1598531100 conn(0x55d355637110 msgr2=0x55d355636310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:30.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.144+0000 7f2cc519e740 1 -- 172.21.15.43:0/1598531100 shutdown_connections 2024-03-26T08:31:30.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.144+0000 7f2cc519e740 1 -- 172.21.15.43:0/1598531100 wait complete. 2024-03-26T08:31:30.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.144+0000 7f2cc519e740 20 client.14790 trim_cache size 0 max 16384 2024-03-26T08:31:30.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.163+0000 7f2cc519e740 1 Processor -- start 2024-03-26T08:31:30.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.163+0000 7f2cc519e740 1 -- start start 2024-03-26T08:31:30.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.163+0000 7f2cc519e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d35563bba0 0x55d3556374f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:30.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.163+0000 7f2cc519e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d35563c590 0x55d35564d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:30.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.163+0000 7f2cc519e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d355641700 0x55d355646060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:30.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.163+0000 7f2cc519e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3556520a0 con 0x55d355641700 2024-03-26T08:31:30.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.163+0000 7f2cc519e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d355652700 con 0x55d35563c590 2024-03-26T08:31:30.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.163+0000 7f2cc519e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d355652d60 con 0x55d35563bba0 2024-03-26T08:31:30.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.164+0000 7f2cbbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d355641700 0x55d355646060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:30.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.164+0000 7f2cbbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d355641700 0x55d355646060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53738/0 (socket says 172.21.15.43:53738) 2024-03-26T08:31:30.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.164+0000 7f2cbbe00640 1 -- 172.21.15.43:0/2263089097 learned_addr learned my addr 172.21.15.43:0/2263089097 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:30.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.164+0000 7f2cbaa00640 1 --2- 172.21.15.43:0/2263089097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d35563c590 0x55d35564d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:30.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.164+0000 7f2cbb400640 1 --2- 172.21.15.43:0/2263089097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d35563bba0 0x55d3556374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:30.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.164+0000 7f2cbbe00640 1 -- 172.21.15.43:0/2263089097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d35563bba0 msgr2=0x55d3556374f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:31:30.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.164+0000 7f2cbbe00640 1 --2- 172.21.15.43:0/2263089097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d35563bba0 0x55d3556374f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.164+0000 7f2cbbe00640 1 -- 172.21.15.43:0/2263089097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d35563c590 msgr2=0x55d35564d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:31:30.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.164+0000 7f2cbbe00640 1 --2- 172.21.15.43:0/2263089097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d35563c590 0x55d35564d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.164+0000 7f2cbbe00640 1 -- 172.21.15.43:0/2263089097 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d35563b990 con 0x55d355641700 2024-03-26T08:31:30.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.164+0000 7f2cbaa00640 1 --2- 172.21.15.43:0/2263089097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d35563c590 0x55d35564d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:31:30.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.165+0000 7f2cbbe00640 1 --2- 172.21.15.43:0/2263089097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d355641700 0x55d355646060 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4010370 tx=0x7f2cb4005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23cbf211b5fe68e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:30.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.165+0000 7f2cabe00640 1 -- 172.21.15.43:0/2263089097 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4016030 con 0x55d355641700 2024-03-26T08:31:30.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.165+0000 7f2cabe00640 1 -- 172.21.15.43:0/2263089097 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2cb4003b70 con 0x55d355641700 2024-03-26T08:31:30.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.165+0000 7f2cabe00640 1 -- 172.21.15.43:0/2263089097 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4012710 con 0x55d355641700 2024-03-26T08:31:30.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.166+0000 7f2cc519e740 1 -- 172.21.15.43:0/2263089097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d355641700 msgr2=0x55d355646060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:30.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.166+0000 7f2cc519e740 1 --2- 172.21.15.43:0/2263089097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d355641700 0x55d355646060 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4010370 tx=0x7f2cb4005040 comp rx=0 tx=0).stop 2024-03-26T08:31:30.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.166+0000 7f2cc519e740 1 -- 172.21.15.43:0/2263089097 shutdown_connections 2024-03-26T08:31:30.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.166+0000 7f2cc519e740 1 --2- 172.21.15.43:0/2263089097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d355641700 0x55d355646060 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.166+0000 7f2cc519e740 1 --2- 172.21.15.43:0/2263089097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d35563c590 0x55d35564d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.166+0000 7f2cc519e740 1 --2- 172.21.15.43:0/2263089097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d35563bba0 0x55d3556374f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.166+0000 7f2cc519e740 1 -- 172.21.15.43:0/2263089097 >> 172.21.15.43:0/2263089097 conn(0x55d355637110 msgr2=0x55d355631e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:30.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.166+0000 7f2cc519e740 1 -- 172.21.15.43:0/2263089097 shutdown_connections 2024-03-26T08:31:30.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.166+0000 7f2cc519e740 1 -- 172.21.15.43:0/2263089097 wait complete. 2024-03-26T08:31:30.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.166+0000 7f2cc519e740 1 -- start start 2024-03-26T08:31:30.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.166+0000 7f2cc519e740 1 Processor -- start 2024-03-26T08:31:30.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.167+0000 7f2cc519e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d35563bba0 0x55d35564d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:30.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.167+0000 7f2cc519e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d35563c590 0x55d355649f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:30.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.167+0000 7f2cc519e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d355641700 0x55d35564a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:30.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.167+0000 7f2cc519e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d355653a20 con 0x55d35563bba0 2024-03-26T08:31:30.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.167+0000 7f2cc519e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3556533c0 con 0x55d35563c590 2024-03-26T08:31:30.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.167+0000 7f2cc519e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d355654080 con 0x55d355641700 2024-03-26T08:31:30.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.167+0000 7f2cc519e740 10 client.0 _ll_register_callbacks cb 0x7f2ca400d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:31:30.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.167+0000 7f2cbb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d35563bba0 0x55d35564d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:30.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.167+0000 7f2cbb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d35563bba0 0x55d35564d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53750/0 (socket says 172.21.15.43:53750) 2024-03-26T08:31:30.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.167+0000 7f2cbb400640 1 -- 172.21.15.43:0/2159326057 learned_addr learned my addr 172.21.15.43:0/2159326057 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:30.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.167+0000 7f2cbbe00640 1 --2- 172.21.15.43:0/2159326057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d355641700 0x55d35564a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:30.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.167+0000 7f2cbaa00640 1 --2- 172.21.15.43:0/2159326057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d35563c590 0x55d355649f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:30.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.168+0000 7f2cbb400640 1 -- 172.21.15.43:0/2159326057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d355641700 msgr2=0x55d35564a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:31:30.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.168+0000 7f2cbb400640 1 --2- 172.21.15.43:0/2159326057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d355641700 0x55d35564a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.168+0000 7f2cbb400640 1 -- 172.21.15.43:0/2159326057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d35563c590 msgr2=0x55d355649f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:31:30.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.168+0000 7f2cbb400640 1 --2- 172.21.15.43:0/2159326057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d35563c590 0x55d355649f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.168+0000 7f2cbb400640 1 -- 172.21.15.43:0/2159326057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3556ad8e0 con 0x55d35563bba0 2024-03-26T08:31:30.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.168+0000 7f2cbaa00640 1 --2- 172.21.15.43:0/2159326057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d35563c590 0x55d355649f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:31:30.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.168+0000 7f2cbb400640 1 --2- 172.21.15.43:0/2159326057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d35563bba0 0x55d35564d490 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2cac02c680 tx=0x7f2cac009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74319e197a407f4d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:30.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.168+0000 7f2ca9600640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-26T08:31:30.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.168+0000 7f2ca9600640 1 -- 172.21.15.43:0/2159326057 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac058070 con 0x55d35563bba0 2024-03-26T08:31:30.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.168+0000 7f2cc519e740 1 -- 172.21.15.43:0/2159326057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2cb4003b70 con 0x55d35563bba0 2024-03-26T08:31:30.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.168+0000 7f2ca9600640 1 -- 172.21.15.43:0/2159326057 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f2cac005040 con 0x55d35563bba0 2024-03-26T08:31:30.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.169+0000 7f2ca9600640 1 -- 172.21.15.43:0/2159326057 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cac005240 con 0x55d35563bba0 2024-03-26T08:31:30.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.169+0000 7f2cc519e740 10 client.14796 fetch_fsmap learned FSMap version 17 2024-03-26T08:31:30.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.169+0000 7f2cc519e740 1 -- 172.21.15.43:0/2159326057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55d3556aebf0 con 0x55d35563bba0 2024-03-26T08:31:30.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.169+0000 7f2ca9600640 1 -- 172.21.15.43:0/2159326057 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2cb4003b70 con 0x55d35563bba0 2024-03-26T08:31:30.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.169+0000 7f2ca9600640 1 -- 172.21.15.43:0/2159326057 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 17) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f2cac0035f0 con 0x55d35563bba0 2024-03-26T08:31:30.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.169+0000 7f2cc519e740 10 client.14796 fetch_fsmap finished waiting for FSMap version 17 2024-03-26T08:31:30.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.169+0000 7f2cc519e740 10 client.14796 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T08:31:30.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.169+0000 7f2cc519e740 20 client.14796 populate_metadata read hostname 'smithi043' 2024-03-26T08:31:30.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.169+0000 7f2cc519e740 1 --2- 172.21.15.43:0/2159326057 >> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] conn(0x55d3556b31d0 0x55d3556aee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:30.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.169+0000 7f2cc519e740 4 client.14796 mds_command: new command op to 34559 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T08:31:30.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.169+0000 7f2cc519e740 1 -- 172.21.15.43:0/2159326057 --> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d3555ae1d0 con 0x55d3556b31d0 2024-03-26T08:31:30.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.170+0000 7f2cbaa00640 1 --2- 172.21.15.43:0/2159326057 >> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] conn(0x55d3556b31d0 0x55d3556aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:30.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.171+0000 7f2cbaa00640 1 --2- 172.21.15.43:0/2159326057 >> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] conn(0x55d3556b31d0 0x55d3556aee80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5a59eb142177b53 server_cookie=5d5b3323a7899ac4 in_seq=0 out_seq=0 2024-03-26T08:31:30.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.171+0000 7f2ca9600640 10 client.14796 ms_handle_connect on v2:172.21.15.52:6838/2551965101 2024-03-26T08:31:30.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.171+0000 7f2ca9600640 1 -- 172.21.15.43:0/2159326057 <== mds.0 v2:172.21.15.52:6838/2551965101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55d3555ae1d0 con 0x55d3556b31d0 2024-03-26T08:31:30.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.171+0000 7f2ca9600640 10 client.14796 handle_command_reply: tid=0 2024-03-26T08:31:30.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.172+0000 7f2cc519e740 1 client.14796 shutdown 2024-03-26T08:31:30.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.172+0000 7f2cc519e740 1 -- 172.21.15.43:0/2159326057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d35563bba0 msgr2=0x55d35564d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:30.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.172+0000 7f2cc519e740 1 --2- 172.21.15.43:0/2159326057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d35563bba0 0x55d35564d490 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2cac02c680 tx=0x7f2cac009260 comp rx=0 tx=0).stop 2024-03-26T08:31:30.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.172+0000 7f2cc519e740 1 -- 172.21.15.43:0/2159326057 shutdown_connections 2024-03-26T08:31:30.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.172+0000 7f2cc519e740 1 --2- 172.21.15.43:0/2159326057 >> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] conn(0x55d3556b31d0 0x55d3556aee80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.172+0000 7f2cc519e740 1 --2- 172.21.15.43:0/2159326057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d355641700 0x55d35564a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.172+0000 7f2cc519e740 1 --2- 172.21.15.43:0/2159326057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d35563c590 0x55d355649f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.172+0000 7f2cc519e740 1 --2- 172.21.15.43:0/2159326057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d35563bba0 0x55d35564d490 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.172+0000 7f2ca9600640 0 client.14796 ms_handle_reset on v2:172.21.15.52:6838/2551965101 2024-03-26T08:31:30.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.172+0000 7f2cc519e740 1 -- 172.21.15.43:0/2159326057 >> 172.21.15.43:0/2159326057 conn(0x55d355637110 msgr2=0x55d3556374f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:30.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.172+0000 7f2cc519e740 1 -- 172.21.15.43:0/2159326057 shutdown_connections 2024-03-26T08:31:30.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.172+0000 7f2cc519e740 1 -- 172.21.15.43:0/2159326057 wait complete. 2024-03-26T08:31:30.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.172+0000 7f2cc519e740 20 client.14796 trim_cache size 0 max 16384 2024-03-26T08:31:30.168 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-26T08:31:30.168 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "88759420-eb49-11ee-95d0-87774f69a715", 2024-03-26T08:31:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-03-26T08:31:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 34559, 2024-03-26T08:31:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-03-26T08:31:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-03-26T08:31:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-03-26T08:31:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 7.7986941779999999, 2024-03-26T08:31:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 17, 2024-03-26T08:31:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 90, 2024-03-26T08:31:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 90, 2024-03-26T08:31:30.169 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 15.980568226000001 2024-03-26T08:31:30.169 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-26T08:31:30.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.173+0000 7f2cc2400640 1 -- 172.21.15.43:0/1445886594 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2ca00a2880 msgr2=0x7f2ca00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:30.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.173+0000 7f2cc2400640 1 --2- 172.21.15.43:0/1445886594 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2ca00a2880 0x7f2ca00a4d40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2cac000c00 tx=0x7f2cac02c7e0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.173+0000 7f2cc2400640 1 -- 172.21.15.43:0/1445886594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cbc0772a0 msgr2=0x7f2cbc19e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:30.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.173+0000 7f2cc2400640 1 --2- 172.21.15.43:0/1445886594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cbc0772a0 0x7f2cbc19e450 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400d770 tx=0x7f2ca400dc30 comp rx=0 tx=0).stop 2024-03-26T08:31:30.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.173+0000 7f2cc2400640 1 -- 172.21.15.43:0/1445886594 shutdown_connections 2024-03-26T08:31:30.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.173+0000 7f2cc2400640 1 --2- 172.21.15.43:0/1445886594 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2ca00a2880 0x7f2ca00a4d40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.173+0000 7f2cc2400640 1 --2- 172.21.15.43:0/1445886594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2cbc077c40 0x7f2cbc1a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.173+0000 7f2cc2400640 1 --2- 172.21.15.43:0/1445886594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cbc0772a0 0x7f2cbc19e450 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.173+0000 7f2cc2400640 1 --2- 172.21.15.43:0/1445886594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2cbc076060 0x7f2cbc19df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:30.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.173+0000 7f2cc2400640 1 -- 172.21.15.43:0/1445886594 >> 172.21.15.43:0/1445886594 conn(0x7f2cbc0fee50 msgr2=0x7f2cbc100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:30.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.173+0000 7f2cc2400640 1 -- 172.21.15.43:0/1445886594 shutdown_connections 2024-03-26T08:31:30.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:30.173+0000 7f2cc2400640 1 -- 172.21.15.43:0/1445886594 wait complete. 2024-03-26T08:31:30.183 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-26T08:31:30.183 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-26T08:31:30.184 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph fs ls 2024-03-26T08:31:30.634 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:31:30.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:30 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/685957686' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:31:30.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:30 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/685957686' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:31:30.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:30 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/685957686' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:31:31.756 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:31 smithi043 ceph-mon[31929]: pgmap v377: 129 pgs: 10 creating+peering, 119 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 690 B/s wr, 2 op/s 2024-03-26T08:31:31.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:31 smithi183 ceph-mon[35818]: pgmap v377: 129 pgs: 10 creating+peering, 119 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 690 B/s wr, 2 op/s 2024-03-26T08:31:31.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:31 smithi052 ceph-mon[35071]: pgmap v377: 129 pgs: 10 creating+peering, 119 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 690 B/s wr, 2 op/s 2024-03-26T08:31:33.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.199+0000 7ff089e00700 1 -- 172.21.15.43:0/2247188520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff084109160 msgr2=0x7ff084115670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:33.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.199+0000 7ff089e00700 1 --2- 172.21.15.43:0/2247188520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff084109160 0x7ff084115670 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00b0a0 tx=0x7ff07c00b3b0 comp rx=0 tx=0).stop 2024-03-26T08:31:33.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.199+0000 7ff089e00700 1 -- 172.21.15.43:0/2247188520 shutdown_connections 2024-03-26T08:31:33.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.199+0000 7ff089e00700 1 --2- 172.21.15.43:0/2247188520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff084109160 0x7ff084115670 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:33.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.199+0000 7ff089e00700 1 --2- 172.21.15.43:0/2247188520 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff08410e7a0 0x7ff08410eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:33.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.199+0000 7ff089e00700 1 --2- 172.21.15.43:0/2247188520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0841087a0 0x7ff084108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:33.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.199+0000 7ff089e00700 1 -- 172.21.15.43:0/2247188520 >> 172.21.15.43:0/2247188520 conn(0x7ff084075960 msgr2=0x7ff084077d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:33.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.199+0000 7ff089e00700 1 -- 172.21.15.43:0/2247188520 shutdown_connections 2024-03-26T08:31:33.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.199+0000 7ff089e00700 1 -- 172.21.15.43:0/2247188520 wait complete. 2024-03-26T08:31:33.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.201+0000 7ff089e00700 1 Processor -- start 2024-03-26T08:31:33.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.201+0000 7ff089e00700 1 -- start start 2024-03-26T08:31:33.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.202+0000 7ff089e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0841087a0 0x7ff08419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:33.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.202+0000 7ff089e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff084109160 0x7ff08419f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:33.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.202+0000 7ff089e00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff08410e7a0 0x7ff0841a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:33.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.202+0000 7ff089e00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff084118a30 con 0x7ff0841087a0 2024-03-26T08:31:33.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.202+0000 7ff089e00700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0841188b0 con 0x7ff08410e7a0 2024-03-26T08:31:33.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.202+0000 7ff089e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff084118bb0 con 0x7ff084109160 2024-03-26T08:31:33.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.202+0000 7ff082a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0841087a0 0x7ff08419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:33.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.202+0000 7ff082a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0841087a0 0x7ff08419ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53770/0 (socket says 172.21.15.43:53770) 2024-03-26T08:31:33.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.202+0000 7ff082a00700 1 -- 172.21.15.43:0/2906619208 learned_addr learned my addr 172.21.15.43:0/2906619208 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:33.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.202+0000 7ff082000700 1 --2- 172.21.15.43:0/2906619208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff084109160 0x7ff08419f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:33.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.202+0000 7ff083400700 1 --2- 172.21.15.43:0/2906619208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff08410e7a0 0x7ff0841a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:33.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.203+0000 7ff082a00700 1 -- 172.21.15.43:0/2906619208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff084109160 msgr2=0x7ff08419f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:33.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.203+0000 7ff082a00700 1 --2- 172.21.15.43:0/2906619208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff084109160 0x7ff08419f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:33.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.203+0000 7ff082a00700 1 -- 172.21.15.43:0/2906619208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff08410e7a0 msgr2=0x7ff0841a3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:33.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.203+0000 7ff082a00700 1 --2- 172.21.15.43:0/2906619208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff08410e7a0 0x7ff0841a3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:33.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.203+0000 7ff082a00700 1 -- 172.21.15.43:0/2906619208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0841a3d70 con 0x7ff0841087a0 2024-03-26T08:31:33.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.203+0000 7ff082a00700 1 --2- 172.21.15.43:0/2906619208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0841087a0 0x7ff08419ed60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff07400c920 tx=0x7ff07400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:33.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.203+0000 7ff088a00700 1 -- 172.21.15.43:0/2906619208 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff074007b00 con 0x7ff0841087a0 2024-03-26T08:31:33.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.203+0000 7ff088a00700 1 -- 172.21.15.43:0/2906619208 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff074004ce0 con 0x7ff0841087a0 2024-03-26T08:31:33.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.203+0000 7ff089e00700 1 -- 172.21.15.43:0/2906619208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0841a4060 con 0x7ff0841087a0 2024-03-26T08:31:33.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.203+0000 7ff088a00700 1 -- 172.21.15.43:0/2906619208 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff074005600 con 0x7ff0841087a0 2024-03-26T08:31:33.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.203+0000 7ff089e00700 1 -- 172.21.15.43:0/2906619208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0841ab8e0 con 0x7ff0841087a0 2024-03-26T08:31:33.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.204+0000 7ff089e00700 1 -- 172.21.15.43:0/2906619208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff084068e50 con 0x7ff0841087a0 2024-03-26T08:31:33.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.207+0000 7ff088a00700 1 -- 172.21.15.43:0/2906619208 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff074007ca0 con 0x7ff0841087a0 2024-03-26T08:31:33.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.208+0000 7ff088a00700 1 --2- 172.21.15.43:0/2906619208 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff0680a2910 0x7ff0680a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:33.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.208+0000 7ff088a00700 1 -- 172.21.15.43:0/2906619208 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff0740ce870 con 0x7ff0841087a0 2024-03-26T08:31:33.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.208+0000 7ff082000700 1 --2- 172.21.15.43:0/2906619208 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff0680a2910 0x7ff0680a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:33.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.209+0000 7ff082000700 1 --2- 172.21.15.43:0/2906619208 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff0680a2910 0x7ff0680a4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff070005ea0 tx=0x7ff070005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:33.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.213+0000 7ff088a00700 1 -- 172.21.15.43:0/2906619208 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff074014030 con 0x7ff0841087a0 2024-03-26T08:31:33.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.495+0000 7ff089e00700 1 -- 172.21.15.43:0/2906619208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff0841abc60 con 0x7ff0841087a0 2024-03-26T08:31:33.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:33 smithi043 ceph-mon[31929]: pgmap v378: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-26T08:31:33.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.497+0000 7ff088a00700 1 -- 172.21.15.43:0/2906619208 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7ff07409b670 con 0x7ff0841087a0 2024-03-26T08:31:33.490 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-26T08:31:33.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.500+0000 7ff089e00700 1 -- 172.21.15.43:0/2906619208 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff0680a2910 msgr2=0x7ff0680a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:33.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.500+0000 7ff089e00700 1 --2- 172.21.15.43:0/2906619208 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff0680a2910 0x7ff0680a4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff070005ea0 tx=0x7ff070005e30 comp rx=0 tx=0).stop 2024-03-26T08:31:33.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.500+0000 7ff089e00700 1 -- 172.21.15.43:0/2906619208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0841087a0 msgr2=0x7ff08419ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:33.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.500+0000 7ff089e00700 1 --2- 172.21.15.43:0/2906619208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0841087a0 0x7ff08419ed60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff07400c920 tx=0x7ff07400cce0 comp rx=0 tx=0).stop 2024-03-26T08:31:33.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.501+0000 7ff089e00700 1 -- 172.21.15.43:0/2906619208 shutdown_connections 2024-03-26T08:31:33.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.501+0000 7ff089e00700 1 --2- 172.21.15.43:0/2906619208 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff0680a2910 0x7ff0680a4dd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:33.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.501+0000 7ff089e00700 1 --2- 172.21.15.43:0/2906619208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff08410e7a0 0x7ff0841a3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:33.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.501+0000 7ff089e00700 1 --2- 172.21.15.43:0/2906619208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0841087a0 0x7ff08419ed60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:33.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.501+0000 7ff089e00700 1 --2- 172.21.15.43:0/2906619208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff084109160 0x7ff08419f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:33.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.501+0000 7ff089e00700 1 -- 172.21.15.43:0/2906619208 >> 172.21.15.43:0/2906619208 conn(0x7ff084075960 msgr2=0x7ff084075de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:33.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.501+0000 7ff089e00700 1 -- 172.21.15.43:0/2906619208 shutdown_connections 2024-03-26T08:31:33.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:33.501+0000 7ff089e00700 1 -- 172.21.15.43:0/2906619208 wait complete. 2024-03-26T08:31:33.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:33 smithi183 ceph-mon[35818]: pgmap v378: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-26T08:31:33.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:33 smithi052 ceph-mon[35071]: pgmap v378: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-26T08:31:34.286 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-26T08:31:34.286 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-26T08:31:34.286 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-03-26T08:31:34.287 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-26T08:31:34.287 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-26T08:31:34.287 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-26T08:31:34.287 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-26T08:31:34.287 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-26T08:31:34.287 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-26T08:31:34.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:31:34.287 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-03-26T08:31:34.349 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-26T08:31:34.349 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-26T08:31:34.349 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-03-26T08:31:34.349 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-26T08:31:34.349 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-03-26T08:31:34.349 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-26T08:31:34.349 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T08:31:34.350 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-03-26T08:31:34.350 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-03-26T08:31:34.350 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T08:31:34.350 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-03-26T08:31:34.350 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-03-26T08:31:34.350 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-26T08:31:34.350 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-03-26T08:31:34.350 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-03-26T08:31:34.350 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-26T08:31:34.350 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 41929sec preferred_lft 41929sec 2024-03-26T08:31:34.350 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link noprefixroute 2024-03-26T08:31:34.350 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-26T08:31:34.350 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T08:31:34.351 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-03-26T08:31:34.351 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-03-26T08:31:34.352 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-26T08:31:34.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:31:34.352 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-03-26T08:31:34.352 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-03-26T08:31:34.352 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add name ceph-brx type bridge 2024-03-26T08:31:34.352 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr flush dev ceph-brx 2024-03-26T08:31:34.353 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set ceph-brx up 2024-03-26T08:31:34.353 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-26T08:31:34.353 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-03-26T08:31:34.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:34 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2906619208' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T08:31:34.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:34 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2906619208' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T08:31:34.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:34 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2906619208' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T08:31:34.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:31:34.842 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-26T08:31:34.926 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-03-26T08:31:34.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:31:34.930 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-03-26T08:31:34.988 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-03-26T08:31:34.989 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-26T08:31:34.989 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-26T08:31:34.989 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-26T08:31:34.989 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-26T08:31:34.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:31:34.990 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-03-26T08:31:34.990 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-03-26T08:31:34.990 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-26T08:31:34.990 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-26T08:31:34.991 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-26T08:31:34.991 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-03-26T08:31:35.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:35 smithi052 ceph-mon[35071]: pgmap v379: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:35.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:35 smithi043 ceph-mon[31929]: pgmap v379: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:35.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:31:35.716 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-03-26T08:31:35.745 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:35 smithi183 ceph-mon[35818]: pgmap v379: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:35.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:31:35.765 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list-id 2024-03-26T08:31:35.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:31:35.815 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-03-26T08:31:35.815 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-03-26T08:31:35.815 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-26T08:31:35.815 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-26T08:31:35.815 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-03-26T08:31:35.948 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-26T08:31:35.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:31:35.949 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-03-26T08:31:35.949 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-03-26T08:31:35.949 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-26T08:31:35.950 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-26T08:31:35.950 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-26T08:31:35.950 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-26T08:31:35.950 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-26T08:31:35.950 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-03-26T08:31:36.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:31:36.237 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-03-26T08:31:36.237 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-03-26T08:31:36.237 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-03-26T08:31:36.237 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set dev brx.0 master ceph-brx 2024-03-26T08:31:36.237 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-03-26T08:31:36.396 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-26T08:31:36.396 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-26T08:31:36.396 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-26T08:31:36.443 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-26T08:31:36.444 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-26T08:31:36.444 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-26T08:31:36.489 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-03-26T08:31:36.604 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:36 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-03-26T08:31:36.604 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:36 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-03-26T08:31:36.604 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:36 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-03-26T08:31:36.604 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:36 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-03-26T08:31:36.604 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:36 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 7]}]: dispatch 2024-03-26T08:31:36.604 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:36 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-03-26T08:31:36.604 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:36 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-03-26T08:31:36.605 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:36 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-03-26T08:31:36.605 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:36 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-03-26T08:31:36.620 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-03-26T08:31:36.664 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-03-26T08:31:36.664 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-03-26T08:31:36.664 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-03-26T08:31:36.664 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-03-26T08:31:36.664 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-03-26T08:31:36.664 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-03-26T08:31:36.664 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-03-26T08:31:36.664 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-03-26T08:31:36.664 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-03-26T08:31:36.664 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-03-26T08:31:36.664 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-03-26T08:31:36.664 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-03-26T08:31:36.665 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T08:31:36.665 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-03-26T08:31:36.665 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-03-26T08:31:36.665 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-03-26T08:31:36.665 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-03-26T08:31:36.665 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-03-26T08:31:36.665 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-03-26T08:31:36.665 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-03-26T08:31:36.665 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-26T08:31:36.665 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-26T08:31:36.665 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-03-26T08:31:36.665 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-03-26T08:31:36.665 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-03-26T08:31:36.665 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay 2024-03-26T08:31:36.665 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/11ba0c83ec55690d45669d206dc55ba30613139ea80436ae3cdd46b540253e72/merged 2024-03-26T08:31:36.666 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/97fe696f1e34509dc86041c67a396fdc77cda785578e668898e68d28eba150cf/merged 2024-03-26T08:31:36.666 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/0 2024-03-26T08:31:36.666 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T08:31:36.666 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/0a3b315194a3cd2e278c3c92d31fc70810e246de129c4a15e55d867020f0c057/merged 2024-03-26T08:31:36.666 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/46b0401bbe4775455adde9cb89058f287288315f170bda4d9d26af266da5aa21/merged 2024-03-26T08:31:36.666 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/5211fce819420e40b58fbf4975b87b4825d6372f9738c6605df2a392286e3f4b/merged 2024-03-26T08:31:36.666 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/6ad88ba55ea27f30c7b0ec77c84d2e5453e87a62bc39a75368ba826be2b1291d/merged 2024-03-26T08:31:36.666 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/d013144c20b550c06081475d3e46065f3f4d252dabc08af06aca254c47a85b50/merged 2024-03-26T08:31:36.666 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/33029eb686fc515d3063e2ff583bcbd0a8ab9c15101baea9ac8c066ce5d23744/merged 2024-03-26T08:31:36.666 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/165e9b1d609cd90aa0578b0fe457f616fabf6ef44c82e53a0fad1bea35209e21/merged 2024-03-26T08:31:36.666 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/24e73d122c85dd284aec81e66c7543cd2bba3cee3b499ae61f083629a49d29fe/merged 2024-03-26T08:31:36.666 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-03-26T08:31:36.666 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-26T08:31:36.666 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-26T08:31:36.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:31:36.668 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-03-26T08:31:36.750 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-26T08:31:36.750 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-26T08:31:36.798 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-03-26T08:31:36.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:36 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-03-26T08:31:36.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:36 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-03-26T08:31:36.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:36 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-03-26T08:31:36.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:36 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-03-26T08:31:36.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:36 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 7]}]: dispatch 2024-03-26T08:31:36.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:36 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-03-26T08:31:36.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:36 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-03-26T08:31:36.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:36 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-03-26T08:31:36.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:36 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-03-26T08:31:36.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:36 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-03-26T08:31:36.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:36 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-03-26T08:31:36.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:36 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-03-26T08:31:36.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:36 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-03-26T08:31:36.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:36 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 7]}]: dispatch 2024-03-26T08:31:36.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:36 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-03-26T08:31:36.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:36 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-03-26T08:31:36.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:36 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-03-26T08:31:36.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:36 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-03-26T08:31:36.880 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-03-26T08:31:36.918 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-03-26T08:31:36.919 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-03-26T08:31:36.919 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-03-26T08:31:36.919 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-03-26T08:31:36.919 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-03-26T08:31:36.919 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-03-26T08:31:36.919 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-03-26T08:31:36.919 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-03-26T08:31:36.919 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-03-26T08:31:36.919 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-03-26T08:31:36.919 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-03-26T08:31:36.919 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-03-26T08:31:36.919 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T08:31:36.919 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-03-26T08:31:36.920 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-03-26T08:31:36.920 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-03-26T08:31:36.920 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-03-26T08:31:36.920 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-03-26T08:31:36.920 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-03-26T08:31:36.920 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-03-26T08:31:36.920 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-26T08:31:36.920 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-26T08:31:36.920 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-03-26T08:31:36.920 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-03-26T08:31:36.920 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-03-26T08:31:36.920 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay 2024-03-26T08:31:36.920 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/11ba0c83ec55690d45669d206dc55ba30613139ea80436ae3cdd46b540253e72/merged 2024-03-26T08:31:36.920 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/97fe696f1e34509dc86041c67a396fdc77cda785578e668898e68d28eba150cf/merged 2024-03-26T08:31:36.920 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/0 2024-03-26T08:31:36.921 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T08:31:36.921 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/0a3b315194a3cd2e278c3c92d31fc70810e246de129c4a15e55d867020f0c057/merged 2024-03-26T08:31:36.921 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/46b0401bbe4775455adde9cb89058f287288315f170bda4d9d26af266da5aa21/merged 2024-03-26T08:31:36.921 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/5211fce819420e40b58fbf4975b87b4825d6372f9738c6605df2a392286e3f4b/merged 2024-03-26T08:31:36.921 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/6ad88ba55ea27f30c7b0ec77c84d2e5453e87a62bc39a75368ba826be2b1291d/merged 2024-03-26T08:31:36.921 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/d013144c20b550c06081475d3e46065f3f4d252dabc08af06aca254c47a85b50/merged 2024-03-26T08:31:36.921 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/33029eb686fc515d3063e2ff583bcbd0a8ab9c15101baea9ac8c066ce5d23744/merged 2024-03-26T08:31:36.921 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/165e9b1d609cd90aa0578b0fe457f616fabf6ef44c82e53a0fad1bea35209e21/merged 2024-03-26T08:31:36.921 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/24e73d122c85dd284aec81e66c7543cd2bba3cee3b499ae61f083629a49d29fe/merged 2024-03-26T08:31:36.921 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-03-26T08:31:36.921 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-26T08:31:36.921 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-26T08:31:36.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:31:36.922 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-03-26T08:31:37.005 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi043.stderr:2024-03-26T08:31:37.012+0000 7fb01343a480 -1 init, newargv = 0x55c023d620c0 newargc=15 2024-03-26T08:31:37.005 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi043.stderr:ceph-fuse[73320]: starting ceph client 2024-03-26T08:31:37.019 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi043.stderr:ceph-fuse[73320]: starting fuse 2024-03-26T08:31:37.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:37 smithi183 ceph-mon[35818]: pgmap v380: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:37.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:37 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-03-26T08:31:37.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:37 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-03-26T08:31:37.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:37 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-03-26T08:31:37.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:37 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-03-26T08:31:37.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:37 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 7]}]': finished 2024-03-26T08:31:37.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:37 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-03-26T08:31:37.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:37 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-03-26T08:31:37.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:37 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-03-26T08:31:37.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:37 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-03-26T08:31:37.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:37 smithi183 ceph-mon[35818]: osdmap e91: 12 total, 12 up, 12 in 2024-03-26T08:31:37.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:37 smithi052 ceph-mon[35071]: pgmap v380: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:37.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:37 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-03-26T08:31:37.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:37 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-03-26T08:31:37.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:37 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-03-26T08:31:37.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:37 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-03-26T08:31:37.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:37 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 7]}]': finished 2024-03-26T08:31:37.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:37 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-03-26T08:31:37.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:37 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-03-26T08:31:37.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:37 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-03-26T08:31:37.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:37 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-03-26T08:31:37.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:37 smithi052 ceph-mon[35071]: osdmap e91: 12 total, 12 up, 12 in 2024-03-26T08:31:37.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:37 smithi043 ceph-mon[31929]: pgmap v380: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:37.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:37 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-03-26T08:31:37.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:37 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-03-26T08:31:37.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:37 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-03-26T08:31:37.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:37 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-03-26T08:31:37.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:37 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9, 5, 7]}]': finished 2024-03-26T08:31:37.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:37 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-03-26T08:31:37.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:37 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-03-26T08:31:37.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:37 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-03-26T08:31:37.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:37 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-03-26T08:31:37.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:37 smithi043 ceph-mon[31929]: osdmap e91: 12 total, 12 up, 12 in 2024-03-26T08:31:38.113 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-03-26T08:31:38.228 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-03-26T08:31:38.283 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-03-26T08:31:38.284 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-03-26T08:31:38.284 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-03-26T08:31:38.284 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-03-26T08:31:38.284 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-03-26T08:31:38.284 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-03-26T08:31:38.284 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-03-26T08:31:38.284 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-03-26T08:31:38.284 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-03-26T08:31:38.284 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-03-26T08:31:38.284 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-03-26T08:31:38.284 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-03-26T08:31:38.284 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T08:31:38.284 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-03-26T08:31:38.284 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-03-26T08:31:38.284 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-03-26T08:31:38.285 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-03-26T08:31:38.285 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-03-26T08:31:38.285 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-03-26T08:31:38.285 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-03-26T08:31:38.285 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-26T08:31:38.285 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-26T08:31:38.285 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-03-26T08:31:38.285 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-03-26T08:31:38.285 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-03-26T08:31:38.285 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay 2024-03-26T08:31:38.285 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/11ba0c83ec55690d45669d206dc55ba30613139ea80436ae3cdd46b540253e72/merged 2024-03-26T08:31:38.285 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/97fe696f1e34509dc86041c67a396fdc77cda785578e668898e68d28eba150cf/merged 2024-03-26T08:31:38.285 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/0 2024-03-26T08:31:38.285 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T08:31:38.285 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/0a3b315194a3cd2e278c3c92d31fc70810e246de129c4a15e55d867020f0c057/merged 2024-03-26T08:31:38.286 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/46b0401bbe4775455adde9cb89058f287288315f170bda4d9d26af266da5aa21/merged 2024-03-26T08:31:38.286 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/5211fce819420e40b58fbf4975b87b4825d6372f9738c6605df2a392286e3f4b/merged 2024-03-26T08:31:38.286 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/6ad88ba55ea27f30c7b0ec77c84d2e5453e87a62bc39a75368ba826be2b1291d/merged 2024-03-26T08:31:38.286 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/d013144c20b550c06081475d3e46065f3f4d252dabc08af06aca254c47a85b50/merged 2024-03-26T08:31:38.286 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/33029eb686fc515d3063e2ff583bcbd0a8ab9c15101baea9ac8c066ce5d23744/merged 2024-03-26T08:31:38.286 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/165e9b1d609cd90aa0578b0fe457f616fabf6ef44c82e53a0fad1bea35209e21/merged 2024-03-26T08:31:38.286 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/24e73d122c85dd284aec81e66c7543cd2bba3cee3b499ae61f083629a49d29fe/merged 2024-03-26T08:31:38.286 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-03-26T08:31:38.286 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-26T08:31:38.286 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-26T08:31:38.286 INFO:teuthology.orchestra.run.smithi043.stdout:/home/ubuntu/cephtest/mnt.admin 2024-03-26T08:31:38.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:31:38.287 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-03-26T08:31:38.339 INFO:teuthology.orchestra.run.smithi043.stdout:118 2024-03-26T08:31:38.339 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [118] 2024-03-26T08:31:38.340 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2024-03-26T08:31:38.340 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2024-03-26T08:31:38.340 DEBUG:teuthology.orchestra.run.smithi043:> import re 2024-03-26T08:31:38.340 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-03-26T08:31:38.340 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2024-03-26T08:31:38.340 DEBUG:teuthology.orchestra.run.smithi043:> 2024-03-26T08:31:38.340 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2024-03-26T08:31:38.340 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-26T08:31:38.340 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2024-03-26T08:31:38.340 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-26T08:31:38.340 DEBUG:teuthology.orchestra.run.smithi043:> 2024-03-26T08:31:38.340 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2024-03-26T08:31:38.340 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2024-03-26T08:31:38.340 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2024-03-26T08:31:38.340 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2024-03-26T08:31:38.341 DEBUG:teuthology.orchestra.run.smithi043:> 2024-03-26T08:31:38.341 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2024-03-26T08:31:38.341 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-26T08:31:38.341 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-26T08:31:38.341 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-26T08:31:38.341 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2024-03-26T08:31:38.341 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2024-03-26T08:31:38.341 DEBUG:teuthology.orchestra.run.smithi043:> return f 2024-03-26T08:31:38.341 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-26T08:31:38.341 DEBUG:teuthology.orchestra.run.smithi043:> 2024-03-26T08:31:38.341 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.admin")) 2024-03-26T08:31:38.341 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-03-26T08:31:38.519 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.admin.73320.asok 2024-03-26T08:31:38.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T08:31:38.537 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.73320.asok 2024-03-26T08:31:38.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:31:38.538 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.73320.asok status 2024-03-26T08:31:38.785 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-26T08:31:38.785 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2024-03-26T08:31:38.785 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "3be9a5c9bc793e11e2800a8c0c696e8b46742033", 2024-03-26T08:31:38.786 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-26T08:31:38.786 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "admin", 2024-03-26T08:31:38.786 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2024-03-26T08:31:38.786 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-26T08:31:38.786 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "73320", 2024-03-26T08:31:38.786 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2024-03-26T08:31:38.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:31:38.786 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2024-03-26T08:31:38.786 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2024-03-26T08:31:38.786 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 14808, 2024-03-26T08:31:38.786 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2024-03-26T08:31:38.786 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2024-03-26T08:31:38.786 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2024-03-26T08:31:38.786 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 14808 2024-03-26T08:31:38.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:31:38.787 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-03-26T08:31:38.787 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-03-26T08:31:38.787 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-03-26T08:31:38.787 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 394128180 2024-03-26T08:31:38.787 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:31:38.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:31:38.787 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-03-26T08:31:38.787 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-03-26T08:31:38.787 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-03-26T08:31:38.787 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 394128180 2024-03-26T08:31:38.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:31:38.787 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.14808 192.168.0.1:0/394128180", 2024-03-26T08:31:38.787 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/394128180", 2024-03-26T08:31:38.787 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2024-03-26T08:31:38.788 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 17, 2024-03-26T08:31:38.788 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 91, 2024-03-26T08:31:38.788 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2024-03-26T08:31:38.788 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2024-03-26T08:31:38.788 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2024-03-26T08:31:38.788 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-26T08:31:38.796 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:38 smithi043 ceph-mon[31929]: osdmap e92: 12 total, 12 up, 12 in 2024-03-26T08:31:38.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:38 smithi183 ceph-mon[35818]: osdmap e92: 12 total, 12 up, 12 in 2024-03-26T08:31:38.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:31:38.801 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2024-03-26T08:31:38.801 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-26T08:31:38.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:38 smithi052 ceph-mon[35071]: osdmap e92: 12 total, 12 up, 12 in 2024-03-26T08:31:38.857 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2024-03-26T08:31:38.858 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-26T08:31:38.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:31:38.858 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-26T08:31:38.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:31:38.965 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-26T08:31:39.065 INFO:teuthology.orchestra.run.smithi043.stdout:# file: . 2024-03-26T08:31:39.065 INFO:teuthology.orchestra.run.smithi043.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-26T08:31:39.065 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:31:39.066 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-26T08:31:39.067 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-26T08:31:39.128 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:31:39.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.367+0000 7f40bbe00640 1 -- 172.21.15.43:0/2746854673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40bc104900 msgr2=0x7f40bc106cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:39.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.367+0000 7f40bbe00640 1 --2- 172.21.15.43:0/2746854673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40bc104900 0x7f40bc106cf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009900 tx=0x7f40a402bc30 comp rx=0 tx=0).stop 2024-03-26T08:31:39.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.367+0000 7f40bbe00640 1 -- 172.21.15.43:0/2746854673 shutdown_connections 2024-03-26T08:31:39.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.367+0000 7f40bbe00640 1 --2- 172.21.15.43:0/2746854673 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40bc107230 0x7f40bc109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:39.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.367+0000 7f40bbe00640 1 --2- 172.21.15.43:0/2746854673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40bc104900 0x7f40bc106cf0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:39.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.367+0000 7f40bbe00640 1 --2- 172.21.15.43:0/2746854673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40bc101fd0 0x7f40bc1043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:39.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.367+0000 7f40bbe00640 1 -- 172.21.15.43:0/2746854673 >> 172.21.15.43:0/2746854673 conn(0x7f40bc0fb980 msgr2=0x7f40bc0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:39.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.367+0000 7f40bbe00640 1 -- 172.21.15.43:0/2746854673 shutdown_connections 2024-03-26T08:31:39.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.367+0000 7f40bbe00640 1 -- 172.21.15.43:0/2746854673 wait complete. 2024-03-26T08:31:39.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.368+0000 7f40bbe00640 1 Processor -- start 2024-03-26T08:31:39.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.368+0000 7f40bbe00640 1 -- start start 2024-03-26T08:31:39.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.369+0000 7f40bbe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40bc101fd0 0x7f40bc1977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:39.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.369+0000 7f40bbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40bc104900 0x7f40bc197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:39.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.369+0000 7f40bbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40bc107230 0x7f40bc19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:39.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.369+0000 7f40bbe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc10d030 con 0x7f40bc107230 2024-03-26T08:31:39.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.369+0000 7f40bbe00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc10ceb0 con 0x7f40bc101fd0 2024-03-26T08:31:39.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.369+0000 7f40bbe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc10d1b0 con 0x7f40bc104900 2024-03-26T08:31:39.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.369+0000 7f40bb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40bc107230 0x7f40bc19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:39.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.369+0000 7f40bb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40bc107230 0x7f40bc19ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51304/0 (socket says 172.21.15.43:51304) 2024-03-26T08:31:39.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.369+0000 7f40bb400640 1 -- 172.21.15.43:0/2099265754 learned_addr learned my addr 172.21.15.43:0/2099265754 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:39.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.369+0000 7f40baa00640 1 --2- 172.21.15.43:0/2099265754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40bc101fd0 0x7f40bc1977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:39.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.369+0000 7f40ba000640 1 --2- 172.21.15.43:0/2099265754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40bc104900 0x7f40bc197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:39.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.370+0000 7f40bb400640 1 -- 172.21.15.43:0/2099265754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40bc104900 msgr2=0x7f40bc197ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:39.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.370+0000 7f40bb400640 1 --2- 172.21.15.43:0/2099265754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40bc104900 0x7f40bc197ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:39.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.370+0000 7f40bb400640 1 -- 172.21.15.43:0/2099265754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40bc101fd0 msgr2=0x7f40bc1977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:39.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.370+0000 7f40bb400640 1 --2- 172.21.15.43:0/2099265754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40bc101fd0 0x7f40bc1977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:39.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.370+0000 7f40bb400640 1 -- 172.21.15.43:0/2099265754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40bc19f370 con 0x7f40bc107230 2024-03-26T08:31:39.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.370+0000 7f40bb400640 1 --2- 172.21.15.43:0/2099265754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40bc107230 0x7f40bc19ed00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f40b400d950 tx=0x7f40b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:39.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.370+0000 7f40b3400640 1 -- 172.21.15.43:0/2099265754 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4017030 con 0x7f40bc107230 2024-03-26T08:31:39.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.370+0000 7f40b3400640 1 -- 172.21.15.43:0/2099265754 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f40b4004020 con 0x7f40bc107230 2024-03-26T08:31:39.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.370+0000 7f40bbe00640 1 -- 172.21.15.43:0/2099265754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40bc19f660 con 0x7f40bc107230 2024-03-26T08:31:39.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.371+0000 7f40b3400640 1 -- 172.21.15.43:0/2099265754 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4003b20 con 0x7f40bc107230 2024-03-26T08:31:39.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.371+0000 7f40bbe00640 1 -- 172.21.15.43:0/2099265754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40bc19fb30 con 0x7f40bc107230 2024-03-26T08:31:39.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.372+0000 7f40bbe00640 1 -- 172.21.15.43:0/2099265754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40bc0fd470 con 0x7f40bc107230 2024-03-26T08:31:39.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.374+0000 7f40b3400640 1 -- 172.21.15.43:0/2099265754 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f40b401a070 con 0x7f40bc107230 2024-03-26T08:31:39.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.376+0000 7f40b3400640 1 --2- 172.21.15.43:0/2099265754 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f409c0a2800 0x7f409c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:39.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.377+0000 7f40b3400640 1 -- 172.21.15.43:0/2099265754 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f40b40ce8d0 con 0x7f40bc107230 2024-03-26T08:31:39.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.377+0000 7f40baa00640 1 --2- 172.21.15.43:0/2099265754 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f409c0a2800 0x7f409c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:39.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.378+0000 7f40baa00640 1 --2- 172.21.15.43:0/2099265754 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f409c0a2800 0x7f409c0a4cc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f40ac005fd0 tx=0x7f40ac009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:39.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.380+0000 7f40b3400640 1 -- 172.21.15.43:0/2099265754 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f40b4021070 con 0x7f40bc107230 2024-03-26T08:31:39.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.585+0000 7f40bbe00640 1 -- 172.21.15.43:0/2099265754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40bc0622a0 con 0x7f40bc107230 2024-03-26T08:31:39.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.590+0000 7f40b3400640 1 -- 172.21.15.43:0/2099265754 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24576 (secure 0 0 0) 0x7f40b409b5d0 con 0x7f40bc107230 2024-03-26T08:31:39.583 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:31:39.583 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":92,"fsid":"88759420-eb49-11ee-95d0-87774f69a715","created":"2024-03-26T08:20:27.817107+0000","modified":"2024-03-26T08:31:37.455519+0000","last_up_change":"2024-03-26T08:30:10.520976+0000","last_in_change":"2024-03-26T08:29:46.394360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:24:54.852427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:31:19.924902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:31:21.323881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T08:31:24.951015+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"28670673-9193-4fa3-9cc0-c1a5a6b27284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6803","nonce":936601146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6805","nonce":936601146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6809","nonce":936601146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6807","nonce":936601146}]},"public_addr":"172.21.15.43:6803/936601146","cluster_addr":"172.21.15.43:6805/936601146","heartbeat_back_addr":"172.21.15.43:6809/936601146","heartbeat_front_addr":"172.21.15.43:6807/936601146","state":["exists","up"]},{"osd":1,"uuid":"fa8ad6b6-277a-42b8-b0f8-b36fc219de2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6801","nonce":3268412328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6803","nonce":3268412328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6807","nonce":3268412328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6805","nonce":3268412328}]},"public_addr":"172.21.15.52:6801/3268412328","cluster_addr":"172.21.15.52:6803/3268412328","heartbeat_back_addr":"172.21.15.52:6807/3268412328","heartbeat_front_addr":"172.21.15.52:6805/3268412328","state":["exists","up"]},{"osd":2,"uuid":"ec0039c3-f9bd-419a-81de-fa4418359808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6801","nonce":2103180908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6803","nonce":2103180908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6807","nonce":2103180908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6805","nonce":2103180908}]},"public_addr":"172.21.15.183:6801/2103180908","cluster_addr":"172.21.15.183:6803/2103180908","heartbeat_back_addr":"172.21.15.183:6807/2103180908","heartbeat_front_addr":"172.21.15.183:6805/2103180908","state":["exists","up"]},{"osd":3,"uuid":"e7dcbf00-6586-4664-a69e-3cb0df8dfd30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6811","nonce":3722846033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6813","nonce":3722846033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6817","nonce":3722846033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6815","nonce":3722846033}]},"public_addr":"172.21.15.43:6811/3722846033","cluster_addr":"172.21.15.43:6813/3722846033","heartbeat_back_addr":"172.21.15.43:6817/3722846033","heartbeat_front_addr":"172.21.15.43:6815/3722846033","state":["exists","up"]},{"osd":4,"uuid":"db089a49-5e98-432d-9687-cdc044114c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6809","nonce":4028436523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6811","nonce":4028436523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6815","nonce":4028436523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6813","nonce":4028436523}]},"public_addr":"172.21.15.52:6809/4028436523","cluster_addr":"172.21.15.52:6811/4028436523","heartbeat_back_addr":"172.21.15.52:6815/4028436523","heartbeat_front_addr":"172.21.15.52:6813/4028436523","state":["exists","up"]},{"osd":5,"uuid":"556ac804-02ea-423b-a599-55baf5be4ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6809","nonce":787132252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6811","nonce":787132252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6815","nonce":787132252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6813","nonce":787132252}]},"public_addr":"172.21.15.183:6809/787132252","cluster_addr":"172.21.15.183:6811/787132252","heartbeat_back_addr":"172.21.15.183:6815/787132252","heartbeat_front_addr":"172.21.15.183:6813/787132252","state":["exists","up"]},{"osd":6,"uuid":"3de2a49d-6cef-4580-9d8a-f90cdfcecd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6819","nonce":4206861158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6821","nonce":4206861158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6825","nonce":4206861158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6823","nonce":4206861158}]},"public_addr":"172.21.15.43:6819/4206861158","cluster_addr":"172.21.15.43:6821/4206861158","heartbeat_back_addr":"172.21.15.43:6825/4206861158","heartbeat_front_addr":"172.21.15.43:6823/4206861158","state":["exists","up"]},{"osd":7,"uuid":"a3b832f7-7d7d-4a62-b679-db6da98275d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6817","nonce":3452064146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6819","nonce":3452064146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6823","nonce":3452064146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6821","nonce":3452064146}]},"public_addr":"172.21.15.52:6817/3452064146","cluster_addr":"172.21.15.52:6819/3452064146","heartbeat_back_addr":"172.21.15.52:6823/3452064146","heartbeat_front_addr":"172.21.15.52:6821/3452064146","state":["exists","up"]},{"osd":8,"uuid":"ddc6b07f-d703-414d-a59a-02316376b13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6817","nonce":1023131725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6819","nonce":1023131725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6823","nonce":1023131725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6821","nonce":1023131725}]},"public_addr":"172.21.15.183:6817/1023131725","cluster_addr":"172.21.15.183:6819/1023131725","heartbeat_back_addr":"172.21.15.183:6823/1023131725","heartbeat_front_addr":"172.21.15.183:6821/1023131725","state":["exists","up"]},{"osd":9,"uuid":"ac16bb0e-6232-45c8-b16a-4d878874c588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6827","nonce":3935195657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6829","nonce":3935195657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6833","nonce":3935195657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6831","nonce":3935195657}]},"public_addr":"172.21.15.43:6827/3935195657","cluster_addr":"172.21.15.43:6829/3935195657","heartbeat_back_addr":"172.21.15.43:6833/3935195657","heartbeat_front_addr":"172.21.15.43:6831/3935195657","state":["exists","up"]},{"osd":10,"uuid":"fd839188-f398-4cff-aa7b-36522ff1a8a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6825","nonce":377628320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6827","nonce":377628320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6831","nonce":377628320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6829","nonce":377628320}]},"public_addr":"172.21.15.52:6825/377628320","cluster_addr":"172.21.15.52:6827/377628320","heartbeat_back_addr":"172.21.15.52:6831/377628320","heartbeat_front_addr":"172.21.15.52:6829/377628320","state":["exists","up"]},{"osd":11,"uuid":"a40282a0-0c64-49ee-b64e-6f3f8119e827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6825","nonce":1373086433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6827","nonce":1373086433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6831","nonce":1373086433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6829","nonce":1373086433}]},"public_addr":"172.21.15.183:6825/1373086433","cluster_addr":"172.21.15.183:6827/1373086433","heartbeat_back_addr":"172.21.15.183:6831/1373086433","heartbeat_front_addr":"172.21.15.183:6829/1373086433","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:23:43.478905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:17.573532+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:50.286610+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:25:27.407663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:03.081294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:36.057416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:12.062013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:47.032271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:21.023201+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:57.364739+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:29:34.039995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:30:08.827577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6801/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/4240775295":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/510203140":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/4251957687":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:6801/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1885244665":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/298320867":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1301092625":"2024-03-27T08:21:36.149092+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:31:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.592+0000 7f40bbe00640 1 -- 172.21.15.43:0/2099265754 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f409c0a2800 msgr2=0x7f409c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.592+0000 7f40bbe00640 1 --2- 172.21.15.43:0/2099265754 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f409c0a2800 0x7f409c0a4cc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f40ac005fd0 tx=0x7f40ac009500 comp rx=0 tx=0).stop 2024-03-26T08:31:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.592+0000 7f40bbe00640 1 -- 172.21.15.43:0/2099265754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40bc107230 msgr2=0x7f40bc19ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.592+0000 7f40bbe00640 1 --2- 172.21.15.43:0/2099265754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40bc107230 0x7f40bc19ed00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f40b400d950 tx=0x7f40b400de10 comp rx=0 tx=0).stop 2024-03-26T08:31:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.593+0000 7f40bbe00640 1 -- 172.21.15.43:0/2099265754 shutdown_connections 2024-03-26T08:31:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.593+0000 7f40bbe00640 1 --2- 172.21.15.43:0/2099265754 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f409c0a2800 0x7f409c0a4cc0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.593+0000 7f40bbe00640 1 --2- 172.21.15.43:0/2099265754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40bc107230 0x7f40bc19ed00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.593+0000 7f40bbe00640 1 --2- 172.21.15.43:0/2099265754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40bc104900 0x7f40bc197ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.593+0000 7f40bbe00640 1 --2- 172.21.15.43:0/2099265754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40bc101fd0 0x7f40bc1977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.593+0000 7f40bbe00640 1 -- 172.21.15.43:0/2099265754 >> 172.21.15.43:0/2099265754 conn(0x7f40bc0fb980 msgr2=0x7f40bc102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:39.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.593+0000 7f40bbe00640 1 -- 172.21.15.43:0/2099265754 shutdown_connections 2024-03-26T08:31:39.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.593+0000 7f40bbe00640 1 -- 172.21.15.43:0/2099265754 wait complete. 2024-03-26T08:31:39.606 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T08:24:54.852427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T08:31:19.924902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T08:31:21.323881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T08:31:24.951015+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T08:31:39.606 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T08:31:39.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:39 smithi183 ceph-mon[35818]: pgmap v383: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:39.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:39 smithi052 ceph-mon[35071]: pgmap v383: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:39.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.840+0000 7f8659600640 1 -- 172.21.15.43:0/3390671407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8654105350 msgr2=0x7f865410bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:39.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.840+0000 7f8659600640 1 --2- 172.21.15.43:0/3390671407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8654105350 0x7f865410bbe0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f864c009860 tx=0x7f864c02bb70 comp rx=0 tx=0).stop 2024-03-26T08:31:39.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.840+0000 7f8659600640 1 -- 172.21.15.43:0/3390671407 shutdown_connections 2024-03-26T08:31:39.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.840+0000 7f8659600640 1 --2- 172.21.15.43:0/3390671407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8654105350 0x7f865410bbe0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:39.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.840+0000 7f8659600640 1 --2- 172.21.15.43:0/3390671407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8654104990 0x7f8654104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:39.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.840+0000 7f8659600640 1 --2- 172.21.15.43:0/3390671407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86541037a0 0x7f8654103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:39.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.840+0000 7f8659600640 1 -- 172.21.15.43:0/3390671407 >> 172.21.15.43:0/3390671407 conn(0x7f86540fed10 msgr2=0x7f8654101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:39.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.840+0000 7f8659600640 1 -- 172.21.15.43:0/3390671407 shutdown_connections 2024-03-26T08:31:39.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.841+0000 7f8659600640 1 -- 172.21.15.43:0/3390671407 wait complete. 2024-03-26T08:31:39.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.842+0000 7f8659600640 1 Processor -- start 2024-03-26T08:31:39.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.842+0000 7f8659600640 1 -- start start 2024-03-26T08:31:39.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.843+0000 7f8659600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86541037a0 0x7f8654199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:39.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.843+0000 7f8659600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8654104990 0x7f8654199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:39.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.843+0000 7f8659600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8654105350 0x7f86541a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:39.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.843+0000 7f8659600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865410f2d0 con 0x7f8654104990 2024-03-26T08:31:39.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.843+0000 7f8659600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865410f150 con 0x7f86541037a0 2024-03-26T08:31:39.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.843+0000 7f8659600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865410f450 con 0x7f8654105350 2024-03-26T08:31:39.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.843+0000 7f8653400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8654104990 0x7f8654199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:39.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.843+0000 7f8653400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8654104990 0x7f8654199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51316/0 (socket says 172.21.15.43:51316) 2024-03-26T08:31:39.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.843+0000 7f8653400640 1 -- 172.21.15.43:0/1978477446 learned_addr learned my addr 172.21.15.43:0/1978477446 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:39.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.843+0000 7f8653e00640 1 --2- 172.21.15.43:0/1978477446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86541037a0 0x7f8654199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:39.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.843+0000 7f8658c00640 1 --2- 172.21.15.43:0/1978477446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8654105350 0x7f86541a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:39.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.844+0000 7f8653400640 1 -- 172.21.15.43:0/1978477446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8654105350 msgr2=0x7f86541a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:39.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.844+0000 7f8653400640 1 --2- 172.21.15.43:0/1978477446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8654105350 0x7f86541a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:39.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.844+0000 7f8653400640 1 -- 172.21.15.43:0/1978477446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86541037a0 msgr2=0x7f8654199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:39.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.844+0000 7f8653400640 1 --2- 172.21.15.43:0/1978477446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86541037a0 0x7f8654199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:39.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.844+0000 7f8653400640 1 -- 172.21.15.43:0/1978477446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86541a1570 con 0x7f8654104990 2024-03-26T08:31:39.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.844+0000 7f8653400640 1 --2- 172.21.15.43:0/1978477446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8654104990 0x7f8654199e90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f864400eb90 tx=0x7f864400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:39.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.844+0000 7f8650c00640 1 -- 172.21.15.43:0/1978477446 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8644004b20 con 0x7f8654104990 2024-03-26T08:31:39.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.844+0000 7f8650c00640 1 -- 172.21.15.43:0/1978477446 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8644004cc0 con 0x7f8654104990 2024-03-26T08:31:39.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.844+0000 7f8659600640 1 -- 172.21.15.43:0/1978477446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86541a1860 con 0x7f8654104990 2024-03-26T08:31:39.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.844+0000 7f8650c00640 1 -- 172.21.15.43:0/1978477446 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8644010690 con 0x7f8654104990 2024-03-26T08:31:39.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.844+0000 7f8659600640 1 -- 172.21.15.43:0/1978477446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86541a1d60 con 0x7f8654104990 2024-03-26T08:31:39.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.846+0000 7f8659600640 1 -- 172.21.15.43:0/1978477446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f865410aac0 con 0x7f8654104990 2024-03-26T08:31:39.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.848+0000 7f8650c00640 1 -- 172.21.15.43:0/1978477446 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8644005020 con 0x7f8654104990 2024-03-26T08:31:39.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.849+0000 7f8650c00640 1 --2- 172.21.15.43:0/1978477446 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f86380a2800 0x7f86380a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:39.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.849+0000 7f8650c00640 1 -- 172.21.15.43:0/1978477446 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f86440cf4b0 con 0x7f8654104990 2024-03-26T08:31:39.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.849+0000 7f8653e00640 1 --2- 172.21.15.43:0/1978477446 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f86380a2800 0x7f86380a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:39.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.850+0000 7f8653e00640 1 --2- 172.21.15.43:0/1978477446 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f86380a2800 0x7f86380a4cc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f86541047f0 tx=0x7f863c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:39.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:39.853+0000 7f8650c00640 1 -- 172.21.15.43:0/1978477446 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f864409c1b0 con 0x7f8654104990 2024-03-26T08:31:39.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:39 smithi043 ceph-mon[31929]: pgmap v383: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:31:40.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.058+0000 7f8659600640 1 -- 172.21.15.43:0/1978477446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f86540622a0 con 0x7f8654104990 2024-03-26T08:31:40.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.059+0000 7f8650c00640 1 -- 172.21.15.43:0/1978477446 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f864409bab0 con 0x7f8654104990 2024-03-26T08:31:40.052 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-26T08:31:40.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.062+0000 7f8659600640 1 -- 172.21.15.43:0/1978477446 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f86380a2800 msgr2=0x7f86380a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:40.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.062+0000 7f8659600640 1 --2- 172.21.15.43:0/1978477446 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f86380a2800 0x7f86380a4cc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f86541047f0 tx=0x7f863c009260 comp rx=0 tx=0).stop 2024-03-26T08:31:40.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.062+0000 7f8659600640 1 -- 172.21.15.43:0/1978477446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8654104990 msgr2=0x7f8654199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:40.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.062+0000 7f8659600640 1 --2- 172.21.15.43:0/1978477446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8654104990 0x7f8654199e90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f864400eb90 tx=0x7f864400ef90 comp rx=0 tx=0).stop 2024-03-26T08:31:40.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.062+0000 7f8659600640 1 -- 172.21.15.43:0/1978477446 shutdown_connections 2024-03-26T08:31:40.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.062+0000 7f8659600640 1 --2- 172.21.15.43:0/1978477446 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f86380a2800 0x7f86380a4cc0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:40.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.062+0000 7f8659600640 1 --2- 172.21.15.43:0/1978477446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8654105350 0x7f86541a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:40.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.062+0000 7f8659600640 1 --2- 172.21.15.43:0/1978477446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8654104990 0x7f8654199e90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:40.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.062+0000 7f8659600640 1 --2- 172.21.15.43:0/1978477446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86541037a0 0x7f8654199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:40.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.062+0000 7f8659600640 1 -- 172.21.15.43:0/1978477446 >> 172.21.15.43:0/1978477446 conn(0x7f86540fed10 msgr2=0x7f8654107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:40.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.063+0000 7f8659600640 1 -- 172.21.15.43:0/1978477446 shutdown_connections 2024-03-26T08:31:40.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.063+0000 7f8659600640 1 -- 172.21.15.43:0/1978477446 wait complete. 2024-03-26T08:31:40.075 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T08:31:40.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.309+0000 7fba3a400640 1 -- 172.21.15.43:0/2911469115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba34105470 msgr2=0x7fba3410bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:40.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.309+0000 7fba3a400640 1 --2- 172.21.15.43:0/2911469115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba34105470 0x7fba3410bd00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fba2c009ca0 tx=0x7fba2c02c0f0 comp rx=0 tx=0).stop 2024-03-26T08:31:40.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.309+0000 7fba3a400640 1 -- 172.21.15.43:0/2911469115 shutdown_connections 2024-03-26T08:31:40.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.309+0000 7fba3a400640 1 --2- 172.21.15.43:0/2911469115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba34105470 0x7fba3410bd00 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:40.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.309+0000 7fba3a400640 1 --2- 172.21.15.43:0/2911469115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba34104ab0 0x7fba34104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:40.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.309+0000 7fba3a400640 1 --2- 172.21.15.43:0/2911469115 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba341038c0 0x7fba34103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:40.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.309+0000 7fba3a400640 1 -- 172.21.15.43:0/2911469115 >> 172.21.15.43:0/2911469115 conn(0x7fba340fee50 msgr2=0x7fba34101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:40.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.309+0000 7fba3a400640 1 -- 172.21.15.43:0/2911469115 shutdown_connections 2024-03-26T08:31:40.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.309+0000 7fba3a400640 1 -- 172.21.15.43:0/2911469115 wait complete. 2024-03-26T08:31:40.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.310+0000 7fba3a400640 1 Processor -- start 2024-03-26T08:31:40.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.310+0000 7fba3a400640 1 -- start start 2024-03-26T08:31:40.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.311+0000 7fba3a400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba341038c0 0x7fba34199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:40.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.311+0000 7fba3a400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba34104ab0 0x7fba34199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:40.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.311+0000 7fba3a400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba34105470 0x7fba341a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:40.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.311+0000 7fba3a400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3410f3d0 con 0x7fba34105470 2024-03-26T08:31:40.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.311+0000 7fba3a400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3410f250 con 0x7fba34104ab0 2024-03-26T08:31:40.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.311+0000 7fba3a400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3410f550 con 0x7fba341038c0 2024-03-26T08:31:40.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.311+0000 7fba33e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba34105470 0x7fba341a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:40.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.311+0000 7fba33e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba34105470 0x7fba341a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51332/0 (socket says 172.21.15.43:51332) 2024-03-26T08:31:40.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.311+0000 7fba33e00640 1 -- 172.21.15.43:0/406416872 learned_addr learned my addr 172.21.15.43:0/406416872 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:40.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.311+0000 7fba33400640 1 --2- 172.21.15.43:0/406416872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba341038c0 0x7fba34199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:40.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.311+0000 7fba32a00640 1 --2- 172.21.15.43:0/406416872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba34104ab0 0x7fba34199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:40.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.312+0000 7fba33e00640 1 -- 172.21.15.43:0/406416872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba341038c0 msgr2=0x7fba34199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:40.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.312+0000 7fba33e00640 1 --2- 172.21.15.43:0/406416872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba341038c0 0x7fba34199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:40.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.312+0000 7fba33e00640 1 -- 172.21.15.43:0/406416872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba34104ab0 msgr2=0x7fba34199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:40.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.312+0000 7fba33e00640 1 --2- 172.21.15.43:0/406416872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba34104ab0 0x7fba34199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:40.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.312+0000 7fba33e00640 1 -- 172.21.15.43:0/406416872 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba341a16b0 con 0x7fba34105470 2024-03-26T08:31:40.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.312+0000 7fba33e00640 1 --2- 172.21.15.43:0/406416872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba34105470 0x7fba341a1040 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fba2c032380 tx=0x7fba2c032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:40.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.312+0000 7fba2be00640 1 -- 172.21.15.43:0/406416872 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c041020 con 0x7fba34105470 2024-03-26T08:31:40.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.312+0000 7fba2be00640 1 -- 172.21.15.43:0/406416872 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fba2c003e20 con 0x7fba34105470 2024-03-26T08:31:40.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.312+0000 7fba3a400640 1 -- 172.21.15.43:0/406416872 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba341a1940 con 0x7fba34105470 2024-03-26T08:31:40.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.313+0000 7fba2be00640 1 -- 172.21.15.43:0/406416872 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c039040 con 0x7fba34105470 2024-03-26T08:31:40.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.313+0000 7fba3a400640 1 -- 172.21.15.43:0/406416872 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba341a1e80 con 0x7fba34105470 2024-03-26T08:31:40.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.314+0000 7fba3a400640 1 -- 172.21.15.43:0/406416872 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba3410acb0 con 0x7fba34105470 2024-03-26T08:31:40.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.316+0000 7fba2be00640 1 -- 172.21.15.43:0/406416872 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fba2c044020 con 0x7fba34105470 2024-03-26T08:31:40.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.318+0000 7fba2be00640 1 --2- 172.21.15.43:0/406416872 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fba180a2880 0x7fba180a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:40.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.319+0000 7fba2be00640 1 -- 172.21.15.43:0/406416872 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7fba2c0f0e60 con 0x7fba34105470 2024-03-26T08:31:40.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.319+0000 7fba33400640 1 --2- 172.21.15.43:0/406416872 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fba180a2880 0x7fba180a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:40.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.319+0000 7fba33400640 1 --2- 172.21.15.43:0/406416872 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fba180a2880 0x7fba180a4d40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fba34104910 tx=0x7fba24005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:40.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.323+0000 7fba2be00640 1 -- 172.21.15.43:0/406416872 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fba2c04b020 con 0x7fba34105470 2024-03-26T08:31:40.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.530+0000 7fba3a400640 1 -- 172.21.15.43:0/406416872 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fba340622a0 con 0x7fba34105470 2024-03-26T08:31:40.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.531+0000 7fba2be00640 1 -- 172.21.15.43:0/406416872 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7fba2c0bdb60 con 0x7fba34105470 2024-03-26T08:31:40.524 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-26T08:31:40.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.534+0000 7fba3a400640 1 -- 172.21.15.43:0/406416872 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fba180a2880 msgr2=0x7fba180a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:40.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.534+0000 7fba3a400640 1 --2- 172.21.15.43:0/406416872 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fba180a2880 0x7fba180a4d40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fba34104910 tx=0x7fba24005f90 comp rx=0 tx=0).stop 2024-03-26T08:31:40.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.534+0000 7fba3a400640 1 -- 172.21.15.43:0/406416872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba34105470 msgr2=0x7fba341a1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:40.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.534+0000 7fba3a400640 1 --2- 172.21.15.43:0/406416872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba34105470 0x7fba341a1040 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fba2c032380 tx=0x7fba2c032820 comp rx=0 tx=0).stop 2024-03-26T08:31:40.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.534+0000 7fba3a400640 1 -- 172.21.15.43:0/406416872 shutdown_connections 2024-03-26T08:31:40.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.534+0000 7fba3a400640 1 --2- 172.21.15.43:0/406416872 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fba180a2880 0x7fba180a4d40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:40.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.534+0000 7fba3a400640 1 --2- 172.21.15.43:0/406416872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba34105470 0x7fba341a1040 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:40.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.534+0000 7fba3a400640 1 --2- 172.21.15.43:0/406416872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba34104ab0 0x7fba34199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:40.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.534+0000 7fba3a400640 1 --2- 172.21.15.43:0/406416872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba341038c0 0x7fba34199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:40.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.534+0000 7fba3a400640 1 -- 172.21.15.43:0/406416872 >> 172.21.15.43:0/406416872 conn(0x7fba340fee50 msgr2=0x7fba34107e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:40.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.534+0000 7fba3a400640 1 -- 172.21.15.43:0/406416872 shutdown_connections 2024-03-26T08:31:40.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.534+0000 7fba3a400640 1 -- 172.21.15.43:0/406416872 wait complete. 2024-03-26T08:31:40.548 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T08:31:40.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.774+0000 7f803f200640 1 -- 172.21.15.43:0/1591305824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8038074100 msgr2=0x7f8038074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:40.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.774+0000 7f803f200640 1 --2- 172.21.15.43:0/1591305824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8038074100 0x7f8038074580 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8028009900 tx=0x7f802802bc30 comp rx=0 tx=0).stop 2024-03-26T08:31:40.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.774+0000 7f803f200640 1 -- 172.21.15.43:0/1591305824 shutdown_connections 2024-03-26T08:31:40.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.774+0000 7f803f200640 1 --2- 172.21.15.43:0/1591305824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8038109440 0x7f803810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:40.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.774+0000 7f803f200640 1 --2- 172.21.15.43:0/1591305824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8038074100 0x7f8038074580 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:40.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.774+0000 7f803f200640 1 --2- 172.21.15.43:0/1591305824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80380756e0 0x7f8038073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:40.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.774+0000 7f803f200640 1 -- 172.21.15.43:0/1591305824 >> 172.21.15.43:0/1591305824 conn(0x7f80380fcb10 msgr2=0x7f80380fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:40.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.774+0000 7f803f200640 1 -- 172.21.15.43:0/1591305824 shutdown_connections 2024-03-26T08:31:40.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.774+0000 7f803f200640 1 -- 172.21.15.43:0/1591305824 wait complete. 2024-03-26T08:31:40.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.775+0000 7f803f200640 1 Processor -- start 2024-03-26T08:31:40.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.775+0000 7f803f200640 1 -- start start 2024-03-26T08:31:40.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.775+0000 7f803f200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8038074100 0x7f8038199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:40.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.776+0000 7f803f200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80380756e0 0x7f8038199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:40.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.776+0000 7f803f200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8038109440 0x7f80381a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:40.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.776+0000 7f803f200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803810f340 con 0x7f8038074100 2024-03-26T08:31:40.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.776+0000 7f803f200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803810f1c0 con 0x7f80380756e0 2024-03-26T08:31:40.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.776+0000 7f803f200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803810f4c0 con 0x7f8038109440 2024-03-26T08:31:40.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.776+0000 7f803de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8038074100 0x7f8038199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:40.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.776+0000 7f803de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8038074100 0x7f8038199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51362/0 (socket says 172.21.15.43:51362) 2024-03-26T08:31:40.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.776+0000 7f803de00640 1 -- 172.21.15.43:0/3264761428 learned_addr learned my addr 172.21.15.43:0/3264761428 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:40.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.776+0000 7f803d400640 1 --2- 172.21.15.43:0/3264761428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80380756e0 0x7f8038199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:40.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.776+0000 7f803e800640 1 --2- 172.21.15.43:0/3264761428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8038109440 0x7f80381a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:40.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.776+0000 7f803de00640 1 -- 172.21.15.43:0/3264761428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8038109440 msgr2=0x7f80381a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:40.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.776+0000 7f803de00640 1 --2- 172.21.15.43:0/3264761428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8038109440 0x7f80381a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:40.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.777+0000 7f803de00640 1 -- 172.21.15.43:0/3264761428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80380756e0 msgr2=0x7f8038199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:40.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.777+0000 7f803de00640 1 --2- 172.21.15.43:0/3264761428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80380756e0 0x7f8038199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:40.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.777+0000 7f803de00640 1 -- 172.21.15.43:0/3264761428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80381a1620 con 0x7f8038074100 2024-03-26T08:31:40.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.777+0000 7f803d400640 1 --2- 172.21.15.43:0/3264761428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80380756e0 0x7f8038199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:31:40.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.777+0000 7f803e800640 1 --2- 172.21.15.43:0/3264761428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8038109440 0x7f80381a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:31:40.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.777+0000 7f803de00640 1 --2- 172.21.15.43:0/3264761428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8038074100 0x7f8038199a00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f802c00d950 tx=0x7f802c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:40.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.777+0000 7f8032a00640 1 -- 172.21.15.43:0/3264761428 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c017070 con 0x7f8038074100 2024-03-26T08:31:40.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.777+0000 7f803f200640 1 -- 172.21.15.43:0/3264761428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80381a1910 con 0x7f8038074100 2024-03-26T08:31:40.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.777+0000 7f8032a00640 1 -- 172.21.15.43:0/3264761428 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f802c009c30 con 0x7f8038074100 2024-03-26T08:31:40.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.777+0000 7f8032a00640 1 -- 172.21.15.43:0/3264761428 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c020060 con 0x7f8038074100 2024-03-26T08:31:40.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.777+0000 7f803f200640 1 -- 172.21.15.43:0/3264761428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80381a1f00 con 0x7f8038074100 2024-03-26T08:31:40.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.779+0000 7f803f200640 1 -- 172.21.15.43:0/3264761428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8038108170 con 0x7f8038074100 2024-03-26T08:31:40.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.781+0000 7f8032a00640 1 -- 172.21.15.43:0/3264761428 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f802c0109a0 con 0x7f8038074100 2024-03-26T08:31:40.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.782+0000 7f8032a00640 1 --2- 172.21.15.43:0/3264761428 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f80200a2800 0x7f80200a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:40.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.782+0000 7f8032a00640 1 -- 172.21.15.43:0/3264761428 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f802c0cee70 con 0x7f8038074100 2024-03-26T08:31:40.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.782+0000 7f803d400640 1 --2- 172.21.15.43:0/3264761428 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f80200a2800 0x7f80200a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:40.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.783+0000 7f803d400640 1 --2- 172.21.15.43:0/3264761428 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f80200a2800 0x7f80200a4cc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8028006010 tx=0x7f802802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:40.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.787+0000 7f8032a00640 1 -- 172.21.15.43:0/3264761428 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f802c009de0 con 0x7f8038074100 2024-03-26T08:31:40.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:40 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2099265754' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:31:40.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:40 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1978477446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:31:40.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:40 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2099265754' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:31:40.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:40 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1978477446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:31:40.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:40 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2099265754' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:31:40.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:40 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1978477446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:31:40.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.989+0000 7f803f200640 1 -- 172.21.15.43:0/3264761428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f80380622a0 con 0x7f8038074100 2024-03-26T08:31:40.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.990+0000 7f8032a00640 1 -- 172.21.15.43:0/3264761428 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f802c09bb70 con 0x7f8038074100 2024-03-26T08:31:40.983 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-26T08:31:40.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.993+0000 7f803f200640 1 -- 172.21.15.43:0/3264761428 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f80200a2800 msgr2=0x7f80200a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:40.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.993+0000 7f803f200640 1 --2- 172.21.15.43:0/3264761428 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f80200a2800 0x7f80200a4cc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8028006010 tx=0x7f802802c7e0 comp rx=0 tx=0).stop 2024-03-26T08:31:40.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.993+0000 7f803f200640 1 -- 172.21.15.43:0/3264761428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8038074100 msgr2=0x7f8038199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:40.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.993+0000 7f803f200640 1 --2- 172.21.15.43:0/3264761428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8038074100 0x7f8038199a00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f802c00d950 tx=0x7f802c00de10 comp rx=0 tx=0).stop 2024-03-26T08:31:40.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.993+0000 7f803f200640 1 -- 172.21.15.43:0/3264761428 shutdown_connections 2024-03-26T08:31:40.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.994+0000 7f803f200640 1 --2- 172.21.15.43:0/3264761428 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f80200a2800 0x7f80200a4cc0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:40.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.994+0000 7f803f200640 1 --2- 172.21.15.43:0/3264761428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8038109440 0x7f80381a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:40.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.994+0000 7f803f200640 1 --2- 172.21.15.43:0/3264761428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80380756e0 0x7f8038199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:40.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.994+0000 7f803f200640 1 --2- 172.21.15.43:0/3264761428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8038074100 0x7f8038199a00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:40.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.994+0000 7f803f200640 1 -- 172.21.15.43:0/3264761428 >> 172.21.15.43:0/3264761428 conn(0x7f80380fcb10 msgr2=0x7f8038105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:40.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.994+0000 7f803f200640 1 -- 172.21.15.43:0/3264761428 shutdown_connections 2024-03-26T08:31:40.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:40.994+0000 7f803f200640 1 -- 172.21.15.43:0/3264761428 wait complete. 2024-03-26T08:31:41.006 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T08:31:41.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.252+0000 7f00c8a00640 1 -- 172.21.15.43:0/3789718567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00c4108db0 msgr2=0x7f00c4109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:41.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.252+0000 7f00c8a00640 1 --2- 172.21.15.43:0/3789718567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00c4108db0 0x7f00c4109230 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f00b0009900 tx=0x7f00b002bc30 comp rx=0 tx=0).stop 2024-03-26T08:31:41.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.252+0000 7f00c8a00640 1 -- 172.21.15.43:0/3789718567 shutdown_connections 2024-03-26T08:31:41.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.252+0000 7f00c8a00640 1 --2- 172.21.15.43:0/3789718567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00c4109770 0x7f00c4110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:41.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.252+0000 7f00c8a00640 1 --2- 172.21.15.43:0/3789718567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00c4108db0 0x7f00c4109230 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:41.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.252+0000 7f00c8a00640 1 --2- 172.21.15.43:0/3789718567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00c4107bc0 0x7f00c4107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:41.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.252+0000 7f00c8a00640 1 -- 172.21.15.43:0/3789718567 >> 172.21.15.43:0/3789718567 conn(0x7f00c4076b70 msgr2=0x7f00c4078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:41.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.252+0000 7f00c8a00640 1 -- 172.21.15.43:0/3789718567 shutdown_connections 2024-03-26T08:31:41.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.252+0000 7f00c8a00640 1 -- 172.21.15.43:0/3789718567 wait complete. 2024-03-26T08:31:41.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.253+0000 7f00c8a00640 1 Processor -- start 2024-03-26T08:31:41.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.253+0000 7f00c8a00640 1 -- start start 2024-03-26T08:31:41.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.254+0000 7f00c8a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00c4107bc0 0x7f00c419ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:41.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.254+0000 7f00c8a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00c4108db0 0x7f00c419e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:41.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.254+0000 7f00c8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00c4109770 0x7f00c41a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:41.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.254+0000 7f00c8a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c41136d0 con 0x7f00c4109770 2024-03-26T08:31:41.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.254+0000 7f00c8a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c4113550 con 0x7f00c4108db0 2024-03-26T08:31:41.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.254+0000 7f00c8a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c4113850 con 0x7f00c4107bc0 2024-03-26T08:31:41.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.254+0000 7f00c3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00c4109770 0x7f00c41a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:41.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.254+0000 7f00c3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00c4109770 0x7f00c41a5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51384/0 (socket says 172.21.15.43:51384) 2024-03-26T08:31:41.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.254+0000 7f00c3e00640 1 -- 172.21.15.43:0/2771287707 learned_addr learned my addr 172.21.15.43:0/2771287707 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:41.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.254+0000 7f00c2a00640 1 --2- 172.21.15.43:0/2771287707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00c4108db0 0x7f00c419e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:41.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.254+0000 7f00c3400640 1 --2- 172.21.15.43:0/2771287707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00c4107bc0 0x7f00c419ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:41.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.255+0000 7f00c3e00640 1 -- 172.21.15.43:0/2771287707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00c4107bc0 msgr2=0x7f00c419ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:41.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.255+0000 7f00c3e00640 1 --2- 172.21.15.43:0/2771287707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00c4107bc0 0x7f00c419ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:41.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.255+0000 7f00c3e00640 1 -- 172.21.15.43:0/2771287707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00c4108db0 msgr2=0x7f00c419e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:41.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.255+0000 7f00c3e00640 1 --2- 172.21.15.43:0/2771287707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00c4108db0 0x7f00c419e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:41.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.255+0000 7f00c3e00640 1 -- 172.21.15.43:0/2771287707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c41a59b0 con 0x7f00c4109770 2024-03-26T08:31:41.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.255+0000 7f00c3e00640 1 --2- 172.21.15.43:0/2771287707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00c4109770 0x7f00c41a5310 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00cce0 tx=0x7f00bc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:41.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.255+0000 7f00bbe00640 1 -- 172.21.15.43:0/2771287707 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc00f450 con 0x7f00c4109770 2024-03-26T08:31:41.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.255+0000 7f00bbe00640 1 -- 172.21.15.43:0/2771287707 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f00bc00faa0 con 0x7f00c4109770 2024-03-26T08:31:41.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.255+0000 7f00c8a00640 1 -- 172.21.15.43:0/2771287707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c41a5ca0 con 0x7f00c4109770 2024-03-26T08:31:41.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.255+0000 7f00bbe00640 1 -- 172.21.15.43:0/2771287707 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc018b70 con 0x7f00c4109770 2024-03-26T08:31:41.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.255+0000 7f00c8a00640 1 -- 172.21.15.43:0/2771287707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c41a61a0 con 0x7f00c4109770 2024-03-26T08:31:41.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.257+0000 7f00c8a00640 1 -- 172.21.15.43:0/2771287707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c410ef00 con 0x7f00c4109770 2024-03-26T08:31:41.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.259+0000 7f00bbe00640 1 -- 172.21.15.43:0/2771287707 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f00bc018d10 con 0x7f00c4109770 2024-03-26T08:31:41.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.260+0000 7f00bbe00640 1 --2- 172.21.15.43:0/2771287707 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f00a80a2880 0x7f00a80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:41.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.260+0000 7f00bbe00640 1 -- 172.21.15.43:0/2771287707 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f00bc0cf2f0 con 0x7f00c4109770 2024-03-26T08:31:41.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.260+0000 7f00c3400640 1 --2- 172.21.15.43:0/2771287707 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f00a80a2880 0x7f00a80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:41.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.261+0000 7f00c3400640 1 --2- 172.21.15.43:0/2771287707 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f00a80a2880 0x7f00a80a4d40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f00b4005fd0 tx=0x7f00b4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.264+0000 7f00bbe00640 1 -- 172.21.15.43:0/2771287707 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f00bc014030 con 0x7f00c4109770 2024-03-26T08:31:41.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.470+0000 7f00c8a00640 1 -- 172.21.15.43:0/2771287707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f00c40622a0 con 0x7f00c4109770 2024-03-26T08:31:41.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.471+0000 7f00bbe00640 1 -- 172.21.15.43:0/2771287707 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f00bc09bff0 con 0x7f00c4109770 2024-03-26T08:31:41.464 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-26T08:31:41.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.474+0000 7f00c8a00640 1 -- 172.21.15.43:0/2771287707 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f00a80a2880 msgr2=0x7f00a80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:41.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.474+0000 7f00c8a00640 1 --2- 172.21.15.43:0/2771287707 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f00a80a2880 0x7f00a80a4d40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f00b4005fd0 tx=0x7f00b4005850 comp rx=0 tx=0).stop 2024-03-26T08:31:41.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.474+0000 7f00c8a00640 1 -- 172.21.15.43:0/2771287707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00c4109770 msgr2=0x7f00c41a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:41.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.474+0000 7f00c8a00640 1 --2- 172.21.15.43:0/2771287707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00c4109770 0x7f00c41a5310 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00cce0 tx=0x7f00bc00a620 comp rx=0 tx=0).stop 2024-03-26T08:31:41.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.474+0000 7f00c8a00640 1 -- 172.21.15.43:0/2771287707 shutdown_connections 2024-03-26T08:31:41.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.474+0000 7f00c8a00640 1 --2- 172.21.15.43:0/2771287707 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f00a80a2880 0x7f00a80a4d40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:41.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.474+0000 7f00c8a00640 1 --2- 172.21.15.43:0/2771287707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00c4109770 0x7f00c41a5310 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:41.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.474+0000 7f00c8a00640 1 --2- 172.21.15.43:0/2771287707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00c4108db0 0x7f00c419e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:41.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.474+0000 7f00c8a00640 1 --2- 172.21.15.43:0/2771287707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00c4107bc0 0x7f00c419ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:41.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.474+0000 7f00c8a00640 1 -- 172.21.15.43:0/2771287707 >> 172.21.15.43:0/2771287707 conn(0x7f00c4076b70 msgr2=0x7f00c410c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:41.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.474+0000 7f00c8a00640 1 -- 172.21.15.43:0/2771287707 shutdown_connections 2024-03-26T08:31:41.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.474+0000 7f00c8a00640 1 -- 172.21.15.43:0/2771287707 wait complete. 2024-03-26T08:31:41.488 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-26T08:31:41.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.727+0000 7f4233200640 1 -- 172.21.15.43:0/935660115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c102760 msgr2=0x7f422c104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:41.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.727+0000 7f4233200640 1 --2- 172.21.15.43:0/935660115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c102760 0x7f422c104bd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f421c009900 tx=0x7f421c02bc40 comp rx=0 tx=0).stop 2024-03-26T08:31:41.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.727+0000 7f4233200640 1 -- 172.21.15.43:0/935660115 shutdown_connections 2024-03-26T08:31:41.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.727+0000 7f4233200640 1 --2- 172.21.15.43:0/935660115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f422c105110 0x7f422c105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:41.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.727+0000 7f4233200640 1 --2- 172.21.15.43:0/935660115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c102760 0x7f422c104bd0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:41.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.727+0000 7f4233200640 1 --2- 172.21.15.43:0/935660115 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f422c0ffdf0 0x7f422c102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:41.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.727+0000 7f4233200640 1 -- 172.21.15.43:0/935660115 >> 172.21.15.43:0/935660115 conn(0x7f422c0fba10 msgr2=0x7f422c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:41.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.728+0000 7f4233200640 1 -- 172.21.15.43:0/935660115 shutdown_connections 2024-03-26T08:31:41.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.728+0000 7f4233200640 1 -- 172.21.15.43:0/935660115 wait complete. 2024-03-26T08:31:41.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.729+0000 7f4233200640 1 Processor -- start 2024-03-26T08:31:41.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.729+0000 7f4233200640 1 -- start start 2024-03-26T08:31:41.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.729+0000 7f4233200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f422c0ffdf0 0x7f422c19dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:41.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.729+0000 7f4233200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c102760 0x7f422c19e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:41.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.729+0000 7f4233200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f422c105110 0x7f422c1a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:41.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.729+0000 7f4233200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c1136a0 con 0x7f422c102760 2024-03-26T08:31:41.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.729+0000 7f4233200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c113520 con 0x7f422c105110 2024-03-26T08:31:41.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.729+0000 7f4233200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c113820 con 0x7f422c0ffdf0 2024-03-26T08:31:41.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.730+0000 7f4231400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c102760 0x7f422c19e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:41.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.730+0000 7f4231400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c102760 0x7f422c19e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51402/0 (socket says 172.21.15.43:51402) 2024-03-26T08:31:41.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.730+0000 7f4231400640 1 -- 172.21.15.43:0/843533466 learned_addr learned my addr 172.21.15.43:0/843533466 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:41.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.730+0000 7f4232800640 1 --2- 172.21.15.43:0/843533466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f422c105110 0x7f422c1a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:41.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.730+0000 7f4231e00640 1 --2- 172.21.15.43:0/843533466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f422c0ffdf0 0x7f422c19dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:41.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.730+0000 7f4231400640 1 -- 172.21.15.43:0/843533466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f422c0ffdf0 msgr2=0x7f422c19dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:41.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.730+0000 7f4231400640 1 --2- 172.21.15.43:0/843533466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f422c0ffdf0 0x7f422c19dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:41.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.730+0000 7f4231400640 1 -- 172.21.15.43:0/843533466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f422c105110 msgr2=0x7f422c1a5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:41.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.730+0000 7f4231400640 1 --2- 172.21.15.43:0/843533466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f422c105110 0x7f422c1a5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:41.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.730+0000 7f4231400640 1 -- 172.21.15.43:0/843533466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c1a5990 con 0x7f422c102760 2024-03-26T08:31:41.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.730+0000 7f4232800640 1 --2- 172.21.15.43:0/843533466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f422c105110 0x7f422c1a5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:31:41.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.731+0000 7f4231400640 1 --2- 172.21.15.43:0/843533466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c102760 0x7f422c19e2b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f421c006010 tx=0x7f421c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:41.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.731+0000 7f4222a00640 1 -- 172.21.15.43:0/843533466 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c043020 con 0x7f422c102760 2024-03-26T08:31:41.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.731+0000 7f4222a00640 1 -- 172.21.15.43:0/843533466 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f421c004aa0 con 0x7f422c102760 2024-03-26T08:31:41.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.731+0000 7f4233200640 1 -- 172.21.15.43:0/843533466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f422c1a5c20 con 0x7f422c102760 2024-03-26T08:31:41.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.731+0000 7f4222a00640 1 -- 172.21.15.43:0/843533466 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c003c60 con 0x7f422c102760 2024-03-26T08:31:41.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.731+0000 7f4233200640 1 -- 172.21.15.43:0/843533466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f422c1a6170 con 0x7f422c102760 2024-03-26T08:31:41.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.732+0000 7f4233200640 1 -- 172.21.15.43:0/843533466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f422c0fd500 con 0x7f422c102760 2024-03-26T08:31:41.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.734+0000 7f4222a00640 1 -- 172.21.15.43:0/843533466 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f421c046020 con 0x7f422c102760 2024-03-26T08:31:41.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.737+0000 7f4222a00640 1 --2- 172.21.15.43:0/843533466 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f42180a2800 0x7f42180a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:41.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.737+0000 7f4222a00640 1 -- 172.21.15.43:0/843533466 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f421c0f12b0 con 0x7f422c102760 2024-03-26T08:31:41.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.737+0000 7f4231e00640 1 --2- 172.21.15.43:0/843533466 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f42180a2800 0x7f42180a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:41.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.738+0000 7f4231e00640 1 --2- 172.21.15.43:0/843533466 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f42180a2800 0x7f42180a4cc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4224005c40 tx=0x7f4224005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:41.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.741+0000 7f4222a00640 1 -- 172.21.15.43:0/843533466 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f421c04d020 con 0x7f422c102760 2024-03-26T08:31:41.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:41 smithi183 ceph-mon[35818]: pgmap v384: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-03-26T08:31:41.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:41 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/406416872' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:31:41.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:41 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3264761428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:31:41.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:41 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2771287707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T08:31:41.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:41 smithi052 ceph-mon[35071]: pgmap v384: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-03-26T08:31:41.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:41 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/406416872' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:31:41.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:41 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3264761428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:31:41.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:41 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2771287707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T08:31:41.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:41 smithi043 ceph-mon[31929]: pgmap v384: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-03-26T08:31:41.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:41 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/406416872' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:31:41.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:41 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3264761428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:31:41.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:41 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2771287707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T08:31:41.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.957+0000 7f4233200640 1 -- 172.21.15.43:0/843533466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f422c02d070 con 0x7f422c102760 2024-03-26T08:31:41.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.958+0000 7f4222a00640 1 -- 172.21.15.43:0/843533466 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v92) v1 ==== 81+0+590 (secure 0 0 0) 0x7f421c0bdfb0 con 0x7f422c102760 2024-03-26T08:31:41.950 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6801/2446912488 2024-03-27T08:21:36.149092+0000 2024-03-26T08:31:41.951 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6800/2446912488 2024-03-27T08:21:36.149092+0000 2024-03-26T08:31:41.951 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/4240775295 2024-03-27T08:21:36.149092+0000 2024-03-26T08:31:41.951 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/510203140 2024-03-27T08:21:01.256355+0000 2024-03-26T08:31:41.951 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/4251957687 2024-03-27T08:21:36.149092+0000 2024-03-26T08:31:41.951 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6800/4108777098 2024-03-27T08:21:01.256355+0000 2024-03-26T08:31:41.951 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6801/4108777098 2024-03-27T08:21:01.256355+0000 2024-03-26T08:31:41.951 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/1885244665 2024-03-27T08:21:01.256355+0000 2024-03-26T08:31:41.951 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/298320867 2024-03-27T08:21:01.256355+0000 2024-03-26T08:31:41.951 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/1301092625 2024-03-27T08:21:36.149092+0000 2024-03-26T08:31:41.952 INFO:teuthology.orchestra.run.smithi043.stderr:listed 10 entries 2024-03-26T08:31:41.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.960+0000 7f4233200640 1 -- 172.21.15.43:0/843533466 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f42180a2800 msgr2=0x7f42180a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:41.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.960+0000 7f4233200640 1 --2- 172.21.15.43:0/843533466 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f42180a2800 0x7f42180a4cc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4224005c40 tx=0x7f4224005bd0 comp rx=0 tx=0).stop 2024-03-26T08:31:41.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.960+0000 7f4233200640 1 -- 172.21.15.43:0/843533466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c102760 msgr2=0x7f422c19e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:41.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.960+0000 7f4233200640 1 --2- 172.21.15.43:0/843533466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c102760 0x7f422c19e2b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f421c006010 tx=0x7f421c0042f0 comp rx=0 tx=0).stop 2024-03-26T08:31:41.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.960+0000 7f4233200640 1 -- 172.21.15.43:0/843533466 shutdown_connections 2024-03-26T08:31:41.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.960+0000 7f4233200640 1 --2- 172.21.15.43:0/843533466 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f42180a2800 0x7f42180a4cc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:41.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.960+0000 7f4233200640 1 --2- 172.21.15.43:0/843533466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f422c105110 0x7f422c1a5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:41.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.960+0000 7f4233200640 1 --2- 172.21.15.43:0/843533466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c102760 0x7f422c19e2b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:41.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.960+0000 7f4233200640 1 --2- 172.21.15.43:0/843533466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f422c0ffdf0 0x7f422c19dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:41.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.960+0000 7f4233200640 1 -- 172.21.15.43:0/843533466 >> 172.21.15.43:0/843533466 conn(0x7f422c0fba10 msgr2=0x7f422c100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:41.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.961+0000 7f4233200640 1 -- 172.21.15.43:0/843533466 shutdown_connections 2024-03-26T08:31:41.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:41.961+0000 7f4233200640 1 -- 172.21.15.43:0/843533466 wait complete. 2024-03-26T08:31:41.975 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-26T08:31:41.975 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-26T08:31:42.031 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:31:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.264+0000 7f799aa00640 1 -- 172.21.15.43:0/2457616436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79941026d0 msgr2=0x7f7994102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.264+0000 7f799aa00640 1 --2- 172.21.15.43:0/2457616436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79941026d0 0x7f7994102b50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7980009900 tx=0x7f798002bc50 comp rx=0 tx=0).stop 2024-03-26T08:31:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.264+0000 7f799aa00640 1 -- 172.21.15.43:0/2457616436 shutdown_connections 2024-03-26T08:31:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.264+0000 7f799aa00640 1 --2- 172.21.15.43:0/2457616436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7994103090 0x7f7994109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:42.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.264+0000 7f799aa00640 1 --2- 172.21.15.43:0/2457616436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79941026d0 0x7f7994102b50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:42.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.264+0000 7f799aa00640 1 --2- 172.21.15.43:0/2457616436 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79941014e0 0x7f7994101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:42.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.264+0000 7f799aa00640 1 -- 172.21.15.43:0/2457616436 >> 172.21.15.43:0/2457616436 conn(0x7f79940fca30 msgr2=0x7f79940fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:42.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.264+0000 7f799aa00640 1 -- 172.21.15.43:0/2457616436 shutdown_connections 2024-03-26T08:31:42.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.264+0000 7f799aa00640 1 -- 172.21.15.43:0/2457616436 wait complete. 2024-03-26T08:31:42.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.265+0000 7f799aa00640 1 Processor -- start 2024-03-26T08:31:42.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.265+0000 7f799aa00640 1 -- start start 2024-03-26T08:31:42.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.266+0000 7f799aa00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79941014e0 0x7f7994199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:42.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.266+0000 7f799aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79941026d0 0x7f7994199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:42.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.266+0000 7f799aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7994103090 0x7f79941a0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:42.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.266+0000 7f799aa00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799410d020 con 0x7f7994103090 2024-03-26T08:31:42.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.266+0000 7f799aa00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799410cea0 con 0x7f79941014e0 2024-03-26T08:31:42.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.266+0000 7f799aa00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799410d1a0 con 0x7f79941026d0 2024-03-26T08:31:42.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.266+0000 7f799a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7994103090 0x7f79941a0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:42.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.266+0000 7f799a000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7994103090 0x7f79941a0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51434/0 (socket says 172.21.15.43:51434) 2024-03-26T08:31:42.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.266+0000 7f799a000640 1 -- 172.21.15.43:0/3626613038 learned_addr learned my addr 172.21.15.43:0/3626613038 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:42.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.266+0000 7f7999600640 1 --2- 172.21.15.43:0/3626613038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79941014e0 0x7f7994199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:42.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.266+0000 7f7998c00640 1 --2- 172.21.15.43:0/3626613038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79941026d0 0x7f7994199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:42.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.266+0000 7f799a000640 1 -- 172.21.15.43:0/3626613038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79941026d0 msgr2=0x7f7994199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:42.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.266+0000 7f799a000640 1 --2- 172.21.15.43:0/3626613038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79941026d0 0x7f7994199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:42.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.267+0000 7f799a000640 1 -- 172.21.15.43:0/3626613038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79941014e0 msgr2=0x7f7994199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:42.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.267+0000 7f799a000640 1 --2- 172.21.15.43:0/3626613038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79941014e0 0x7f7994199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:42.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.267+0000 7f799a000640 1 -- 172.21.15.43:0/3626613038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79941a1520 con 0x7f7994103090 2024-03-26T08:31:42.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.267+0000 7f799a000640 1 --2- 172.21.15.43:0/3626613038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7994103090 0x7f79941a0eb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f798c00cce0 tx=0x7f798c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:42.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.267+0000 7f7992000640 1 -- 172.21.15.43:0/3626613038 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c00f450 con 0x7f7994103090 2024-03-26T08:31:42.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.267+0000 7f7992000640 1 -- 172.21.15.43:0/3626613038 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f798c00faa0 con 0x7f7994103090 2024-03-26T08:31:42.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.267+0000 7f799aa00640 1 -- 172.21.15.43:0/3626613038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79941a1810 con 0x7f7994103090 2024-03-26T08:31:42.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.267+0000 7f7992000640 1 -- 172.21.15.43:0/3626613038 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c018b70 con 0x7f7994103090 2024-03-26T08:31:42.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.267+0000 7f799aa00640 1 -- 172.21.15.43:0/3626613038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79941a1d60 con 0x7f7994103090 2024-03-26T08:31:42.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.269+0000 7f799aa00640 1 -- 172.21.15.43:0/3626613038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79941a1fc0 con 0x7f7994103090 2024-03-26T08:31:42.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.270+0000 7f7992000640 1 -- 172.21.15.43:0/3626613038 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f798c018d10 con 0x7f7994103090 2024-03-26T08:31:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.272+0000 7f7992000640 1 --2- 172.21.15.43:0/3626613038 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f797c0a2880 0x7f797c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.272+0000 7f7992000640 1 -- 172.21.15.43:0/3626613038 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f798c0cf260 con 0x7f7994103090 2024-03-26T08:31:42.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.272+0000 7f7999600640 1 --2- 172.21.15.43:0/3626613038 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f797c0a2880 0x7f797c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:42.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.273+0000 7f7999600640 1 --2- 172.21.15.43:0/3626613038 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f797c0a2880 0x7f797c0a4d40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7988005fd0 tx=0x7f7988005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.276+0000 7f7992000640 1 -- 172.21.15.43:0/3626613038 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f798c014030 con 0x7f7994103090 2024-03-26T08:31:42.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.486+0000 7f799aa00640 1 -- 172.21.15.43:0/3626613038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79941a2190 con 0x7f7994103090 2024-03-26T08:31:42.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.488+0000 7f7992000640 1 -- 172.21.15.43:0/3626613038 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24576 (secure 0 0 0) 0x7f798c09bf60 con 0x7f7994103090 2024-03-26T08:31:42.481 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:31:42.482 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":92,"fsid":"88759420-eb49-11ee-95d0-87774f69a715","created":"2024-03-26T08:20:27.817107+0000","modified":"2024-03-26T08:31:37.455519+0000","last_up_change":"2024-03-26T08:30:10.520976+0000","last_in_change":"2024-03-26T08:29:46.394360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:24:54.852427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:31:19.924902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:31:21.323881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T08:31:24.951015+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"28670673-9193-4fa3-9cc0-c1a5a6b27284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6803","nonce":936601146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6805","nonce":936601146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6809","nonce":936601146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6807","nonce":936601146}]},"public_addr":"172.21.15.43:6803/936601146","cluster_addr":"172.21.15.43:6805/936601146","heartbeat_back_addr":"172.21.15.43:6809/936601146","heartbeat_front_addr":"172.21.15.43:6807/936601146","state":["exists","up"]},{"osd":1,"uuid":"fa8ad6b6-277a-42b8-b0f8-b36fc219de2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6801","nonce":3268412328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6803","nonce":3268412328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6807","nonce":3268412328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6805","nonce":3268412328}]},"public_addr":"172.21.15.52:6801/3268412328","cluster_addr":"172.21.15.52:6803/3268412328","heartbeat_back_addr":"172.21.15.52:6807/3268412328","heartbeat_front_addr":"172.21.15.52:6805/3268412328","state":["exists","up"]},{"osd":2,"uuid":"ec0039c3-f9bd-419a-81de-fa4418359808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6801","nonce":2103180908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6803","nonce":2103180908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6807","nonce":2103180908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6805","nonce":2103180908}]},"public_addr":"172.21.15.183:6801/2103180908","cluster_addr":"172.21.15.183:6803/2103180908","heartbeat_back_addr":"172.21.15.183:6807/2103180908","heartbeat_front_addr":"172.21.15.183:6805/2103180908","state":["exists","up"]},{"osd":3,"uuid":"e7dcbf00-6586-4664-a69e-3cb0df8dfd30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6811","nonce":3722846033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6813","nonce":3722846033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6817","nonce":3722846033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6815","nonce":3722846033}]},"public_addr":"172.21.15.43:6811/3722846033","cluster_addr":"172.21.15.43:6813/3722846033","heartbeat_back_addr":"172.21.15.43:6817/3722846033","heartbeat_front_addr":"172.21.15.43:6815/3722846033","state":["exists","up"]},{"osd":4,"uuid":"db089a49-5e98-432d-9687-cdc044114c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6809","nonce":4028436523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6811","nonce":4028436523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6815","nonce":4028436523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6813","nonce":4028436523}]},"public_addr":"172.21.15.52:6809/4028436523","cluster_addr":"172.21.15.52:6811/4028436523","heartbeat_back_addr":"172.21.15.52:6815/4028436523","heartbeat_front_addr":"172.21.15.52:6813/4028436523","state":["exists","up"]},{"osd":5,"uuid":"556ac804-02ea-423b-a599-55baf5be4ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6809","nonce":787132252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6811","nonce":787132252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6815","nonce":787132252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6813","nonce":787132252}]},"public_addr":"172.21.15.183:6809/787132252","cluster_addr":"172.21.15.183:6811/787132252","heartbeat_back_addr":"172.21.15.183:6815/787132252","heartbeat_front_addr":"172.21.15.183:6813/787132252","state":["exists","up"]},{"osd":6,"uuid":"3de2a49d-6cef-4580-9d8a-f90cdfcecd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6819","nonce":4206861158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6821","nonce":4206861158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6825","nonce":4206861158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6823","nonce":4206861158}]},"public_addr":"172.21.15.43:6819/4206861158","cluster_addr":"172.21.15.43:6821/4206861158","heartbeat_back_addr":"172.21.15.43:6825/4206861158","heartbeat_front_addr":"172.21.15.43:6823/4206861158","state":["exists","up"]},{"osd":7,"uuid":"a3b832f7-7d7d-4a62-b679-db6da98275d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6817","nonce":3452064146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6819","nonce":3452064146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6823","nonce":3452064146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6821","nonce":3452064146}]},"public_addr":"172.21.15.52:6817/3452064146","cluster_addr":"172.21.15.52:6819/3452064146","heartbeat_back_addr":"172.21.15.52:6823/3452064146","heartbeat_front_addr":"172.21.15.52:6821/3452064146","state":["exists","up"]},{"osd":8,"uuid":"ddc6b07f-d703-414d-a59a-02316376b13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6817","nonce":1023131725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6819","nonce":1023131725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6823","nonce":1023131725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6821","nonce":1023131725}]},"public_addr":"172.21.15.183:6817/1023131725","cluster_addr":"172.21.15.183:6819/1023131725","heartbeat_back_addr":"172.21.15.183:6823/1023131725","heartbeat_front_addr":"172.21.15.183:6821/1023131725","state":["exists","up"]},{"osd":9,"uuid":"ac16bb0e-6232-45c8-b16a-4d878874c588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6827","nonce":3935195657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6829","nonce":3935195657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6833","nonce":3935195657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6831","nonce":3935195657}]},"public_addr":"172.21.15.43:6827/3935195657","cluster_addr":"172.21.15.43:6829/3935195657","heartbeat_back_addr":"172.21.15.43:6833/3935195657","heartbeat_front_addr":"172.21.15.43:6831/3935195657","state":["exists","up"]},{"osd":10,"uuid":"fd839188-f398-4cff-aa7b-36522ff1a8a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6825","nonce":377628320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6827","nonce":377628320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6831","nonce":377628320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6829","nonce":377628320}]},"public_addr":"172.21.15.52:6825/377628320","cluster_addr":"172.21.15.52:6827/377628320","heartbeat_back_addr":"172.21.15.52:6831/377628320","heartbeat_front_addr":"172.21.15.52:6829/377628320","state":["exists","up"]},{"osd":11,"uuid":"a40282a0-0c64-49ee-b64e-6f3f8119e827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6825","nonce":1373086433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6827","nonce":1373086433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6831","nonce":1373086433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6829","nonce":1373086433}]},"public_addr":"172.21.15.183:6825/1373086433","cluster_addr":"172.21.15.183:6827/1373086433","heartbeat_back_addr":"172.21.15.183:6831/1373086433","heartbeat_front_addr":"172.21.15.183:6829/1373086433","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:23:43.478905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:17.573532+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:50.286610+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:25:27.407663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:03.081294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:36.057416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:12.062013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:47.032271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:21.023201+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:57.364739+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:29:34.039995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:30:08.827577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6801/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/4240775295":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/510203140":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/4251957687":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:6801/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1885244665":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/298320867":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1301092625":"2024-03-27T08:21:36.149092+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:31:42.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.491+0000 7f799aa00640 1 -- 172.21.15.43:0/3626613038 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f797c0a2880 msgr2=0x7f797c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:42.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.491+0000 7f799aa00640 1 --2- 172.21.15.43:0/3626613038 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f797c0a2880 0x7f797c0a4d40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7988005fd0 tx=0x7f7988005850 comp rx=0 tx=0).stop 2024-03-26T08:31:42.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.491+0000 7f799aa00640 1 -- 172.21.15.43:0/3626613038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7994103090 msgr2=0x7f79941a0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:42.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.491+0000 7f799aa00640 1 --2- 172.21.15.43:0/3626613038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7994103090 0x7f79941a0eb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f798c00cce0 tx=0x7f798c00a620 comp rx=0 tx=0).stop 2024-03-26T08:31:42.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.491+0000 7f799aa00640 1 -- 172.21.15.43:0/3626613038 shutdown_connections 2024-03-26T08:31:42.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.491+0000 7f799aa00640 1 --2- 172.21.15.43:0/3626613038 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f797c0a2880 0x7f797c0a4d40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:42.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.491+0000 7f799aa00640 1 --2- 172.21.15.43:0/3626613038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7994103090 0x7f79941a0eb0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:42.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.491+0000 7f799aa00640 1 --2- 172.21.15.43:0/3626613038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79941026d0 0x7f7994199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:42.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.491+0000 7f799aa00640 1 --2- 172.21.15.43:0/3626613038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79941014e0 0x7f7994199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:42.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.491+0000 7f799aa00640 1 -- 172.21.15.43:0/3626613038 >> 172.21.15.43:0/3626613038 conn(0x7f79940fca30 msgr2=0x7f79940738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:42.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.491+0000 7f799aa00640 1 -- 172.21.15.43:0/3626613038 shutdown_connections 2024-03-26T08:31:42.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.491+0000 7f799aa00640 1 -- 172.21.15.43:0/3626613038 wait complete. 2024-03-26T08:31:42.505 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T08:24:54.852427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T08:31:19.924902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T08:31:21.323881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T08:31:24.951015+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T08:31:42.505 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T08:31:42.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.735+0000 7f4cf2a00640 1 -- 172.21.15.43:0/607890191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cec1015a0 msgr2=0x7f4cec1019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:42.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.735+0000 7f4cf2a00640 1 --2- 172.21.15.43:0/607890191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cec1015a0 0x7f4cec1019c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc009ca0 tx=0x7f4cdc02c0f0 comp rx=0 tx=0).stop 2024-03-26T08:31:42.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.736+0000 7f4cf2a00640 1 -- 172.21.15.43:0/607890191 shutdown_connections 2024-03-26T08:31:42.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.736+0000 7f4cf2a00640 1 --2- 172.21.15.43:0/607890191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cec103150 0x7f4cec1099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:42.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.736+0000 7f4cf2a00640 1 --2- 172.21.15.43:0/607890191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cec102790 0x7f4cec102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:42.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.736+0000 7f4cf2a00640 1 --2- 172.21.15.43:0/607890191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cec1015a0 0x7f4cec1019c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:42.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.736+0000 7f4cf2a00640 1 -- 172.21.15.43:0/607890191 >> 172.21.15.43:0/607890191 conn(0x7f4cec0fcb10 msgr2=0x7f4cec0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:42.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.736+0000 7f4cf2a00640 1 -- 172.21.15.43:0/607890191 shutdown_connections 2024-03-26T08:31:42.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.736+0000 7f4cf2a00640 1 -- 172.21.15.43:0/607890191 wait complete. 2024-03-26T08:31:42.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.737+0000 7f4cf2a00640 1 Processor -- start 2024-03-26T08:31:42.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.737+0000 7f4cf2a00640 1 -- start start 2024-03-26T08:31:42.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.737+0000 7f4cf2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cec1015a0 0x7f4cec199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:42.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.737+0000 7f4cf2a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cec102790 0x7f4cec199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:42.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.737+0000 7f4cf2a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cec103150 0x7f4cec1a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:42.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.737+0000 7f4cf2a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec10d0e0 con 0x7f4cec1015a0 2024-03-26T08:31:42.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.737+0000 7f4cf2a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec10cf60 con 0x7f4cec102790 2024-03-26T08:31:42.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.737+0000 7f4cf2a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec10d260 con 0x7f4cec103150 2024-03-26T08:31:42.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.737+0000 7f4cf1600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cec1015a0 0x7f4cec199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:42.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.737+0000 7f4cf1600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cec1015a0 0x7f4cec199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51440/0 (socket says 172.21.15.43:51440) 2024-03-26T08:31:42.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.738+0000 7f4cf1600640 1 -- 172.21.15.43:0/3928407713 learned_addr learned my addr 172.21.15.43:0/3928407713 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:42.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.738+0000 7f4cf0c00640 1 --2- 172.21.15.43:0/3928407713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cec102790 0x7f4cec199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:42.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.738+0000 7f4cf2000640 1 --2- 172.21.15.43:0/3928407713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cec103150 0x7f4cec1a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:42.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.738+0000 7f4cf1600640 1 -- 172.21.15.43:0/3928407713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cec103150 msgr2=0x7f4cec1a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:42.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.738+0000 7f4cf1600640 1 --2- 172.21.15.43:0/3928407713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cec103150 0x7f4cec1a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:42.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.738+0000 7f4cf1600640 1 -- 172.21.15.43:0/3928407713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cec102790 msgr2=0x7f4cec199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:42.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.738+0000 7f4cf1600640 1 --2- 172.21.15.43:0/3928407713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cec102790 0x7f4cec199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:42.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.738+0000 7f4cf1600640 1 -- 172.21.15.43:0/3928407713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cec1a1620 con 0x7f4cec1015a0 2024-03-26T08:31:42.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.738+0000 7f4cf0c00640 1 --2- 172.21.15.43:0/3928407713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cec102790 0x7f4cec199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:31:42.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.738+0000 7f4cf1600640 1 --2- 172.21.15.43:0/3928407713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cec1015a0 0x7f4cec199a00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc032380 tx=0x7f4cdc032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:42.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.739+0000 7f4cea000640 1 -- 172.21.15.43:0/3928407713 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc041020 con 0x7f4cec1015a0 2024-03-26T08:31:42.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.739+0000 7f4cea000640 1 -- 172.21.15.43:0/3928407713 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4cdc003e20 con 0x7f4cec1015a0 2024-03-26T08:31:42.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.739+0000 7f4cea000640 1 -- 172.21.15.43:0/3928407713 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc039040 con 0x7f4cec1015a0 2024-03-26T08:31:42.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.739+0000 7f4cf2a00640 1 -- 172.21.15.43:0/3928407713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cec1a18b0 con 0x7f4cec1015a0 2024-03-26T08:31:42.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.739+0000 7f4cf2a00640 1 -- 172.21.15.43:0/3928407713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cec1a1d50 con 0x7f4cec1015a0 2024-03-26T08:31:42.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.740+0000 7f4cf2a00640 1 -- 172.21.15.43:0/3928407713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cec1a2010 con 0x7f4cec1015a0 2024-03-26T08:31:42.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.742+0000 7f4cea000640 1 -- 172.21.15.43:0/3928407713 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f4cdc044020 con 0x7f4cec1015a0 2024-03-26T08:31:42.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.744+0000 7f4cea000640 1 --2- 172.21.15.43:0/3928407713 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f4cd00a2880 0x7f4cd00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:42.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.745+0000 7f4cea000640 1 -- 172.21.15.43:0/3928407713 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f4cdc0f1110 con 0x7f4cec1015a0 2024-03-26T08:31:42.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.745+0000 7f4cf0c00640 1 --2- 172.21.15.43:0/3928407713 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f4cd00a2880 0x7f4cd00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:42.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.746+0000 7f4cf0c00640 1 --2- 172.21.15.43:0/3928407713 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f4cd00a2880 0x7f4cd00a4d40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4cec19af10 tx=0x7f4cd8005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:42.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.749+0000 7f4cea000640 1 -- 172.21.15.43:0/3928407713 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4cdc04b020 con 0x7f4cec1015a0 2024-03-26T08:31:42.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:42 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/843533466' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T08:31:42.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:42 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3626613038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:31:42.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:42 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/843533466' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T08:31:42.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:42 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3626613038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:31:42.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:42 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/843533466' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T08:31:42.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:42 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3626613038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:31:42.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.949+0000 7f4cf2a00640 1 -- 172.21.15.43:0/3928407713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4cec1a23a0 con 0x7f4cec1015a0 2024-03-26T08:31:42.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.950+0000 7f4cea000640 1 -- 172.21.15.43:0/3928407713 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4cdc03b2f0 con 0x7f4cec1015a0 2024-03-26T08:31:42.943 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-26T08:31:42.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.953+0000 7f4cf2a00640 1 -- 172.21.15.43:0/3928407713 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f4cd00a2880 msgr2=0x7f4cd00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:42.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.953+0000 7f4cf2a00640 1 --2- 172.21.15.43:0/3928407713 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f4cd00a2880 0x7f4cd00a4d40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4cec19af10 tx=0x7f4cd8005e50 comp rx=0 tx=0).stop 2024-03-26T08:31:42.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.953+0000 7f4cf2a00640 1 -- 172.21.15.43:0/3928407713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cec1015a0 msgr2=0x7f4cec199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:42.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.953+0000 7f4cf2a00640 1 --2- 172.21.15.43:0/3928407713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cec1015a0 0x7f4cec199a00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc032380 tx=0x7f4cdc032820 comp rx=0 tx=0).stop 2024-03-26T08:31:42.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.953+0000 7f4cf2a00640 1 -- 172.21.15.43:0/3928407713 shutdown_connections 2024-03-26T08:31:42.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.953+0000 7f4cf2a00640 1 --2- 172.21.15.43:0/3928407713 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f4cd00a2880 0x7f4cd00a4d40 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:42.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.953+0000 7f4cf2a00640 1 --2- 172.21.15.43:0/3928407713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cec103150 0x7f4cec1a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:42.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.953+0000 7f4cf2a00640 1 --2- 172.21.15.43:0/3928407713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4cec102790 0x7f4cec199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:42.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.953+0000 7f4cf2a00640 1 --2- 172.21.15.43:0/3928407713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4cec1015a0 0x7f4cec199a00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:42.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.953+0000 7f4cf2a00640 1 -- 172.21.15.43:0/3928407713 >> 172.21.15.43:0/3928407713 conn(0x7f4cec0fcb10 msgr2=0x7f4cec0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:42.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.953+0000 7f4cf2a00640 1 -- 172.21.15.43:0/3928407713 shutdown_connections 2024-03-26T08:31:42.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:42.953+0000 7f4cf2a00640 1 -- 172.21.15.43:0/3928407713 wait complete. 2024-03-26T08:31:42.967 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T08:31:43.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.203+0000 7fc6d3200640 1 -- 172.21.15.43:0/2010070052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6d40740a0 msgr2=0x7fc6d4074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:43.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.203+0000 7fc6d3200640 1 --2- 172.21.15.43:0/2010070052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6d40740a0 0x7fc6d4074500 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009900 tx=0x7fc6bc02bcd0 comp rx=0 tx=0).stop 2024-03-26T08:31:43.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.203+0000 7fc6d3200640 1 -- 172.21.15.43:0/2010070052 shutdown_connections 2024-03-26T08:31:43.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.203+0000 7fc6d3200640 1 --2- 172.21.15.43:0/2010070052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6d4109430 0x7fc6d410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:43.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.203+0000 7fc6d3200640 1 --2- 172.21.15.43:0/2010070052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6d40740a0 0x7fc6d4074500 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:43.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.203+0000 7fc6d3200640 1 --2- 172.21.15.43:0/2010070052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6d4075680 0x7fc6d4073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:43.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.203+0000 7fc6d3200640 1 -- 172.21.15.43:0/2010070052 >> 172.21.15.43:0/2010070052 conn(0x7fc6d40fcb10 msgr2=0x7fc6d40fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:43.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.203+0000 7fc6d3200640 1 -- 172.21.15.43:0/2010070052 shutdown_connections 2024-03-26T08:31:43.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.203+0000 7fc6d3200640 1 -- 172.21.15.43:0/2010070052 wait complete. 2024-03-26T08:31:43.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.204+0000 7fc6d3200640 1 Processor -- start 2024-03-26T08:31:43.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.204+0000 7fc6d3200640 1 -- start start 2024-03-26T08:31:43.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.205+0000 7fc6d3200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6d40740a0 0x7fc6d4199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:43.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.205+0000 7fc6d3200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6d4075680 0x7fc6d4199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:43.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.205+0000 7fc6d3200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6d4109430 0x7fc6d41a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:43.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.205+0000 7fc6d3200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d410f2b0 con 0x7fc6d4109430 2024-03-26T08:31:43.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.205+0000 7fc6d3200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d410f130 con 0x7fc6d40740a0 2024-03-26T08:31:43.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.205+0000 7fc6d3200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d410f430 con 0x7fc6d4075680 2024-03-26T08:31:43.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.205+0000 7fc6d2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6d4109430 0x7fc6d41a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:43.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.205+0000 7fc6d2800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6d4109430 0x7fc6d41a0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51456/0 (socket says 172.21.15.43:51456) 2024-03-26T08:31:43.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.205+0000 7fc6d2800640 1 -- 172.21.15.43:0/4170343164 learned_addr learned my addr 172.21.15.43:0/4170343164 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:43.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.205+0000 7fc6d1e00640 1 --2- 172.21.15.43:0/4170343164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6d40740a0 0x7fc6d4199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:43.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.205+0000 7fc6c9400640 1 --2- 172.21.15.43:0/4170343164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6d4075680 0x7fc6d4199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:43.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.205+0000 7fc6d2800640 1 -- 172.21.15.43:0/4170343164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6d4075680 msgr2=0x7fc6d4199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:43.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.205+0000 7fc6d2800640 1 --2- 172.21.15.43:0/4170343164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6d4075680 0x7fc6d4199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:43.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.205+0000 7fc6d2800640 1 -- 172.21.15.43:0/4170343164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6d40740a0 msgr2=0x7fc6d4199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:43.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.205+0000 7fc6d2800640 1 --2- 172.21.15.43:0/4170343164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6d40740a0 0x7fc6d4199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:43.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.205+0000 7fc6d2800640 1 -- 172.21.15.43:0/4170343164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d41a1560 con 0x7fc6d4109430 2024-03-26T08:31:43.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.206+0000 7fc6d2800640 1 --2- 172.21.15.43:0/4170343164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6d4109430 0x7fc6d41a0ef0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc00eaa0 tx=0x7fc6cc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:43.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.206+0000 7fc6cb400640 1 -- 172.21.15.43:0/4170343164 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc00cdb0 con 0x7fc6d4109430 2024-03-26T08:31:43.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.206+0000 7fc6cb400640 1 -- 172.21.15.43:0/4170343164 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc6cc005020 con 0x7fc6d4109430 2024-03-26T08:31:43.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.206+0000 7fc6d3200640 1 -- 172.21.15.43:0/4170343164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d41a1850 con 0x7fc6d4109430 2024-03-26T08:31:43.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.206+0000 7fc6cb400640 1 -- 172.21.15.43:0/4170343164 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc0116d0 con 0x7fc6d4109430 2024-03-26T08:31:43.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.206+0000 7fc6d3200640 1 -- 172.21.15.43:0/4170343164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d41a1da0 con 0x7fc6d4109430 2024-03-26T08:31:43.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.207+0000 7fc6d3200640 1 -- 172.21.15.43:0/4170343164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6d41081f0 con 0x7fc6d4109430 2024-03-26T08:31:43.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.209+0000 7fc6cb400640 1 -- 172.21.15.43:0/4170343164 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fc6cc0051d0 con 0x7fc6d4109430 2024-03-26T08:31:43.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.211+0000 7fc6cb400640 1 --2- 172.21.15.43:0/4170343164 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fc6b40a2880 0x7fc6b40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:43.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.211+0000 7fc6cb400640 1 -- 172.21.15.43:0/4170343164 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7fc6cc0cf400 con 0x7fc6d4109430 2024-03-26T08:31:43.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.211+0000 7fc6d1e00640 1 --2- 172.21.15.43:0/4170343164 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fc6b40a2880 0x7fc6b40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:43.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.212+0000 7fc6d1e00640 1 --2- 172.21.15.43:0/4170343164 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fc6b40a2880 0x7fc6b40a4d40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4005e10 tx=0x7fc6c4005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:43.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.214+0000 7fc6cb400640 1 -- 172.21.15.43:0/4170343164 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc6cc016030 con 0x7fc6d4109430 2024-03-26T08:31:43.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.417+0000 7fc6d3200640 1 -- 172.21.15.43:0/4170343164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc6d402d200 con 0x7fc6d4109430 2024-03-26T08:31:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.418+0000 7fc6cb400640 1 -- 172.21.15.43:0/4170343164 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc6cc09c180 con 0x7fc6d4109430 2024-03-26T08:31:43.415 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-26T08:31:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.421+0000 7fc6d3200640 1 -- 172.21.15.43:0/4170343164 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fc6b40a2880 msgr2=0x7fc6b40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.421+0000 7fc6d3200640 1 --2- 172.21.15.43:0/4170343164 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fc6b40a2880 0x7fc6b40a4d40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4005e10 tx=0x7fc6c4005da0 comp rx=0 tx=0).stop 2024-03-26T08:31:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.421+0000 7fc6d3200640 1 -- 172.21.15.43:0/4170343164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6d4109430 msgr2=0x7fc6d41a0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.421+0000 7fc6d3200640 1 --2- 172.21.15.43:0/4170343164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6d4109430 0x7fc6d41a0ef0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc00eaa0 tx=0x7fc6cc00ef60 comp rx=0 tx=0).stop 2024-03-26T08:31:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.421+0000 7fc6d3200640 1 -- 172.21.15.43:0/4170343164 shutdown_connections 2024-03-26T08:31:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.421+0000 7fc6d3200640 1 --2- 172.21.15.43:0/4170343164 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fc6b40a2880 0x7fc6b40a4d40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.421+0000 7fc6d3200640 1 --2- 172.21.15.43:0/4170343164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6d4109430 0x7fc6d41a0ef0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.421+0000 7fc6d3200640 1 --2- 172.21.15.43:0/4170343164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6d4075680 0x7fc6d4199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.421+0000 7fc6d3200640 1 --2- 172.21.15.43:0/4170343164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6d40740a0 0x7fc6d4199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.421+0000 7fc6d3200640 1 -- 172.21.15.43:0/4170343164 >> 172.21.15.43:0/4170343164 conn(0x7fc6d40fcb10 msgr2=0x7fc6d41053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.421+0000 7fc6d3200640 1 -- 172.21.15.43:0/4170343164 shutdown_connections 2024-03-26T08:31:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.421+0000 7fc6d3200640 1 -- 172.21.15.43:0/4170343164 wait complete. 2024-03-26T08:31:43.433 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T08:31:43.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.658+0000 7fb507800640 1 -- 172.21.15.43:0/2572110889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb500108eb0 msgr2=0x7fb500109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:43.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.658+0000 7fb507800640 1 --2- 172.21.15.43:0/2572110889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb500108eb0 0x7fb500109330 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009900 tx=0x7fb4f002bc50 comp rx=0 tx=0).stop 2024-03-26T08:31:43.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.658+0000 7fb507800640 1 -- 172.21.15.43:0/2572110889 shutdown_connections 2024-03-26T08:31:43.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.658+0000 7fb507800640 1 --2- 172.21.15.43:0/2572110889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb500109870 0x7fb500110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:43.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.658+0000 7fb507800640 1 --2- 172.21.15.43:0/2572110889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb500108eb0 0x7fb500109330 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:43.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.658+0000 7fb507800640 1 --2- 172.21.15.43:0/2572110889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb500107cc0 0x7fb5001080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:43.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.658+0000 7fb507800640 1 -- 172.21.15.43:0/2572110889 >> 172.21.15.43:0/2572110889 conn(0x7fb500076c60 msgr2=0x7fb5000790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:43.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.658+0000 7fb507800640 1 -- 172.21.15.43:0/2572110889 shutdown_connections 2024-03-26T08:31:43.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.658+0000 7fb507800640 1 -- 172.21.15.43:0/2572110889 wait complete. 2024-03-26T08:31:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.659+0000 7fb507800640 1 Processor -- start 2024-03-26T08:31:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.660+0000 7fb507800640 1 -- start start 2024-03-26T08:31:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.660+0000 7fb507800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb500107cc0 0x7fb50019df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.660+0000 7fb507800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb500108eb0 0x7fb50019e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.660+0000 7fb507800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb500109870 0x7fb5001a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.660+0000 7fb507800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb500113850 con 0x7fb500107cc0 2024-03-26T08:31:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.660+0000 7fb507800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5001136d0 con 0x7fb500108eb0 2024-03-26T08:31:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.660+0000 7fb507800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5001139d0 con 0x7fb500109870 2024-03-26T08:31:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.660+0000 7fb504e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb500107cc0 0x7fb50019df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.660+0000 7fb504e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb500107cc0 0x7fb50019df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51470/0 (socket says 172.21.15.43:51470) 2024-03-26T08:31:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.660+0000 7fb504e00640 1 -- 172.21.15.43:0/2903050058 learned_addr learned my addr 172.21.15.43:0/2903050058 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.661+0000 7fb4f7e00640 1 --2- 172.21.15.43:0/2903050058 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb500108eb0 0x7fb50019e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.661+0000 7fb504e00640 1 -- 172.21.15.43:0/2903050058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb500109870 msgr2=0x7fb5001a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:43.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.661+0000 7fb505800640 1 --2- 172.21.15.43:0/2903050058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb500109870 0x7fb5001a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:43.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.661+0000 7fb504e00640 1 --2- 172.21.15.43:0/2903050058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb500109870 0x7fb5001a5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:43.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.661+0000 7fb504e00640 1 -- 172.21.15.43:0/2903050058 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb500108eb0 msgr2=0x7fb50019e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:43.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.661+0000 7fb504e00640 1 --2- 172.21.15.43:0/2903050058 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb500108eb0 0x7fb50019e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:43.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.661+0000 7fb504e00640 1 -- 172.21.15.43:0/2903050058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5001a5b70 con 0x7fb500107cc0 2024-03-26T08:31:43.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.661+0000 7fb4f7e00640 1 --2- 172.21.15.43:0/2903050058 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb500108eb0 0x7fb50019e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:31:43.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.661+0000 7fb504e00640 1 --2- 172.21.15.43:0/2903050058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb500107cc0 0x7fb50019df50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb4e800c970 tx=0x7fb4e800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:43.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.661+0000 7fb4fe000640 1 -- 172.21.15.43:0/2903050058 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8004220 con 0x7fb500107cc0 2024-03-26T08:31:43.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.661+0000 7fb4fe000640 1 -- 172.21.15.43:0/2903050058 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb4e80043c0 con 0x7fb500107cc0 2024-03-26T08:31:43.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.661+0000 7fb4fe000640 1 -- 172.21.15.43:0/2903050058 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e800f450 con 0x7fb500107cc0 2024-03-26T08:31:43.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.661+0000 7fb507800640 1 -- 172.21.15.43:0/2903050058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5001a5e60 con 0x7fb500107cc0 2024-03-26T08:31:43.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.662+0000 7fb507800640 1 -- 172.21.15.43:0/2903050058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5001a63a0 con 0x7fb500107cc0 2024-03-26T08:31:43.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.663+0000 7fb507800640 1 -- 172.21.15.43:0/2903050058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb50010f060 con 0x7fb500107cc0 2024-03-26T08:31:43.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.665+0000 7fb4fe000640 1 -- 172.21.15.43:0/2903050058 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb4e80079b0 con 0x7fb500107cc0 2024-03-26T08:31:43.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.666+0000 7fb4fe000640 1 --2- 172.21.15.43:0/2903050058 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb4e00a2880 0x7fb4e00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:43.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.666+0000 7fb4fe000640 1 -- 172.21.15.43:0/2903050058 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7fb4e80cef80 con 0x7fb500107cc0 2024-03-26T08:31:43.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.666+0000 7fb4f7e00640 1 --2- 172.21.15.43:0/2903050058 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb4e00a2880 0x7fb4e00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:43.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.667+0000 7fb4f7e00640 1 --2- 172.21.15.43:0/2903050058 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb4e00a2880 0x7fb4e00a4d40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb4f00096c0 tx=0x7fb4f0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:43.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.668+0000 7fb4fe000640 1 -- 172.21.15.43:0/2903050058 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb4e8014030 con 0x7fb500107cc0 2024-03-26T08:31:43.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:43 smithi183 ceph-mon[35818]: pgmap v385: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-03-26T08:31:43.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:43 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3928407713' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:31:43.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:43 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/4170343164' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:31:43.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:43 smithi052 ceph-mon[35071]: pgmap v385: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-03-26T08:31:43.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:43 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3928407713' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:31:43.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:43 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/4170343164' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:31:43.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.872+0000 7fb507800640 1 -- 172.21.15.43:0/2903050058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb50002d070 con 0x7fb500107cc0 2024-03-26T08:31:43.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.873+0000 7fb4fe000640 1 -- 172.21.15.43:0/2903050058 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb4e809bd00 con 0x7fb500107cc0 2024-03-26T08:31:43.866 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-26T08:31:43.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.876+0000 7fb507800640 1 -- 172.21.15.43:0/2903050058 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb4e00a2880 msgr2=0x7fb4e00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:43.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.876+0000 7fb507800640 1 --2- 172.21.15.43:0/2903050058 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb4e00a2880 0x7fb4e00a4d40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb4f00096c0 tx=0x7fb4f0000bc0 comp rx=0 tx=0).stop 2024-03-26T08:31:43.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.876+0000 7fb507800640 1 -- 172.21.15.43:0/2903050058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb500107cc0 msgr2=0x7fb50019df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:43.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.876+0000 7fb507800640 1 --2- 172.21.15.43:0/2903050058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb500107cc0 0x7fb50019df50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb4e800c970 tx=0x7fb4e800ce30 comp rx=0 tx=0).stop 2024-03-26T08:31:43.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.876+0000 7fb507800640 1 -- 172.21.15.43:0/2903050058 shutdown_connections 2024-03-26T08:31:43.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.876+0000 7fb507800640 1 --2- 172.21.15.43:0/2903050058 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb4e00a2880 0x7fb4e00a4d40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:43.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.876+0000 7fb507800640 1 --2- 172.21.15.43:0/2903050058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb500109870 0x7fb5001a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:43.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.876+0000 7fb507800640 1 --2- 172.21.15.43:0/2903050058 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb500108eb0 0x7fb50019e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:43.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.876+0000 7fb507800640 1 --2- 172.21.15.43:0/2903050058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb500107cc0 0x7fb50019df50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:43.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.876+0000 7fb507800640 1 -- 172.21.15.43:0/2903050058 >> 172.21.15.43:0/2903050058 conn(0x7fb500076c60 msgr2=0x7fb50010c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:43.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.876+0000 7fb507800640 1 -- 172.21.15.43:0/2903050058 shutdown_connections 2024-03-26T08:31:43.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:43.876+0000 7fb507800640 1 -- 172.21.15.43:0/2903050058 wait complete. 2024-03-26T08:31:43.885 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:43 smithi043 ceph-mon[31929]: pgmap v385: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-03-26T08:31:43.885 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:43 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3928407713' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:31:43.885 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:43 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/4170343164' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:31:43.889 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T08:31:44.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.114+0000 7fdaf6200640 1 -- 172.21.15.43:0/1732287071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdaf0108db0 msgr2=0x7fdaf0109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.114+0000 7fdaf6200640 1 --2- 172.21.15.43:0/1732287071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdaf0108db0 0x7fdaf0109230 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdadc009900 tx=0x7fdadc02bc40 comp rx=0 tx=0).stop 2024-03-26T08:31:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.114+0000 7fdaf6200640 1 -- 172.21.15.43:0/1732287071 shutdown_connections 2024-03-26T08:31:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.114+0000 7fdaf6200640 1 --2- 172.21.15.43:0/1732287071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdaf0109770 0x7fdaf0110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.114+0000 7fdaf6200640 1 --2- 172.21.15.43:0/1732287071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdaf0108db0 0x7fdaf0109230 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.114+0000 7fdaf6200640 1 --2- 172.21.15.43:0/1732287071 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaf0107bc0 0x7fdaf0107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.114+0000 7fdaf6200640 1 -- 172.21.15.43:0/1732287071 >> 172.21.15.43:0/1732287071 conn(0x7fdaf0076b70 msgr2=0x7fdaf0078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.114+0000 7fdaf6200640 1 -- 172.21.15.43:0/1732287071 shutdown_connections 2024-03-26T08:31:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.115+0000 7fdaf6200640 1 -- 172.21.15.43:0/1732287071 wait complete. 2024-03-26T08:31:44.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.116+0000 7fdaf6200640 1 Processor -- start 2024-03-26T08:31:44.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.116+0000 7fdaf6200640 1 -- start start 2024-03-26T08:31:44.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.116+0000 7fdaf6200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaf0107bc0 0x7fdaf019ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:44.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.116+0000 7fdaf6200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdaf0108db0 0x7fdaf019e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:44.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.116+0000 7fdaf6200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdaf0109770 0x7fdaf01a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:44.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.116+0000 7fdaf6200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf0113720 con 0x7fdaf0109770 2024-03-26T08:31:44.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.116+0000 7fdaf6200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf01135a0 con 0x7fdaf0107bc0 2024-03-26T08:31:44.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.116+0000 7fdaf6200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf01138a0 con 0x7fdaf0108db0 2024-03-26T08:31:44.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.116+0000 7fdaf5800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdaf0109770 0x7fdaf01a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:44.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.117+0000 7fdaf5800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdaf0109770 0x7fdaf01a5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57218/0 (socket says 172.21.15.43:57218) 2024-03-26T08:31:44.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.117+0000 7fdaf5800640 1 -- 172.21.15.43:0/1801246779 learned_addr learned my addr 172.21.15.43:0/1801246779 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:44.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.117+0000 7fdaf4e00640 1 --2- 172.21.15.43:0/1801246779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaf0107bc0 0x7fdaf019ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:44.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.117+0000 7fdaefe00640 1 --2- 172.21.15.43:0/1801246779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdaf0108db0 0x7fdaf019e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:44.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.117+0000 7fdaf5800640 1 -- 172.21.15.43:0/1801246779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdaf0108db0 msgr2=0x7fdaf019e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:44.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.117+0000 7fdaf5800640 1 --2- 172.21.15.43:0/1801246779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdaf0108db0 0x7fdaf019e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:44.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.117+0000 7fdaf5800640 1 -- 172.21.15.43:0/1801246779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaf0107bc0 msgr2=0x7fdaf019ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:44.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.117+0000 7fdaf5800640 1 --2- 172.21.15.43:0/1801246779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaf0107bc0 0x7fdaf019ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:44.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.117+0000 7fdaf5800640 1 -- 172.21.15.43:0/1801246779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf01a59c0 con 0x7fdaf0109770 2024-03-26T08:31:44.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.117+0000 7fdaefe00640 1 --2- 172.21.15.43:0/1801246779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdaf0108db0 0x7fdaf019e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:31:44.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.117+0000 7fdaf5800640 1 --2- 172.21.15.43:0/1801246779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdaf0109770 0x7fdaf01a5320 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdae800cce0 tx=0x7fdae800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:44.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.118+0000 7fdaed600640 1 -- 172.21.15.43:0/1801246779 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae800f450 con 0x7fdaf0109770 2024-03-26T08:31:44.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.118+0000 7fdaed600640 1 -- 172.21.15.43:0/1801246779 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdae800faa0 con 0x7fdaf0109770 2024-03-26T08:31:44.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.118+0000 7fdaed600640 1 -- 172.21.15.43:0/1801246779 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae8018b70 con 0x7fdaf0109770 2024-03-26T08:31:44.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.118+0000 7fdaf6200640 1 -- 172.21.15.43:0/1801246779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf01a5cb0 con 0x7fdaf0109770 2024-03-26T08:31:44.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.118+0000 7fdaf6200640 1 -- 172.21.15.43:0/1801246779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf01a61b0 con 0x7fdaf0109770 2024-03-26T08:31:44.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.119+0000 7fdaf6200640 1 -- 172.21.15.43:0/1801246779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf010eee0 con 0x7fdaf0109770 2024-03-26T08:31:44.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.121+0000 7fdaed600640 1 -- 172.21.15.43:0/1801246779 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fdae8018d10 con 0x7fdaf0109770 2024-03-26T08:31:44.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.123+0000 7fdaed600640 1 --2- 172.21.15.43:0/1801246779 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fdad80a2880 0x7fdad80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:44.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.123+0000 7fdaed600640 1 -- 172.21.15.43:0/1801246779 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7fdae80cf2f0 con 0x7fdaf0109770 2024-03-26T08:31:44.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.124+0000 7fdaf4e00640 1 --2- 172.21.15.43:0/1801246779 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fdad80a2880 0x7fdad80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:44.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.124+0000 7fdaf4e00640 1 --2- 172.21.15.43:0/1801246779 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fdad80a2880 0x7fdad80a4d40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdae4005fd0 tx=0x7fdae4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:44.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.128+0000 7fdaed600640 1 -- 172.21.15.43:0/1801246779 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdae8014030 con 0x7fdaf0109770 2024-03-26T08:31:44.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.328+0000 7fdaf6200640 1 -- 172.21.15.43:0/1801246779 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdaf002d270 con 0x7fdaf0109770 2024-03-26T08:31:44.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.330+0000 7fdaed600640 1 -- 172.21.15.43:0/1801246779 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdae809bff0 con 0x7fdaf0109770 2024-03-26T08:31:44.322 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-26T08:31:44.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.332+0000 7fdaf6200640 1 -- 172.21.15.43:0/1801246779 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fdad80a2880 msgr2=0x7fdad80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:44.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.332+0000 7fdaf6200640 1 --2- 172.21.15.43:0/1801246779 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fdad80a2880 0x7fdad80a4d40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdae4005fd0 tx=0x7fdae4005850 comp rx=0 tx=0).stop 2024-03-26T08:31:44.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.332+0000 7fdaf6200640 1 -- 172.21.15.43:0/1801246779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdaf0109770 msgr2=0x7fdaf01a5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:44.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.332+0000 7fdaf6200640 1 --2- 172.21.15.43:0/1801246779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdaf0109770 0x7fdaf01a5320 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdae800cce0 tx=0x7fdae800a620 comp rx=0 tx=0).stop 2024-03-26T08:31:44.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.332+0000 7fdaf6200640 1 -- 172.21.15.43:0/1801246779 shutdown_connections 2024-03-26T08:31:44.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.332+0000 7fdaf6200640 1 --2- 172.21.15.43:0/1801246779 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fdad80a2880 0x7fdad80a4d40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:44.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.332+0000 7fdaf6200640 1 --2- 172.21.15.43:0/1801246779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdaf0109770 0x7fdaf01a5320 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:44.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.332+0000 7fdaf6200640 1 --2- 172.21.15.43:0/1801246779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdaf0108db0 0x7fdaf019e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:44.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.333+0000 7fdaf6200640 1 --2- 172.21.15.43:0/1801246779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaf0107bc0 0x7fdaf019ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:44.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.333+0000 7fdaf6200640 1 -- 172.21.15.43:0/1801246779 >> 172.21.15.43:0/1801246779 conn(0x7fdaf0076b70 msgr2=0x7fdaf010c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:44.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.333+0000 7fdaf6200640 1 -- 172.21.15.43:0/1801246779 shutdown_connections 2024-03-26T08:31:44.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.333+0000 7fdaf6200640 1 -- 172.21.15.43:0/1801246779 wait complete. 2024-03-26T08:31:44.345 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-26T08:31:44.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.580+0000 7f2008600640 1 -- 172.21.15.43:0/1080553709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2000104ad0 msgr2=0x7f2000104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:44.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.580+0000 7f2008600640 1 --2- 172.21.15.43:0/1080553709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2000104ad0 0x7f2000104f50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009900 tx=0x7f1fec02bc30 comp rx=0 tx=0).stop 2024-03-26T08:31:44.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.580+0000 7f2008600640 1 -- 172.21.15.43:0/1080553709 shutdown_connections 2024-03-26T08:31:44.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.580+0000 7f2008600640 1 --2- 172.21.15.43:0/1080553709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2000105490 0x7f200010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:44.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.580+0000 7f2008600640 1 --2- 172.21.15.43:0/1080553709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2000104ad0 0x7f2000104f50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:44.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.580+0000 7f2008600640 1 --2- 172.21.15.43:0/1080553709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20001038e0 0x7f2000103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:44.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.580+0000 7f2008600640 1 -- 172.21.15.43:0/1080553709 >> 172.21.15.43:0/1080553709 conn(0x7f20000fee50 msgr2=0x7f2000101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:44.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.580+0000 7f2008600640 1 -- 172.21.15.43:0/1080553709 shutdown_connections 2024-03-26T08:31:44.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.580+0000 7f2008600640 1 -- 172.21.15.43:0/1080553709 wait complete. 2024-03-26T08:31:44.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.581+0000 7f2008600640 1 Processor -- start 2024-03-26T08:31:44.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.581+0000 7f2008600640 1 -- start start 2024-03-26T08:31:44.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.581+0000 7f2008600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20001038e0 0x7f2000199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:44.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.581+0000 7f2008600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2000104ad0 0x7f200019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:44.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.582+0000 7f2008600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2000105490 0x7f20001a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:44.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.582+0000 7f2008600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200010f3f0 con 0x7f2000104ad0 2024-03-26T08:31:44.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.582+0000 7f2008600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200010f270 con 0x7f2000105490 2024-03-26T08:31:44.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.582+0000 7f2008600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200010f570 con 0x7f20001038e0 2024-03-26T08:31:44.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.582+0000 7f2005200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2000104ad0 0x7f200019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:44.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.582+0000 7f2005200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2000104ad0 0x7f200019a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57236/0 (socket says 172.21.15.43:57236) 2024-03-26T08:31:44.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.582+0000 7f2005200640 1 -- 172.21.15.43:0/1990228864 learned_addr learned my addr 172.21.15.43:0/1990228864 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:44.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.582+0000 7f2006600640 1 --2- 172.21.15.43:0/1990228864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2000105490 0x7f20001a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:44.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.582+0000 7f2005200640 1 -- 172.21.15.43:0/1990228864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20001038e0 msgr2=0x7f2000199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:44.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.582+0000 7f2005200640 1 --2- 172.21.15.43:0/1990228864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20001038e0 0x7f2000199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:44.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.582+0000 7f2005200640 1 -- 172.21.15.43:0/1990228864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2000105490 msgr2=0x7f20001a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:44.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.582+0000 7f2005200640 1 --2- 172.21.15.43:0/1990228864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2000105490 0x7f20001a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:44.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.582+0000 7f2005200640 1 -- 172.21.15.43:0/1990228864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20001a16f0 con 0x7f2000104ad0 2024-03-26T08:31:44.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.582+0000 7f2006600640 1 --2- 172.21.15.43:0/1990228864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2000105490 0x7f20001a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:31:44.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.583+0000 7f2005200640 1 --2- 172.21.15.43:0/1990228864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2000104ad0 0x7f200019a010 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1fec006010 tx=0x7f1fec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:44.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.583+0000 7f1ffe000640 1 -- 172.21.15.43:0/1990228864 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fec043020 con 0x7f2000104ad0 2024-03-26T08:31:44.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.583+0000 7f1ffe000640 1 -- 172.21.15.43:0/1990228864 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1fec004aa0 con 0x7f2000104ad0 2024-03-26T08:31:44.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.583+0000 7f2008600640 1 -- 172.21.15.43:0/1990228864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20001a1980 con 0x7f2000104ad0 2024-03-26T08:31:44.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.583+0000 7f1ffe000640 1 -- 172.21.15.43:0/1990228864 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fec003c60 con 0x7f2000104ad0 2024-03-26T08:31:44.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.583+0000 7f2008600640 1 -- 172.21.15.43:0/1990228864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20001a1ed0 con 0x7f2000104ad0 2024-03-26T08:31:44.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.584+0000 7f2008600640 1 -- 172.21.15.43:0/1990228864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f200010ad20 con 0x7f2000104ad0 2024-03-26T08:31:44.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.586+0000 7f1ffe000640 1 -- 172.21.15.43:0/1990228864 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1fec046020 con 0x7f2000104ad0 2024-03-26T08:31:44.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.588+0000 7f1ffe000640 1 --2- 172.21.15.43:0/1990228864 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1fe80a2800 0x7f1fe80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:44.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.588+0000 7f1ffe000640 1 -- 172.21.15.43:0/1990228864 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f1fec0f1870 con 0x7f2000104ad0 2024-03-26T08:31:44.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.588+0000 7f2005c00640 1 --2- 172.21.15.43:0/1990228864 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1fe80a2800 0x7f1fe80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:44.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.589+0000 7f2005c00640 1 --2- 172.21.15.43:0/1990228864 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1fe80a2800 0x7f1fe80a4cc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2000104930 tx=0x7f1ff4005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:44.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.591+0000 7f1ffe000640 1 -- 172.21.15.43:0/1990228864 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1fec04d020 con 0x7f2000104ad0 2024-03-26T08:31:44.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:44 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2903050058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:31:44.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:44 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1801246779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T08:31:44.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:44 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2903050058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:31:44.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:44 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1801246779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T08:31:44.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.811+0000 7f2008600640 1 -- 172.21.15.43:0/1990228864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f200002d230 con 0x7f2000104ad0 2024-03-26T08:31:44.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.812+0000 7f1ffe000640 1 -- 172.21.15.43:0/1990228864 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v92) v1 ==== 81+0+590 (secure 0 0 0) 0x7f1fec0be690 con 0x7f2000104ad0 2024-03-26T08:31:44.805 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6801/2446912488 2024-03-27T08:21:36.149092+0000 2024-03-26T08:31:44.805 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6800/2446912488 2024-03-27T08:21:36.149092+0000 2024-03-26T08:31:44.805 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/4240775295 2024-03-27T08:21:36.149092+0000 2024-03-26T08:31:44.805 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/510203140 2024-03-27T08:21:01.256355+0000 2024-03-26T08:31:44.805 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/4251957687 2024-03-27T08:21:36.149092+0000 2024-03-26T08:31:44.805 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6800/4108777098 2024-03-27T08:21:01.256355+0000 2024-03-26T08:31:44.806 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6801/4108777098 2024-03-27T08:21:01.256355+0000 2024-03-26T08:31:44.806 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/1885244665 2024-03-27T08:21:01.256355+0000 2024-03-26T08:31:44.806 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/298320867 2024-03-27T08:21:01.256355+0000 2024-03-26T08:31:44.806 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/1301092625 2024-03-27T08:21:36.149092+0000 2024-03-26T08:31:44.806 INFO:teuthology.orchestra.run.smithi043.stderr:listed 10 entries 2024-03-26T08:31:44.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.815+0000 7f2008600640 1 -- 172.21.15.43:0/1990228864 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1fe80a2800 msgr2=0x7f1fe80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:44.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.815+0000 7f2008600640 1 --2- 172.21.15.43:0/1990228864 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1fe80a2800 0x7f1fe80a4cc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2000104930 tx=0x7f1ff4005c90 comp rx=0 tx=0).stop 2024-03-26T08:31:44.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.815+0000 7f2008600640 1 -- 172.21.15.43:0/1990228864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2000104ad0 msgr2=0x7f200019a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:44.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.815+0000 7f2008600640 1 --2- 172.21.15.43:0/1990228864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2000104ad0 0x7f200019a010 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1fec006010 tx=0x7f1fec0042f0 comp rx=0 tx=0).stop 2024-03-26T08:31:44.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.815+0000 7f2008600640 1 -- 172.21.15.43:0/1990228864 shutdown_connections 2024-03-26T08:31:44.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.815+0000 7f2008600640 1 --2- 172.21.15.43:0/1990228864 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1fe80a2800 0x7f1fe80a4cc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:44.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.815+0000 7f2008600640 1 --2- 172.21.15.43:0/1990228864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2000105490 0x7f20001a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:44.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.815+0000 7f2008600640 1 --2- 172.21.15.43:0/1990228864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2000104ad0 0x7f200019a010 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:44.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.815+0000 7f2008600640 1 --2- 172.21.15.43:0/1990228864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20001038e0 0x7f2000199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:44.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.815+0000 7f2008600640 1 -- 172.21.15.43:0/1990228864 >> 172.21.15.43:0/1990228864 conn(0x7f20000fee50 msgr2=0x7f2000107ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:44.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.815+0000 7f2008600640 1 -- 172.21.15.43:0/1990228864 shutdown_connections 2024-03-26T08:31:44.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:44.816+0000 7f2008600640 1 -- 172.21.15.43:0/1990228864 wait complete. 2024-03-26T08:31:44.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:44 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2903050058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:31:44.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:44 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1801246779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T08:31:44.827 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2024-03-26T08:31:44.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:31:44.827 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-26T08:31:44.919 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi043.stderr:ceph-fuse[73320]: fuse finished with error 0 and tester_r 0 2024-03-26T08:31:44.924 INFO:teuthology.orchestra.run:waiting for 300 2024-03-26T08:31:45.000 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-03-26T08:31:45.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:31:45.000 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-26T08:31:45.065 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:31:45.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.297+0000 7f2c6d600640 1 -- 172.21.15.43:0/3410078752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c681048e0 msgr2=0x7f2c68104d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:45.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.298+0000 7f2c6d600640 1 --2- 172.21.15.43:0/3410078752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c681048e0 0x7f2c68104d60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2c50009900 tx=0x7f2c5002bc30 comp rx=0 tx=0).stop 2024-03-26T08:31:45.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.298+0000 7f2c6d600640 1 -- 172.21.15.43:0/3410078752 shutdown_connections 2024-03-26T08:31:45.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.298+0000 7f2c6d600640 1 --2- 172.21.15.43:0/3410078752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c681052a0 0x7f2c6810bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:45.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.298+0000 7f2c6d600640 1 --2- 172.21.15.43:0/3410078752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c681048e0 0x7f2c68104d60 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:45.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.298+0000 7f2c6d600640 1 --2- 172.21.15.43:0/3410078752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c681036f0 0x7f2c68103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:45.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.298+0000 7f2c6d600640 1 -- 172.21.15.43:0/3410078752 >> 172.21.15.43:0/3410078752 conn(0x7f2c680fec80 msgr2=0x7f2c681010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:45.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.298+0000 7f2c6d600640 1 -- 172.21.15.43:0/3410078752 shutdown_connections 2024-03-26T08:31:45.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.298+0000 7f2c6d600640 1 -- 172.21.15.43:0/3410078752 wait complete. 2024-03-26T08:31:45.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.299+0000 7f2c6d600640 1 Processor -- start 2024-03-26T08:31:45.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.299+0000 7f2c6d600640 1 -- start start 2024-03-26T08:31:45.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.299+0000 7f2c6d600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c681036f0 0x7f2c68199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:45.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.299+0000 7f2c6d600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c681048e0 0x7f2c68199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:45.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.299+0000 7f2c6d600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c681052a0 0x7f2c681a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:45.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.299+0000 7f2c6d600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6810f290 con 0x7f2c681036f0 2024-03-26T08:31:45.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.299+0000 7f2c6d600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6810f110 con 0x7f2c681052a0 2024-03-26T08:31:45.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.299+0000 7f2c6d600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6810f410 con 0x7f2c681048e0 2024-03-26T08:31:45.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.299+0000 7f2c67e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c681036f0 0x7f2c68199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:45.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.300+0000 7f2c67e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c681036f0 0x7f2c68199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57248/0 (socket says 172.21.15.43:57248) 2024-03-26T08:31:45.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.300+0000 7f2c67e00640 1 -- 172.21.15.43:0/993589181 learned_addr learned my addr 172.21.15.43:0/993589181 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:45.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.300+0000 7f2c6cc00640 1 --2- 172.21.15.43:0/993589181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c681052a0 0x7f2c681a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:45.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.300+0000 7f2c5f400640 1 --2- 172.21.15.43:0/993589181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c681048e0 0x7f2c68199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:45.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.300+0000 7f2c67e00640 1 -- 172.21.15.43:0/993589181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c681048e0 msgr2=0x7f2c68199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:45.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.300+0000 7f2c67e00640 1 --2- 172.21.15.43:0/993589181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c681048e0 0x7f2c68199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:45.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.300+0000 7f2c67e00640 1 -- 172.21.15.43:0/993589181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c681052a0 msgr2=0x7f2c681a0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:45.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.300+0000 7f2c67e00640 1 --2- 172.21.15.43:0/993589181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c681052a0 0x7f2c681a0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:45.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.300+0000 7f2c67e00640 1 -- 172.21.15.43:0/993589181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c681a1530 con 0x7f2c681036f0 2024-03-26T08:31:45.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.300+0000 7f2c67e00640 1 --2- 172.21.15.43:0/993589181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c681036f0 0x7f2c68199960 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800b9c0 tx=0x7f2c5800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:45.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.301+0000 7f2c65600640 1 -- 172.21.15.43:0/993589181 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c58019070 con 0x7f2c681036f0 2024-03-26T08:31:45.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.301+0000 7f2c65600640 1 -- 172.21.15.43:0/993589181 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2c5800cba0 con 0x7f2c681036f0 2024-03-26T08:31:45.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.301+0000 7f2c6d600640 1 -- 172.21.15.43:0/993589181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c681a1820 con 0x7f2c681036f0 2024-03-26T08:31:45.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.301+0000 7f2c65600640 1 -- 172.21.15.43:0/993589181 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c580125a0 con 0x7f2c681036f0 2024-03-26T08:31:45.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.301+0000 7f2c6d600640 1 -- 172.21.15.43:0/993589181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c681a1df0 con 0x7f2c681036f0 2024-03-26T08:31:45.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.302+0000 7f2c6d600640 1 -- 172.21.15.43:0/993589181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c6810aa10 con 0x7f2c681036f0 2024-03-26T08:31:45.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.304+0000 7f2c65600640 1 -- 172.21.15.43:0/993589181 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f2c5801c070 con 0x7f2c681036f0 2024-03-26T08:31:45.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.305+0000 7f2c65600640 1 --2- 172.21.15.43:0/993589181 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2c4c0a2800 0x7f2c4c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:45.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.305+0000 7f2c65600640 1 -- 172.21.15.43:0/993589181 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f2c580ce420 con 0x7f2c681036f0 2024-03-26T08:31:45.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.306+0000 7f2c5f400640 1 --2- 172.21.15.43:0/993589181 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2c4c0a2800 0x7f2c4c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:45.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.306+0000 7f2c5f400640 1 --2- 172.21.15.43:0/993589181 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2c4c0a2800 0x7f2c4c0a4cc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2c50036040 tx=0x7f2c50000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:45.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.310+0000 7f2c65600640 1 -- 172.21.15.43:0/993589181 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2c5809b120 con 0x7f2c681036f0 2024-03-26T08:31:45.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.522+0000 7f2c6d600640 1 -- 172.21.15.43:0/993589181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c680622a0 con 0x7f2c681036f0 2024-03-26T08:31:45.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.527+0000 7f2c65600640 1 -- 172.21.15.43:0/993589181 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24576 (secure 0 0 0) 0x7f2c5809aa20 con 0x7f2c681036f0 2024-03-26T08:31:45.520 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:31:45.521 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":92,"fsid":"88759420-eb49-11ee-95d0-87774f69a715","created":"2024-03-26T08:20:27.817107+0000","modified":"2024-03-26T08:31:37.455519+0000","last_up_change":"2024-03-26T08:30:10.520976+0000","last_in_change":"2024-03-26T08:29:46.394360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:24:54.852427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:31:19.924902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:31:21.323881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T08:31:24.951015+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"28670673-9193-4fa3-9cc0-c1a5a6b27284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6803","nonce":936601146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6805","nonce":936601146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6809","nonce":936601146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6807","nonce":936601146}]},"public_addr":"172.21.15.43:6803/936601146","cluster_addr":"172.21.15.43:6805/936601146","heartbeat_back_addr":"172.21.15.43:6809/936601146","heartbeat_front_addr":"172.21.15.43:6807/936601146","state":["exists","up"]},{"osd":1,"uuid":"fa8ad6b6-277a-42b8-b0f8-b36fc219de2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6801","nonce":3268412328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6803","nonce":3268412328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6807","nonce":3268412328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6805","nonce":3268412328}]},"public_addr":"172.21.15.52:6801/3268412328","cluster_addr":"172.21.15.52:6803/3268412328","heartbeat_back_addr":"172.21.15.52:6807/3268412328","heartbeat_front_addr":"172.21.15.52:6805/3268412328","state":["exists","up"]},{"osd":2,"uuid":"ec0039c3-f9bd-419a-81de-fa4418359808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6801","nonce":2103180908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6803","nonce":2103180908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6807","nonce":2103180908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6805","nonce":2103180908}]},"public_addr":"172.21.15.183:6801/2103180908","cluster_addr":"172.21.15.183:6803/2103180908","heartbeat_back_addr":"172.21.15.183:6807/2103180908","heartbeat_front_addr":"172.21.15.183:6805/2103180908","state":["exists","up"]},{"osd":3,"uuid":"e7dcbf00-6586-4664-a69e-3cb0df8dfd30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6811","nonce":3722846033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6813","nonce":3722846033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6817","nonce":3722846033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6815","nonce":3722846033}]},"public_addr":"172.21.15.43:6811/3722846033","cluster_addr":"172.21.15.43:6813/3722846033","heartbeat_back_addr":"172.21.15.43:6817/3722846033","heartbeat_front_addr":"172.21.15.43:6815/3722846033","state":["exists","up"]},{"osd":4,"uuid":"db089a49-5e98-432d-9687-cdc044114c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6809","nonce":4028436523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6811","nonce":4028436523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6815","nonce":4028436523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6813","nonce":4028436523}]},"public_addr":"172.21.15.52:6809/4028436523","cluster_addr":"172.21.15.52:6811/4028436523","heartbeat_back_addr":"172.21.15.52:6815/4028436523","heartbeat_front_addr":"172.21.15.52:6813/4028436523","state":["exists","up"]},{"osd":5,"uuid":"556ac804-02ea-423b-a599-55baf5be4ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6809","nonce":787132252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6811","nonce":787132252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6815","nonce":787132252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6813","nonce":787132252}]},"public_addr":"172.21.15.183:6809/787132252","cluster_addr":"172.21.15.183:6811/787132252","heartbeat_back_addr":"172.21.15.183:6815/787132252","heartbeat_front_addr":"172.21.15.183:6813/787132252","state":["exists","up"]},{"osd":6,"uuid":"3de2a49d-6cef-4580-9d8a-f90cdfcecd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6819","nonce":4206861158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6821","nonce":4206861158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6825","nonce":4206861158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6823","nonce":4206861158}]},"public_addr":"172.21.15.43:6819/4206861158","cluster_addr":"172.21.15.43:6821/4206861158","heartbeat_back_addr":"172.21.15.43:6825/4206861158","heartbeat_front_addr":"172.21.15.43:6823/4206861158","state":["exists","up"]},{"osd":7,"uuid":"a3b832f7-7d7d-4a62-b679-db6da98275d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6817","nonce":3452064146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6819","nonce":3452064146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6823","nonce":3452064146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6821","nonce":3452064146}]},"public_addr":"172.21.15.52:6817/3452064146","cluster_addr":"172.21.15.52:6819/3452064146","heartbeat_back_addr":"172.21.15.52:6823/3452064146","heartbeat_front_addr":"172.21.15.52:6821/3452064146","state":["exists","up"]},{"osd":8,"uuid":"ddc6b07f-d703-414d-a59a-02316376b13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6817","nonce":1023131725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6819","nonce":1023131725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6823","nonce":1023131725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6821","nonce":1023131725}]},"public_addr":"172.21.15.183:6817/1023131725","cluster_addr":"172.21.15.183:6819/1023131725","heartbeat_back_addr":"172.21.15.183:6823/1023131725","heartbeat_front_addr":"172.21.15.183:6821/1023131725","state":["exists","up"]},{"osd":9,"uuid":"ac16bb0e-6232-45c8-b16a-4d878874c588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6827","nonce":3935195657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6829","nonce":3935195657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6833","nonce":3935195657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6831","nonce":3935195657}]},"public_addr":"172.21.15.43:6827/3935195657","cluster_addr":"172.21.15.43:6829/3935195657","heartbeat_back_addr":"172.21.15.43:6833/3935195657","heartbeat_front_addr":"172.21.15.43:6831/3935195657","state":["exists","up"]},{"osd":10,"uuid":"fd839188-f398-4cff-aa7b-36522ff1a8a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6825","nonce":377628320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6827","nonce":377628320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6831","nonce":377628320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6829","nonce":377628320}]},"public_addr":"172.21.15.52:6825/377628320","cluster_addr":"172.21.15.52:6827/377628320","heartbeat_back_addr":"172.21.15.52:6831/377628320","heartbeat_front_addr":"172.21.15.52:6829/377628320","state":["exists","up"]},{"osd":11,"uuid":"a40282a0-0c64-49ee-b64e-6f3f8119e827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6825","nonce":1373086433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6827","nonce":1373086433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6831","nonce":1373086433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6829","nonce":1373086433}]},"public_addr":"172.21.15.183:6825/1373086433","cluster_addr":"172.21.15.183:6827/1373086433","heartbeat_back_addr":"172.21.15.183:6831/1373086433","heartbeat_front_addr":"172.21.15.183:6829/1373086433","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:23:43.478905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:17.573532+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:50.286610+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:25:27.407663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:03.081294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:36.057416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:12.062013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:47.032271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:21.023201+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:57.364739+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:29:34.039995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:30:08.827577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6801/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/4240775295":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/510203140":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/4251957687":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:6801/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1885244665":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/298320867":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1301092625":"2024-03-27T08:21:36.149092+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:31:45.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.530+0000 7f2c6d600640 1 -- 172.21.15.43:0/993589181 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2c4c0a2800 msgr2=0x7f2c4c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:45.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.530+0000 7f2c6d600640 1 --2- 172.21.15.43:0/993589181 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2c4c0a2800 0x7f2c4c0a4cc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2c50036040 tx=0x7f2c50000bc0 comp rx=0 tx=0).stop 2024-03-26T08:31:45.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.530+0000 7f2c6d600640 1 -- 172.21.15.43:0/993589181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c681036f0 msgr2=0x7f2c68199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:45.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.530+0000 7f2c6d600640 1 --2- 172.21.15.43:0/993589181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c681036f0 0x7f2c68199960 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800b9c0 tx=0x7f2c5800be80 comp rx=0 tx=0).stop 2024-03-26T08:31:45.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.530+0000 7f2c6d600640 1 -- 172.21.15.43:0/993589181 shutdown_connections 2024-03-26T08:31:45.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.530+0000 7f2c6d600640 1 --2- 172.21.15.43:0/993589181 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2c4c0a2800 0x7f2c4c0a4cc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:45.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.530+0000 7f2c6d600640 1 --2- 172.21.15.43:0/993589181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c681052a0 0x7f2c681a0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:45.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.530+0000 7f2c6d600640 1 --2- 172.21.15.43:0/993589181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c681048e0 0x7f2c68199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:45.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.530+0000 7f2c6d600640 1 --2- 172.21.15.43:0/993589181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c681036f0 0x7f2c68199960 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:45.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.530+0000 7f2c6d600640 1 -- 172.21.15.43:0/993589181 >> 172.21.15.43:0/993589181 conn(0x7f2c680fec80 msgr2=0x7f2c68107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:45.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.530+0000 7f2c6d600640 1 -- 172.21.15.43:0/993589181 shutdown_connections 2024-03-26T08:31:45.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.530+0000 7f2c6d600640 1 -- 172.21.15.43:0/993589181 wait complete. 2024-03-26T08:31:45.545 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:31:45.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.770+0000 7f7034000640 1 -- 172.21.15.43:0/2966245547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f702c1038e0 msgr2=0x7f702c103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:45.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.770+0000 7f7034000640 1 --2- 172.21.15.43:0/2966245547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f702c1038e0 0x7f702c103d00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7020009ca0 tx=0x7f702002c100 comp rx=0 tx=0).stop 2024-03-26T08:31:45.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.770+0000 7f7034000640 1 -- 172.21.15.43:0/2966245547 shutdown_connections 2024-03-26T08:31:45.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.770+0000 7f7034000640 1 --2- 172.21.15.43:0/2966245547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f702c105490 0x7f702c10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:45.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.770+0000 7f7034000640 1 --2- 172.21.15.43:0/2966245547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f702c104ad0 0x7f702c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:45.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.770+0000 7f7034000640 1 --2- 172.21.15.43:0/2966245547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f702c1038e0 0x7f702c103d00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:45.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.770+0000 7f7034000640 1 -- 172.21.15.43:0/2966245547 >> 172.21.15.43:0/2966245547 conn(0x7f702c0fee50 msgr2=0x7f702c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:45.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.771+0000 7f7034000640 1 -- 172.21.15.43:0/2966245547 shutdown_connections 2024-03-26T08:31:45.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.771+0000 7f7034000640 1 -- 172.21.15.43:0/2966245547 wait complete. 2024-03-26T08:31:45.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.772+0000 7f7034000640 1 Processor -- start 2024-03-26T08:31:45.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.772+0000 7f7034000640 1 -- start start 2024-03-26T08:31:45.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.772+0000 7f7034000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f702c1038e0 0x7f702c199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:45.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.772+0000 7f7034000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f702c104ad0 0x7f702c199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:45.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.772+0000 7f7034000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f702c105490 0x7f702c1a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:45.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.772+0000 7f7034000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702c10f3e0 con 0x7f702c105490 2024-03-26T08:31:45.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.772+0000 7f7034000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702c10f260 con 0x7f702c104ad0 2024-03-26T08:31:45.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.772+0000 7f7034000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702c10f560 con 0x7f702c1038e0 2024-03-26T08:31:45.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.773+0000 7f7032000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f702c105490 0x7f702c1a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:45.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.773+0000 7f7032000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f702c105490 0x7f702c1a1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57270/0 (socket says 172.21.15.43:57270) 2024-03-26T08:31:45.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.773+0000 7f7032000640 1 -- 172.21.15.43:0/1786637888 learned_addr learned my addr 172.21.15.43:0/1786637888 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:45.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.773+0000 7f7030c00640 1 --2- 172.21.15.43:0/1786637888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f702c104ad0 0x7f702c199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:45.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.773+0000 7f7031600640 1 --2- 172.21.15.43:0/1786637888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f702c1038e0 0x7f702c199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:45.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.773+0000 7f7032000640 1 -- 172.21.15.43:0/1786637888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f702c1038e0 msgr2=0x7f702c199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:45.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.773+0000 7f7032000640 1 --2- 172.21.15.43:0/1786637888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f702c1038e0 0x7f702c199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:45.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.773+0000 7f7032000640 1 -- 172.21.15.43:0/1786637888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f702c104ad0 msgr2=0x7f702c199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:45.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.773+0000 7f7032000640 1 --2- 172.21.15.43:0/1786637888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f702c104ad0 0x7f702c199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:45.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.773+0000 7f7032000640 1 -- 172.21.15.43:0/1786637888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f702c1a16a0 con 0x7f702c105490 2024-03-26T08:31:45.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.773+0000 7f7030c00640 1 --2- 172.21.15.43:0/1786637888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f702c104ad0 0x7f702c199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:31:45.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.774+0000 7f7032000640 1 --2- 172.21.15.43:0/1786637888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f702c105490 0x7f702c1a1030 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f702400d750 tx=0x7f702400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:45.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.774+0000 7f702a000640 1 -- 172.21.15.43:0/1786637888 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7024017070 con 0x7f702c105490 2024-03-26T08:31:45.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.774+0000 7f702a000640 1 -- 172.21.15.43:0/1786637888 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f70240054d0 con 0x7f702c105490 2024-03-26T08:31:45.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.774+0000 7f7034000640 1 -- 172.21.15.43:0/1786637888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f702c1a1990 con 0x7f702c105490 2024-03-26T08:31:45.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.774+0000 7f702a000640 1 -- 172.21.15.43:0/1786637888 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7024004c60 con 0x7f702c105490 2024-03-26T08:31:45.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.774+0000 7f7034000640 1 -- 172.21.15.43:0/1786637888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f702c1a1e90 con 0x7f702c105490 2024-03-26T08:31:45.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.775+0000 7f7034000640 1 -- 172.21.15.43:0/1786637888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f702c10ac80 con 0x7f702c105490 2024-03-26T08:31:45.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.777+0000 7f702a000640 1 -- 172.21.15.43:0/1786637888 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f7024004e00 con 0x7f702c105490 2024-03-26T08:31:45.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.779+0000 7f702a000640 1 --2- 172.21.15.43:0/1786637888 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f70140a2800 0x7f70140a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:45.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.780+0000 7f702a000640 1 -- 172.21.15.43:0/1786637888 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f70240ce880 con 0x7f702c105490 2024-03-26T08:31:45.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.780+0000 7f7031600640 1 --2- 172.21.15.43:0/1786637888 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f70140a2800 0x7f70140a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:45.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.781+0000 7f7031600640 1 --2- 172.21.15.43:0/1786637888 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f70140a2800 0x7f70140a4cc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f702002c710 tx=0x7f7020000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:45.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.783+0000 7f702a000640 1 -- 172.21.15.43:0/1786637888 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f702409b580 con 0x7f702c105490 2024-03-26T08:31:45.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:45 smithi183 ceph-mon[35818]: pgmap v386: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-03-26T08:31:45.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:45 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1990228864' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T08:31:45.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:45 smithi052 ceph-mon[35071]: pgmap v386: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-03-26T08:31:45.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:45 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1990228864' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T08:31:45.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:45 smithi043 ceph-mon[31929]: pgmap v386: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-03-26T08:31:45.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:45 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1990228864' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T08:31:45.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.986+0000 7f7034000640 1 -- 172.21.15.43:0/1786637888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f702c0622a0 con 0x7f702c105490 2024-03-26T08:31:45.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.991+0000 7f702a000640 1 -- 172.21.15.43:0/1786637888 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24576 (secure 0 0 0) 0x7f702409ae80 con 0x7f702c105490 2024-03-26T08:31:45.984 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:31:45.985 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":92,"fsid":"88759420-eb49-11ee-95d0-87774f69a715","created":"2024-03-26T08:20:27.817107+0000","modified":"2024-03-26T08:31:37.455519+0000","last_up_change":"2024-03-26T08:30:10.520976+0000","last_in_change":"2024-03-26T08:29:46.394360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:24:54.852427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:31:19.924902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:31:21.323881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T08:31:24.951015+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"28670673-9193-4fa3-9cc0-c1a5a6b27284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6803","nonce":936601146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6805","nonce":936601146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6809","nonce":936601146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6807","nonce":936601146}]},"public_addr":"172.21.15.43:6803/936601146","cluster_addr":"172.21.15.43:6805/936601146","heartbeat_back_addr":"172.21.15.43:6809/936601146","heartbeat_front_addr":"172.21.15.43:6807/936601146","state":["exists","up"]},{"osd":1,"uuid":"fa8ad6b6-277a-42b8-b0f8-b36fc219de2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6801","nonce":3268412328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6803","nonce":3268412328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6807","nonce":3268412328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6805","nonce":3268412328}]},"public_addr":"172.21.15.52:6801/3268412328","cluster_addr":"172.21.15.52:6803/3268412328","heartbeat_back_addr":"172.21.15.52:6807/3268412328","heartbeat_front_addr":"172.21.15.52:6805/3268412328","state":["exists","up"]},{"osd":2,"uuid":"ec0039c3-f9bd-419a-81de-fa4418359808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6801","nonce":2103180908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6803","nonce":2103180908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6807","nonce":2103180908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6805","nonce":2103180908}]},"public_addr":"172.21.15.183:6801/2103180908","cluster_addr":"172.21.15.183:6803/2103180908","heartbeat_back_addr":"172.21.15.183:6807/2103180908","heartbeat_front_addr":"172.21.15.183:6805/2103180908","state":["exists","up"]},{"osd":3,"uuid":"e7dcbf00-6586-4664-a69e-3cb0df8dfd30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6811","nonce":3722846033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6813","nonce":3722846033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6817","nonce":3722846033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6815","nonce":3722846033}]},"public_addr":"172.21.15.43:6811/3722846033","cluster_addr":"172.21.15.43:6813/3722846033","heartbeat_back_addr":"172.21.15.43:6817/3722846033","heartbeat_front_addr":"172.21.15.43:6815/3722846033","state":["exists","up"]},{"osd":4,"uuid":"db089a49-5e98-432d-9687-cdc044114c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6809","nonce":4028436523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6811","nonce":4028436523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6815","nonce":4028436523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6813","nonce":4028436523}]},"public_addr":"172.21.15.52:6809/4028436523","cluster_addr":"172.21.15.52:6811/4028436523","heartbeat_back_addr":"172.21.15.52:6815/4028436523","heartbeat_front_addr":"172.21.15.52:6813/4028436523","state":["exists","up"]},{"osd":5,"uuid":"556ac804-02ea-423b-a599-55baf5be4ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6809","nonce":787132252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6811","nonce":787132252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6815","nonce":787132252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6813","nonce":787132252}]},"public_addr":"172.21.15.183:6809/787132252","cluster_addr":"172.21.15.183:6811/787132252","heartbeat_back_addr":"172.21.15.183:6815/787132252","heartbeat_front_addr":"172.21.15.183:6813/787132252","state":["exists","up"]},{"osd":6,"uuid":"3de2a49d-6cef-4580-9d8a-f90cdfcecd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6819","nonce":4206861158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6821","nonce":4206861158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6825","nonce":4206861158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6823","nonce":4206861158}]},"public_addr":"172.21.15.43:6819/4206861158","cluster_addr":"172.21.15.43:6821/4206861158","heartbeat_back_addr":"172.21.15.43:6825/4206861158","heartbeat_front_addr":"172.21.15.43:6823/4206861158","state":["exists","up"]},{"osd":7,"uuid":"a3b832f7-7d7d-4a62-b679-db6da98275d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6817","nonce":3452064146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6819","nonce":3452064146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6823","nonce":3452064146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6821","nonce":3452064146}]},"public_addr":"172.21.15.52:6817/3452064146","cluster_addr":"172.21.15.52:6819/3452064146","heartbeat_back_addr":"172.21.15.52:6823/3452064146","heartbeat_front_addr":"172.21.15.52:6821/3452064146","state":["exists","up"]},{"osd":8,"uuid":"ddc6b07f-d703-414d-a59a-02316376b13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6817","nonce":1023131725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6819","nonce":1023131725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6823","nonce":1023131725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6821","nonce":1023131725}]},"public_addr":"172.21.15.183:6817/1023131725","cluster_addr":"172.21.15.183:6819/1023131725","heartbeat_back_addr":"172.21.15.183:6823/1023131725","heartbeat_front_addr":"172.21.15.183:6821/1023131725","state":["exists","up"]},{"osd":9,"uuid":"ac16bb0e-6232-45c8-b16a-4d878874c588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6827","nonce":3935195657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6829","nonce":3935195657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6833","nonce":3935195657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6831","nonce":3935195657}]},"public_addr":"172.21.15.43:6827/3935195657","cluster_addr":"172.21.15.43:6829/3935195657","heartbeat_back_addr":"172.21.15.43:6833/3935195657","heartbeat_front_addr":"172.21.15.43:6831/3935195657","state":["exists","up"]},{"osd":10,"uuid":"fd839188-f398-4cff-aa7b-36522ff1a8a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6825","nonce":377628320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6827","nonce":377628320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6831","nonce":377628320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6829","nonce":377628320}]},"public_addr":"172.21.15.52:6825/377628320","cluster_addr":"172.21.15.52:6827/377628320","heartbeat_back_addr":"172.21.15.52:6831/377628320","heartbeat_front_addr":"172.21.15.52:6829/377628320","state":["exists","up"]},{"osd":11,"uuid":"a40282a0-0c64-49ee-b64e-6f3f8119e827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6825","nonce":1373086433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6827","nonce":1373086433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6831","nonce":1373086433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6829","nonce":1373086433}]},"public_addr":"172.21.15.183:6825/1373086433","cluster_addr":"172.21.15.183:6827/1373086433","heartbeat_back_addr":"172.21.15.183:6831/1373086433","heartbeat_front_addr":"172.21.15.183:6829/1373086433","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:23:43.478905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:17.573532+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:50.286610+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:25:27.407663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:03.081294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:36.057416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:12.062013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:47.032271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:21.023201+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:57.364739+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:29:34.039995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:30:08.827577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6801/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/4240775295":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/510203140":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/4251957687":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:6801/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1885244665":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/298320867":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1301092625":"2024-03-27T08:21:36.149092+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:31:45.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.994+0000 7f7034000640 1 -- 172.21.15.43:0/1786637888 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f70140a2800 msgr2=0x7f70140a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:45.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.994+0000 7f7034000640 1 --2- 172.21.15.43:0/1786637888 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f70140a2800 0x7f70140a4cc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f702002c710 tx=0x7f7020000f40 comp rx=0 tx=0).stop 2024-03-26T08:31:45.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.994+0000 7f7034000640 1 -- 172.21.15.43:0/1786637888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f702c105490 msgr2=0x7f702c1a1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:45.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.994+0000 7f7034000640 1 --2- 172.21.15.43:0/1786637888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f702c105490 0x7f702c1a1030 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f702400d750 tx=0x7f702400dc10 comp rx=0 tx=0).stop 2024-03-26T08:31:45.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.994+0000 7f7034000640 1 -- 172.21.15.43:0/1786637888 shutdown_connections 2024-03-26T08:31:45.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.994+0000 7f7034000640 1 --2- 172.21.15.43:0/1786637888 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f70140a2800 0x7f70140a4cc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:45.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.994+0000 7f7034000640 1 --2- 172.21.15.43:0/1786637888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f702c105490 0x7f702c1a1030 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:45.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.994+0000 7f7034000640 1 --2- 172.21.15.43:0/1786637888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f702c104ad0 0x7f702c199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:45.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.994+0000 7f7034000640 1 --2- 172.21.15.43:0/1786637888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f702c1038e0 0x7f702c199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:45.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.994+0000 7f7034000640 1 -- 172.21.15.43:0/1786637888 >> 172.21.15.43:0/1786637888 conn(0x7f702c0fee50 msgr2=0x7f702c107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:45.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.995+0000 7f7034000640 1 -- 172.21.15.43:0/1786637888 shutdown_connections 2024-03-26T08:31:45.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:45.995+0000 7f7034000640 1 -- 172.21.15.43:0/1786637888 wait complete. 2024-03-26T08:31:46.009 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-26T08:31:46.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.243+0000 7ff8ebe00640 1 -- 172.21.15.43:0/2841912734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8ec109450 msgr2=0x7ff8ec10b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:46.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.243+0000 7ff8ebe00640 1 --2- 172.21.15.43:0/2841912734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8ec109450 0x7ff8ec10b8e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009900 tx=0x7ff8e402bcd0 comp rx=0 tx=0).stop 2024-03-26T08:31:46.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.243+0000 7ff8ebe00640 1 -- 172.21.15.43:0/2841912734 shutdown_connections 2024-03-26T08:31:46.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.243+0000 7ff8ebe00640 1 --2- 172.21.15.43:0/2841912734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8ec109450 0x7ff8ec10b8e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:46.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.243+0000 7ff8ebe00640 1 --2- 172.21.15.43:0/2841912734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8ec074080 0x7ff8ec0744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:46.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.243+0000 7ff8ebe00640 1 --2- 172.21.15.43:0/2841912734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8ec075660 0x7ff8ec073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:46.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.243+0000 7ff8ebe00640 1 -- 172.21.15.43:0/2841912734 >> 172.21.15.43:0/2841912734 conn(0x7ff8ec0fcb10 msgr2=0x7ff8ec0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:46.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.243+0000 7ff8ebe00640 1 -- 172.21.15.43:0/2841912734 shutdown_connections 2024-03-26T08:31:46.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.243+0000 7ff8ebe00640 1 -- 172.21.15.43:0/2841912734 wait complete. 2024-03-26T08:31:46.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.244+0000 7ff8ebe00640 1 Processor -- start 2024-03-26T08:31:46.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.244+0000 7ff8ebe00640 1 -- start start 2024-03-26T08:31:46.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.244+0000 7ff8ebe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8ec074080 0x7ff8ec199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:46.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.244+0000 7ff8ebe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8ec075660 0x7ff8ec199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:46.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.244+0000 7ff8ebe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8ec109450 0x7ff8ec1a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:46.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.244+0000 7ff8ebe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec10f330 con 0x7ff8ec074080 2024-03-26T08:31:46.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.244+0000 7ff8ebe00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec10f1b0 con 0x7ff8ec109450 2024-03-26T08:31:46.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.244+0000 7ff8ebe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec10f4b0 con 0x7ff8ec075660 2024-03-26T08:31:46.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.245+0000 7ff8eaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8ec074080 0x7ff8ec199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:46.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.245+0000 7ff8eaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8ec074080 0x7ff8ec199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57288/0 (socket says 172.21.15.43:57288) 2024-03-26T08:31:46.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.245+0000 7ff8eaa00640 1 -- 172.21.15.43:0/1053567992 learned_addr learned my addr 172.21.15.43:0/1053567992 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:46.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.245+0000 7ff8eb400640 1 --2- 172.21.15.43:0/1053567992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8ec109450 0x7ff8ec1a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:46.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.245+0000 7ff8ea000640 1 --2- 172.21.15.43:0/1053567992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8ec075660 0x7ff8ec199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:46.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.245+0000 7ff8eaa00640 1 -- 172.21.15.43:0/1053567992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8ec075660 msgr2=0x7ff8ec199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:46.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.245+0000 7ff8eaa00640 1 --2- 172.21.15.43:0/1053567992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8ec075660 0x7ff8ec199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:46.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.245+0000 7ff8eaa00640 1 -- 172.21.15.43:0/1053567992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8ec109450 msgr2=0x7ff8ec1a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:46.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.245+0000 7ff8eaa00640 1 --2- 172.21.15.43:0/1053567992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8ec109450 0x7ff8ec1a0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:46.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.245+0000 7ff8eaa00640 1 -- 172.21.15.43:0/1053567992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ec1a1560 con 0x7ff8ec074080 2024-03-26T08:31:46.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.245+0000 7ff8eb400640 1 --2- 172.21.15.43:0/1053567992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8ec109450 0x7ff8ec1a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:31:46.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.246+0000 7ff8eaa00640 1 --2- 172.21.15.43:0/1053567992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8ec074080 0x7ff8ec199940 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00ea80 tx=0x7ff8dc00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:46.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.246+0000 7ff8e3400640 1 -- 172.21.15.43:0/1053567992 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc00ce40 con 0x7ff8ec074080 2024-03-26T08:31:46.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.246+0000 7ff8e3400640 1 -- 172.21.15.43:0/1053567992 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff8dc005020 con 0x7ff8ec074080 2024-03-26T08:31:46.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.246+0000 7ff8e3400640 1 -- 172.21.15.43:0/1053567992 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc0116d0 con 0x7ff8ec074080 2024-03-26T08:31:46.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.246+0000 7ff8ebe00640 1 -- 172.21.15.43:0/1053567992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8ec1a1850 con 0x7ff8ec074080 2024-03-26T08:31:46.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.246+0000 7ff8ebe00640 1 -- 172.21.15.43:0/1053567992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8ec1a1da0 con 0x7ff8ec074080 2024-03-26T08:31:46.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.247+0000 7ff8ebe00640 1 -- 172.21.15.43:0/1053567992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8ec108210 con 0x7ff8ec074080 2024-03-26T08:31:46.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.250+0000 7ff8e3400640 1 -- 172.21.15.43:0/1053567992 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff8dc0051d0 con 0x7ff8ec074080 2024-03-26T08:31:46.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.251+0000 7ff8e3400640 1 --2- 172.21.15.43:0/1053567992 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff8d00a2880 0x7ff8d00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:46.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.251+0000 7ff8e3400640 1 -- 172.21.15.43:0/1053567992 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7ff8dc0cf4d0 con 0x7ff8ec074080 2024-03-26T08:31:46.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.251+0000 7ff8ea000640 1 --2- 172.21.15.43:0/1053567992 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff8d00a2880 0x7ff8d00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:46.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.252+0000 7ff8ea000640 1 --2- 172.21.15.43:0/1053567992 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff8d00a2880 0x7ff8d00a4d40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8005fd0 tx=0x7ff8d8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:46.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.254+0000 7ff8e3400640 1 -- 172.21.15.43:0/1053567992 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff8dc016030 con 0x7ff8ec074080 2024-03-26T08:31:46.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.507+0000 7ff8ebe00640 1 -- 172.21.15.43:0/1053567992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff8ec02d070 con 0x7ff8ec074080 2024-03-26T08:31:46.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.538+0000 7ff8e3400640 1 -- 172.21.15.43:0/1053567992 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff8dc09c2b0 con 0x7ff8ec074080 2024-03-26T08:31:46.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.543+0000 7ff8ebe00640 1 -- 172.21.15.43:0/1053567992 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff8d00a2880 msgr2=0x7ff8d00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:46.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.543+0000 7ff8ebe00640 1 --2- 172.21.15.43:0/1053567992 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff8d00a2880 0x7ff8d00a4d40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8005fd0 tx=0x7ff8d8005850 comp rx=0 tx=0).stop 2024-03-26T08:31:46.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.543+0000 7ff8ebe00640 1 -- 172.21.15.43:0/1053567992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8ec074080 msgr2=0x7ff8ec199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:46.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.543+0000 7ff8ebe00640 1 --2- 172.21.15.43:0/1053567992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8ec074080 0x7ff8ec199940 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00ea80 tx=0x7ff8dc00ef40 comp rx=0 tx=0).stop 2024-03-26T08:31:46.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.543+0000 7ff8ebe00640 1 -- 172.21.15.43:0/1053567992 shutdown_connections 2024-03-26T08:31:46.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.543+0000 7ff8ebe00640 1 --2- 172.21.15.43:0/1053567992 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff8d00a2880 0x7ff8d00a4d40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:46.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.543+0000 7ff8ebe00640 1 --2- 172.21.15.43:0/1053567992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8ec109450 0x7ff8ec1a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:46.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.543+0000 7ff8ebe00640 1 --2- 172.21.15.43:0/1053567992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8ec075660 0x7ff8ec199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:46.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.543+0000 7ff8ebe00640 1 --2- 172.21.15.43:0/1053567992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8ec074080 0x7ff8ec199940 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:46.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.543+0000 7ff8ebe00640 1 -- 172.21.15.43:0/1053567992 >> 172.21.15.43:0/1053567992 conn(0x7ff8ec0fcb10 msgr2=0x7ff8ec1053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:46.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.543+0000 7ff8ebe00640 1 -- 172.21.15.43:0/1053567992 shutdown_connections 2024-03-26T08:31:46.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.543+0000 7ff8ebe00640 1 -- 172.21.15.43:0/1053567992 wait complete. 2024-03-26T08:31:46.555 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 1, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-03-26T08:31:46.555 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-26T08:31:46.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.797+0000 7ff675200640 1 -- 172.21.15.43:0/944502547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff670077c40 msgr2=0x7ff6700784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:46.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.797+0000 7ff675200640 1 --2- 172.21.15.43:0/944502547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff670077c40 0x7ff6700784e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff668009ca0 tx=0x7ff66802c0f0 comp rx=0 tx=0).stop 2024-03-26T08:31:46.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.797+0000 7ff675200640 1 -- 172.21.15.43:0/944502547 shutdown_connections 2024-03-26T08:31:46.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.797+0000 7ff675200640 1 --2- 172.21.15.43:0/944502547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff670077c40 0x7ff6700784e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:46.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.797+0000 7ff675200640 1 --2- 172.21.15.43:0/944502547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6700772a0 0x7ff670077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:46.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.797+0000 7ff675200640 1 --2- 172.21.15.43:0/944502547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff670076060 0x7ff670076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:46.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.797+0000 7ff675200640 1 -- 172.21.15.43:0/944502547 >> 172.21.15.43:0/944502547 conn(0x7ff6700fee50 msgr2=0x7ff670101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:46.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.797+0000 7ff675200640 1 -- 172.21.15.43:0/944502547 shutdown_connections 2024-03-26T08:31:46.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.797+0000 7ff675200640 1 -- 172.21.15.43:0/944502547 wait complete. 2024-03-26T08:31:46.831 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:46 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/993589181' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:31:46.831 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:46 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1786637888' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:31:46.831 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:46 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1053567992' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-26T08:31:46.831 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:46 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/993589181' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:31:46.831 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:46 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1786637888' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:31:46.831 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:46 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1053567992' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-26T08:31:46.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.798+0000 7ff675200640 1 Processor -- start 2024-03-26T08:31:46.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.798+0000 7ff675200640 1 -- start start 2024-03-26T08:31:46.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.799+0000 7ff675200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff670076060 0x7ff67019df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:46.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.799+0000 7ff675200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6700772a0 0x7ff67019e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:46.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.799+0000 7ff675200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff670077c40 0x7ff6701a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:46.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.799+0000 7ff675200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff670113840 con 0x7ff6700772a0 2024-03-26T08:31:46.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.799+0000 7ff675200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6701136c0 con 0x7ff670076060 2024-03-26T08:31:46.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.799+0000 7ff675200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6701139c0 con 0x7ff670077c40 2024-03-26T08:31:46.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.799+0000 7ff66d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6700772a0 0x7ff67019e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:46.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.799+0000 7ff66d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6700772a0 0x7ff67019e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57308/0 (socket says 172.21.15.43:57308) 2024-03-26T08:31:46.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.799+0000 7ff66d400640 1 -- 172.21.15.43:0/3927219033 learned_addr learned my addr 172.21.15.43:0/3927219033 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:46.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.799+0000 7ff66de00640 1 --2- 172.21.15.43:0/3927219033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff670076060 0x7ff67019df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:46.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.799+0000 7ff66e800640 1 --2- 172.21.15.43:0/3927219033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff670077c40 0x7ff6701a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:46.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.799+0000 7ff66d400640 1 -- 172.21.15.43:0/3927219033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff670077c40 msgr2=0x7ff6701a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:46.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.799+0000 7ff66d400640 1 --2- 172.21.15.43:0/3927219033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff670077c40 0x7ff6701a54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:46.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.799+0000 7ff66d400640 1 -- 172.21.15.43:0/3927219033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff670076060 msgr2=0x7ff67019df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:46.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.799+0000 7ff66d400640 1 --2- 172.21.15.43:0/3927219033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff670076060 0x7ff67019df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:46.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.799+0000 7ff66d400640 1 -- 172.21.15.43:0/3927219033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6701a5b60 con 0x7ff6700772a0 2024-03-26T08:31:46.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.800+0000 7ff66d400640 1 --2- 172.21.15.43:0/3927219033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6700772a0 0x7ff67019e480 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff65c00d950 tx=0x7ff65c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:46.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.800+0000 7ff666a00640 1 -- 172.21.15.43:0/3927219033 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65c017030 con 0x7ff6700772a0 2024-03-26T08:31:46.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.800+0000 7ff666a00640 1 -- 172.21.15.43:0/3927219033 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff65c004020 con 0x7ff6700772a0 2024-03-26T08:31:46.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.800+0000 7ff666a00640 1 -- 172.21.15.43:0/3927219033 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65c0047a0 con 0x7ff6700772a0 2024-03-26T08:31:46.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.800+0000 7ff675200640 1 -- 172.21.15.43:0/3927219033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6701a5e50 con 0x7ff6700772a0 2024-03-26T08:31:46.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.800+0000 7ff675200640 1 -- 172.21.15.43:0/3927219033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6701a62c0 con 0x7ff6700772a0 2024-03-26T08:31:46.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.802+0000 7ff675200640 1 -- 172.21.15.43:0/3927219033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff670068920 con 0x7ff6700772a0 2024-03-26T08:31:46.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.805+0000 7ff666a00640 1 -- 172.21.15.43:0/3927219033 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff65c0041d0 con 0x7ff6700772a0 2024-03-26T08:31:46.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.806+0000 7ff666a00640 1 --2- 172.21.15.43:0/3927219033 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff6540a2800 0x7ff6540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:46.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.806+0000 7ff666a00640 1 -- 172.21.15.43:0/3927219033 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7ff65c0ce980 con 0x7ff6700772a0 2024-03-26T08:31:46.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.806+0000 7ff66de00640 1 --2- 172.21.15.43:0/3927219033 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff6540a2800 0x7ff6540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:46.837 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:46 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/993589181' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:31:46.837 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:46 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1786637888' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:31:46.837 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:46 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1053567992' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-26T08:31:46.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.807+0000 7ff66de00640 1 --2- 172.21.15.43:0/3927219033 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff6540a2800 0x7ff6540a4cc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff660009890 tx=0x7ff660008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:46.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:46.811+0000 7ff666a00640 1 -- 172.21.15.43:0/3927219033 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff65c09b680 con 0x7ff6700772a0 2024-03-26T08:31:47.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.054+0000 7ff675200640 1 -- 172.21.15.43:0/3927219033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7ff6700622d0 con 0x7ff6700772a0 2024-03-26T08:31:47.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.554+0000 7ff666a00640 1 -- 172.21.15.43:0/3927219033 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) v1 ==== 121+0+0 (secure 0 0 0) 0x7ff65c09af80 con 0x7ff6700772a0 2024-03-26T08:31:47.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.558+0000 7ff675200640 1 -- 172.21.15.43:0/3927219033 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff6540a2800 msgr2=0x7ff6540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:47.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.558+0000 7ff675200640 1 --2- 172.21.15.43:0/3927219033 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff6540a2800 0x7ff6540a4cc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff660009890 tx=0x7ff660008040 comp rx=0 tx=0).stop 2024-03-26T08:31:47.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.558+0000 7ff675200640 1 -- 172.21.15.43:0/3927219033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6700772a0 msgr2=0x7ff67019e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:47.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.558+0000 7ff675200640 1 --2- 172.21.15.43:0/3927219033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6700772a0 0x7ff67019e480 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff65c00d950 tx=0x7ff65c00de10 comp rx=0 tx=0).stop 2024-03-26T08:31:47.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.559+0000 7ff675200640 1 -- 172.21.15.43:0/3927219033 shutdown_connections 2024-03-26T08:31:47.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.559+0000 7ff675200640 1 --2- 172.21.15.43:0/3927219033 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff6540a2800 0x7ff6540a4cc0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:47.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.559+0000 7ff675200640 1 --2- 172.21.15.43:0/3927219033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff670077c40 0x7ff6701a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:47.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.559+0000 7ff675200640 1 --2- 172.21.15.43:0/3927219033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6700772a0 0x7ff67019e480 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:47.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.559+0000 7ff675200640 1 --2- 172.21.15.43:0/3927219033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff670076060 0x7ff67019df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:47.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.559+0000 7ff675200640 1 -- 172.21.15.43:0/3927219033 >> 172.21.15.43:0/3927219033 conn(0x7ff6700fee50 msgr2=0x7ff670100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:47.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.559+0000 7ff675200640 1 -- 172.21.15.43:0/3927219033 shutdown_connections 2024-03-26T08:31:47.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.559+0000 7ff675200640 1 -- 172.21.15.43:0/3927219033 wait complete. 2024-03-26T08:31:47.579 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-26T08:31:47.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:47 smithi183 ceph-mon[35818]: pgmap v387: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 518 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-03-26T08:31:47.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:47 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1053567992' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-26T08:31:47.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:47 smithi183 ceph-mon[35818]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T08:31:47.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:47 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3927219033' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-26T08:31:47.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.818+0000 7f3e0a200640 1 -- 172.21.15.43:0/566156996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e04104ad0 msgr2=0x7f3e04104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:47.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.818+0000 7f3e0a200640 1 --2- 172.21.15.43:0/566156996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e04104ad0 0x7f3e04104f50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3df0009900 tx=0x7f3df002bc50 comp rx=0 tx=0).stop 2024-03-26T08:31:47.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.818+0000 7f3e0a200640 1 -- 172.21.15.43:0/566156996 shutdown_connections 2024-03-26T08:31:47.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.818+0000 7f3e0a200640 1 --2- 172.21.15.43:0/566156996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e04105490 0x7f3e0410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:47.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.818+0000 7f3e0a200640 1 --2- 172.21.15.43:0/566156996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e04104ad0 0x7f3e04104f50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:47.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.818+0000 7f3e0a200640 1 --2- 172.21.15.43:0/566156996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e041038e0 0x7f3e04103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:47.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.818+0000 7f3e0a200640 1 -- 172.21.15.43:0/566156996 >> 172.21.15.43:0/566156996 conn(0x7f3e040fee50 msgr2=0x7f3e04101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:47.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.818+0000 7f3e0a200640 1 -- 172.21.15.43:0/566156996 shutdown_connections 2024-03-26T08:31:47.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.818+0000 7f3e0a200640 1 -- 172.21.15.43:0/566156996 wait complete. 2024-03-26T08:31:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.819+0000 7f3e0a200640 1 Processor -- start 2024-03-26T08:31:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.820+0000 7f3e0a200640 1 -- start start 2024-03-26T08:31:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.820+0000 7f3e0a200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e041038e0 0x7f3e04199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.820+0000 7f3e0a200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e04104ad0 0x7f3e0419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.820+0000 7f3e0a200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e04105490 0x7f3e041a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.820+0000 7f3e0a200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0410f3f0 con 0x7f3e04105490 2024-03-26T08:31:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.820+0000 7f3e0a200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0410f270 con 0x7f3e041038e0 2024-03-26T08:31:47.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.820+0000 7f3e0a200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0410f570 con 0x7f3e04104ad0 2024-03-26T08:31:47.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.820+0000 7f3e03e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e04105490 0x7f3e041a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:47.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.820+0000 7f3e03e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e04105490 0x7f3e041a10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57316/0 (socket says 172.21.15.43:57316) 2024-03-26T08:31:47.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.820+0000 7f3e03e00640 1 -- 172.21.15.43:0/1186877598 learned_addr learned my addr 172.21.15.43:0/1186877598 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:47.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.821+0000 7f3e03400640 1 --2- 172.21.15.43:0/1186877598 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e041038e0 0x7f3e04199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:47.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.821+0000 7f3e02a00640 1 --2- 172.21.15.43:0/1186877598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e04104ad0 0x7f3e0419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:47.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.821+0000 7f3e03e00640 1 -- 172.21.15.43:0/1186877598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e04104ad0 msgr2=0x7f3e0419a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:47.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.821+0000 7f3e03e00640 1 --2- 172.21.15.43:0/1186877598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e04104ad0 0x7f3e0419a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:47.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.821+0000 7f3e03e00640 1 -- 172.21.15.43:0/1186877598 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e041038e0 msgr2=0x7f3e04199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:47.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.821+0000 7f3e03e00640 1 --2- 172.21.15.43:0/1186877598 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e041038e0 0x7f3e04199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:47.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.821+0000 7f3e03e00640 1 -- 172.21.15.43:0/1186877598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e041a1750 con 0x7f3e04105490 2024-03-26T08:31:47.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.821+0000 7f3e03e00640 1 --2- 172.21.15.43:0/1186877598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e04105490 0x7f3e041a10e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc00eaa0 tx=0x7f3dfc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:47.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.822+0000 7f3dfbe00640 1 -- 172.21.15.43:0/1186877598 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc00cdb0 con 0x7f3e04105490 2024-03-26T08:31:47.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.822+0000 7f3dfbe00640 1 -- 172.21.15.43:0/1186877598 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3dfc005020 con 0x7f3e04105490 2024-03-26T08:31:47.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.822+0000 7f3dfbe00640 1 -- 172.21.15.43:0/1186877598 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc0116d0 con 0x7f3e04105490 2024-03-26T08:31:47.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.822+0000 7f3e0a200640 1 -- 172.21.15.43:0/1186877598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e041a1a40 con 0x7f3e04105490 2024-03-26T08:31:47.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.822+0000 7f3e0a200640 1 -- 172.21.15.43:0/1186877598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e041a1f40 con 0x7f3e04105490 2024-03-26T08:31:47.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.823+0000 7f3e0a200640 1 -- 172.21.15.43:0/1186877598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e0410ad20 con 0x7f3e04105490 2024-03-26T08:31:47.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.825+0000 7f3dfbe00640 1 -- 172.21.15.43:0/1186877598 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f3dfc0051d0 con 0x7f3e04105490 2024-03-26T08:31:47.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.826+0000 7f3dfbe00640 1 --2- 172.21.15.43:0/1186877598 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3de80a2800 0x7f3de80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:47.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.826+0000 7f3dfbe00640 1 -- 172.21.15.43:0/1186877598 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f3dfc0cf0c0 con 0x7f3e04105490 2024-03-26T08:31:47.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.827+0000 7f3e03400640 1 --2- 172.21.15.43:0/1186877598 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3de80a2800 0x7f3de80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:47.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.827+0000 7f3e03400640 1 --2- 172.21.15.43:0/1186877598 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3de80a2800 0x7f3de80a4cc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3df4005fd0 tx=0x7f3df4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:47.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:47 smithi052 ceph-mon[35071]: pgmap v387: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 518 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-03-26T08:31:47.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:47 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1053567992' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-26T08:31:47.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:47 smithi052 ceph-mon[35071]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T08:31:47.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:47 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3927219033' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-26T08:31:47.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:47.831+0000 7f3dfbe00640 1 -- 172.21.15.43:0/1186877598 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3dfc016030 con 0x7f3e04105490 2024-03-26T08:31:47.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:47 smithi043 ceph-mon[31929]: pgmap v387: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 518 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-03-26T08:31:47.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:47 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1053567992' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-26T08:31:47.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:47 smithi043 ceph-mon[31929]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T08:31:47.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:47 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3927219033' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-26T08:31:48.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.078+0000 7f3e0a200640 1 -- 172.21.15.43:0/1186877598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f3e040622a0 con 0x7f3e04105490 2024-03-26T08:31:48.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.568+0000 7f3dfbe00640 1 -- 172.21.15.43:0/1186877598 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) v1 ==== 115+0+0 (secure 0 0 0) 0x7f3dfc09bdc0 con 0x7f3e04105490 2024-03-26T08:31:48.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.574+0000 7f3e0a200640 1 -- 172.21.15.43:0/1186877598 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3de80a2800 msgr2=0x7f3de80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:48.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.574+0000 7f3e0a200640 1 --2- 172.21.15.43:0/1186877598 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3de80a2800 0x7f3de80a4cc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3df4005fd0 tx=0x7f3df4005850 comp rx=0 tx=0).stop 2024-03-26T08:31:48.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.574+0000 7f3e0a200640 1 -- 172.21.15.43:0/1186877598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e04105490 msgr2=0x7f3e041a10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:48.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.574+0000 7f3e0a200640 1 --2- 172.21.15.43:0/1186877598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e04105490 0x7f3e041a10e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc00eaa0 tx=0x7f3dfc00ef60 comp rx=0 tx=0).stop 2024-03-26T08:31:48.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.574+0000 7f3e0a200640 1 -- 172.21.15.43:0/1186877598 shutdown_connections 2024-03-26T08:31:48.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.574+0000 7f3e0a200640 1 --2- 172.21.15.43:0/1186877598 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3de80a2800 0x7f3de80a4cc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:48.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.574+0000 7f3e0a200640 1 --2- 172.21.15.43:0/1186877598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e04105490 0x7f3e041a10e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:48.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.574+0000 7f3e0a200640 1 --2- 172.21.15.43:0/1186877598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e04104ad0 0x7f3e0419a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:48.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.574+0000 7f3e0a200640 1 --2- 172.21.15.43:0/1186877598 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e041038e0 0x7f3e04199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:48.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.575+0000 7f3e0a200640 1 -- 172.21.15.43:0/1186877598 >> 172.21.15.43:0/1186877598 conn(0x7f3e040fee50 msgr2=0x7f3e04107ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:48.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.575+0000 7f3e0a200640 1 -- 172.21.15.43:0/1186877598 shutdown_connections 2024-03-26T08:31:48.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.575+0000 7f3e0a200640 1 -- 172.21.15.43:0/1186877598 wait complete. 2024-03-26T08:31:48.590 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-03-26T08:31:48.591 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-26T08:31:48.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:48 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3927219033' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-26T08:31:48.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:48 smithi052 ceph-mon[35071]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T08:31:48.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:48 smithi052 ceph-mon[35071]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-03-26T08:31:48.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:48 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1186877598' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-26T08:31:48.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.832+0000 7f123fe00640 1 -- 172.21.15.43:0/933460248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1240105280 msgr2=0x7f124010bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:48.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.832+0000 7f123fe00640 1 --2- 172.21.15.43:0/933460248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1240105280 0x7f124010bb10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1238009ca0 tx=0x7f123802c100 comp rx=0 tx=0).stop 2024-03-26T08:31:48.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.832+0000 7f123fe00640 1 -- 172.21.15.43:0/933460248 shutdown_connections 2024-03-26T08:31:48.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.832+0000 7f123fe00640 1 --2- 172.21.15.43:0/933460248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1240105280 0x7f124010bb10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:48.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.832+0000 7f123fe00640 1 --2- 172.21.15.43:0/933460248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12401048c0 0x7f1240104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:48.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.832+0000 7f123fe00640 1 --2- 172.21.15.43:0/933460248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12401036d0 0x7f1240103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:48.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.832+0000 7f123fe00640 1 -- 172.21.15.43:0/933460248 >> 172.21.15.43:0/933460248 conn(0x7f12400fec40 msgr2=0x7f1240101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:48.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.832+0000 7f123fe00640 1 -- 172.21.15.43:0/933460248 shutdown_connections 2024-03-26T08:31:48.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.832+0000 7f123fe00640 1 -- 172.21.15.43:0/933460248 wait complete. 2024-03-26T08:31:48.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.833+0000 7f123fe00640 1 Processor -- start 2024-03-26T08:31:48.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.833+0000 7f123fe00640 1 -- start start 2024-03-26T08:31:48.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.834+0000 7f123fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12401036d0 0x7f1240199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.834+0000 7f123fe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12401048c0 0x7f1240199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.834+0000 7f123fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1240105280 0x7f12401a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.834+0000 7f123fe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124010f1e0 con 0x7f1240105280 2024-03-26T08:31:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.834+0000 7f123fe00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124010f060 con 0x7f12401048c0 2024-03-26T08:31:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.834+0000 7f123fe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124010f360 con 0x7f12401036d0 2024-03-26T08:31:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.834+0000 7f123f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1240105280 0x7f12401a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.834+0000 7f123f400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1240105280 0x7f12401a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57334/0 (socket says 172.21.15.43:57334) 2024-03-26T08:31:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.834+0000 7f123f400640 1 -- 172.21.15.43:0/2008079710 learned_addr learned my addr 172.21.15.43:0/2008079710 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.834+0000 7f123ea00640 1 --2- 172.21.15.43:0/2008079710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12401036d0 0x7f1240199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.834+0000 7f123e000640 1 --2- 172.21.15.43:0/2008079710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12401048c0 0x7f1240199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.834+0000 7f123f400640 1 -- 172.21.15.43:0/2008079710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12401036d0 msgr2=0x7f1240199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.834+0000 7f123f400640 1 --2- 172.21.15.43:0/2008079710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12401036d0 0x7f1240199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.834+0000 7f123f400640 1 -- 172.21.15.43:0/2008079710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12401048c0 msgr2=0x7f1240199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.835+0000 7f123f400640 1 --2- 172.21.15.43:0/2008079710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12401048c0 0x7f1240199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:48.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.835+0000 7f123f400640 1 -- 172.21.15.43:0/2008079710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12401a1350 con 0x7f1240105280 2024-03-26T08:31:48.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.835+0000 7f123e000640 1 --2- 172.21.15.43:0/2008079710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12401048c0 0x7f1240199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:31:48.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.835+0000 7f123f400640 1 --2- 172.21.15.43:0/2008079710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1240105280 0x7f12401a0ce0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1238032380 tx=0x7f1238035e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:48.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.835+0000 7f1237400640 1 -- 172.21.15.43:0/2008079710 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1238039040 con 0x7f1240105280 2024-03-26T08:31:48.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.835+0000 7f1237400640 1 -- 172.21.15.43:0/2008079710 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1238044070 con 0x7f1240105280 2024-03-26T08:31:48.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.835+0000 7f1237400640 1 -- 172.21.15.43:0/2008079710 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1238004670 con 0x7f1240105280 2024-03-26T08:31:48.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.835+0000 7f123fe00640 1 -- 172.21.15.43:0/2008079710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12401a15e0 con 0x7f1240105280 2024-03-26T08:31:48.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.835+0000 7f123fe00640 1 -- 172.21.15.43:0/2008079710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12401a1a80 con 0x7f1240105280 2024-03-26T08:31:48.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.836+0000 7f123fe00640 1 -- 172.21.15.43:0/2008079710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f124010a920 con 0x7f1240105280 2024-03-26T08:31:48.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.839+0000 7f1237400640 1 -- 172.21.15.43:0/2008079710 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1238003970 con 0x7f1240105280 2024-03-26T08:31:48.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.840+0000 7f1237400640 1 --2- 172.21.15.43:0/2008079710 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f12240a2800 0x7f12240a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:48.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.840+0000 7f1237400640 1 -- 172.21.15.43:0/2008079710 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f12380f1e40 con 0x7f1240105280 2024-03-26T08:31:48.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.840+0000 7f123ea00640 1 --2- 172.21.15.43:0/2008079710 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f12240a2800 0x7f12240a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:48.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.841+0000 7f123ea00640 1 --2- 172.21.15.43:0/2008079710 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f12240a2800 0x7f12240a4cc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f12300058c0 tx=0x7f1230005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:48.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:48.842+0000 7f1237400640 1 -- 172.21.15.43:0/2008079710 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f12380beb40 con 0x7f1240105280 2024-03-26T08:31:48.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:48 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3927219033' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-26T08:31:48.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:48 smithi043 ceph-mon[31929]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T08:31:48.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:48 smithi043 ceph-mon[31929]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-03-26T08:31:48.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:48 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1186877598' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-26T08:31:49.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:48 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3927219033' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-26T08:31:49.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:48 smithi183 ceph-mon[35818]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T08:31:49.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:48 smithi183 ceph-mon[35818]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-03-26T08:31:49.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:48 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1186877598' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-26T08:31:49.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.074+0000 7f123fe00640 1 -- 172.21.15.43:0/2008079710 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f1240061ae0 con 0x7f12240a2800 2024-03-26T08:31:49.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.120+0000 7f1237400640 1 -- 172.21.15.43:0/2008079710 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1240061ae0 con 0x7f12240a2800 2024-03-26T08:31:49.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.123+0000 7f123fe00640 1 -- 172.21.15.43:0/2008079710 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f12240a2800 msgr2=0x7f12240a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:49.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.123+0000 7f123fe00640 1 --2- 172.21.15.43:0/2008079710 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f12240a2800 0x7f12240a4cc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f12300058c0 tx=0x7f1230005850 comp rx=0 tx=0).stop 2024-03-26T08:31:49.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.123+0000 7f123fe00640 1 -- 172.21.15.43:0/2008079710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1240105280 msgr2=0x7f12401a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:49.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.123+0000 7f123fe00640 1 --2- 172.21.15.43:0/2008079710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1240105280 0x7f12401a0ce0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1238032380 tx=0x7f1238035e20 comp rx=0 tx=0).stop 2024-03-26T08:31:49.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.123+0000 7f123fe00640 1 -- 172.21.15.43:0/2008079710 shutdown_connections 2024-03-26T08:31:49.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.123+0000 7f123fe00640 1 --2- 172.21.15.43:0/2008079710 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f12240a2800 0x7f12240a4cc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:49.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.123+0000 7f123fe00640 1 --2- 172.21.15.43:0/2008079710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1240105280 0x7f12401a0ce0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:49.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.123+0000 7f123fe00640 1 --2- 172.21.15.43:0/2008079710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12401048c0 0x7f1240199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:49.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.123+0000 7f123fe00640 1 --2- 172.21.15.43:0/2008079710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12401036d0 0x7f1240199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:49.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.123+0000 7f123fe00640 1 -- 172.21.15.43:0/2008079710 >> 172.21.15.43:0/2008079710 conn(0x7f12400fec40 msgr2=0x7f1240107af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:49.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.124+0000 7f123fe00640 1 -- 172.21.15.43:0/2008079710 shutdown_connections 2024-03-26T08:31:49.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.124+0000 7f123fe00640 1 -- 172.21.15.43:0/2008079710 wait complete. 2024-03-26T08:31:49.137 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-26T08:31:49.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.359+0000 7f38b1000640 1 -- 172.21.15.43:0/583190281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ac102000 msgr2=0x7f38ac1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:49.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.359+0000 7f38b1000640 1 --2- 172.21.15.43:0/583190281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ac102000 0x7f38ac1043f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f389c009900 tx=0x7f389c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:31:49.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.359+0000 7f38b1000640 1 -- 172.21.15.43:0/583190281 shutdown_connections 2024-03-26T08:31:49.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.359+0000 7f38b1000640 1 --2- 172.21.15.43:0/583190281 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38ac107260 0x7f38ac1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:49.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.359+0000 7f38b1000640 1 --2- 172.21.15.43:0/583190281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38ac104930 0x7f38ac106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:49.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.359+0000 7f38b1000640 1 --2- 172.21.15.43:0/583190281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ac102000 0x7f38ac1043f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:49.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.359+0000 7f38b1000640 1 -- 172.21.15.43:0/583190281 >> 172.21.15.43:0/583190281 conn(0x7f38ac0fb9f0 msgr2=0x7f38ac0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:49.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.360+0000 7f38b1000640 1 -- 172.21.15.43:0/583190281 shutdown_connections 2024-03-26T08:31:49.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.360+0000 7f38b1000640 1 -- 172.21.15.43:0/583190281 wait complete. 2024-03-26T08:31:49.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.361+0000 7f38b1000640 1 Processor -- start 2024-03-26T08:31:49.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.361+0000 7f38b1000640 1 -- start start 2024-03-26T08:31:49.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.361+0000 7f38b1000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38ac102000 0x7f38ac1977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:49.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.361+0000 7f38b1000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ac104930 0x7f38ac197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:49.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.361+0000 7f38b1000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38ac107260 0x7f38ac19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:49.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.361+0000 7f38b1000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac10d110 con 0x7f38ac104930 2024-03-26T08:31:49.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.361+0000 7f38b1000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac10cf90 con 0x7f38ac107260 2024-03-26T08:31:49.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.361+0000 7f38b1000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ac10d290 con 0x7f38ac102000 2024-03-26T08:31:49.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.361+0000 7f38aaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ac104930 0x7f38ac197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:49.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.362+0000 7f38aaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ac104930 0x7f38ac197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57392/0 (socket says 172.21.15.43:57392) 2024-03-26T08:31:49.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.362+0000 7f38aaa00640 1 -- 172.21.15.43:0/3478845662 learned_addr learned my addr 172.21.15.43:0/3478845662 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:49.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.362+0000 7f38ab400640 1 --2- 172.21.15.43:0/3478845662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38ac102000 0x7f38ac1977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:49.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.362+0000 7f38abe00640 1 --2- 172.21.15.43:0/3478845662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38ac107260 0x7f38ac19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:49.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.362+0000 7f38aaa00640 1 -- 172.21.15.43:0/3478845662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38ac102000 msgr2=0x7f38ac1977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:49.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.362+0000 7f38aaa00640 1 --2- 172.21.15.43:0/3478845662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38ac102000 0x7f38ac1977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:49.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.362+0000 7f38aaa00640 1 -- 172.21.15.43:0/3478845662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38ac107260 msgr2=0x7f38ac19ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:49.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.362+0000 7f38aaa00640 1 --2- 172.21.15.43:0/3478845662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38ac107260 0x7f38ac19ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:49.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.362+0000 7f38aaa00640 1 -- 172.21.15.43:0/3478845662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ac19f400 con 0x7f38ac104930 2024-03-26T08:31:49.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.362+0000 7f38ab400640 1 --2- 172.21.15.43:0/3478845662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38ac102000 0x7f38ac1977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:31:49.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.362+0000 7f38aaa00640 1 --2- 172.21.15.43:0/3478845662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ac104930 0x7f38ac197d20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f389800d740 tx=0x7f389800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:49.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.363+0000 7f38a3e00640 1 -- 172.21.15.43:0/3478845662 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3898017070 con 0x7f38ac104930 2024-03-26T08:31:49.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.363+0000 7f38a3e00640 1 -- 172.21.15.43:0/3478845662 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f389800de50 con 0x7f38ac104930 2024-03-26T08:31:49.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.363+0000 7f38b1000640 1 -- 172.21.15.43:0/3478845662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ac19f6f0 con 0x7f38ac104930 2024-03-26T08:31:49.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.363+0000 7f38a3e00640 1 -- 172.21.15.43:0/3478845662 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3898004d60 con 0x7f38ac104930 2024-03-26T08:31:49.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.363+0000 7f38b1000640 1 -- 172.21.15.43:0/3478845662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ac19fbf0 con 0x7f38ac104930 2024-03-26T08:31:49.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.364+0000 7f38b1000640 1 -- 172.21.15.43:0/3478845662 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38ac0fd4e0 con 0x7f38ac104930 2024-03-26T08:31:49.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.367+0000 7f38a3e00640 1 -- 172.21.15.43:0/3478845662 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f3898004f00 con 0x7f38ac104930 2024-03-26T08:31:49.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.369+0000 7f38a3e00640 1 --2- 172.21.15.43:0/3478845662 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f38900a2800 0x7f38900a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:49.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.369+0000 7f38a3e00640 1 -- 172.21.15.43:0/3478845662 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f38980ce860 con 0x7f38ac104930 2024-03-26T08:31:49.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.370+0000 7f38ab400640 1 --2- 172.21.15.43:0/3478845662 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f38900a2800 0x7f38900a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:49.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.370+0000 7f38ab400640 1 --2- 172.21.15.43:0/3478845662 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f38900a2800 0x7f38900a4cc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f389c006010 tx=0x7f389c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:49.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.374+0000 7f38a3e00640 1 -- 172.21.15.43:0/3478845662 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f389809b560 con 0x7f38ac104930 2024-03-26T08:31:49.402 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:31:49 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:31:49.083+0000 7fbed7e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T08:31:49.402 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:31:49 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:31:49.083+0000 7fbed7e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T08:31:49.402 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:31:49 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:31:49.083+0000 7fbed7e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T08:31:49.403 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:31:49 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:31:49.083+0000 7fbed7e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T08:31:49.403 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:31:49 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:31:49.083+0000 7fbed7e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T08:31:49.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.611+0000 7f38b1000640 1 -- 172.21.15.43:0/3478845662 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f38ac061b70 con 0x7f38900a2800 2024-03-26T08:31:49.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.681+0000 7f38a3e00640 1 -- 172.21.15.43:0/3478845662 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f38ac061b70 con 0x7f38900a2800 2024-03-26T08:31:49.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.684+0000 7f38b1000640 1 -- 172.21.15.43:0/3478845662 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f38900a2800 msgr2=0x7f38900a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:49.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.684+0000 7f38b1000640 1 --2- 172.21.15.43:0/3478845662 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f38900a2800 0x7f38900a4cc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f389c006010 tx=0x7f389c02c7e0 comp rx=0 tx=0).stop 2024-03-26T08:31:49.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.684+0000 7f38b1000640 1 -- 172.21.15.43:0/3478845662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ac104930 msgr2=0x7f38ac197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:49.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.684+0000 7f38b1000640 1 --2- 172.21.15.43:0/3478845662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ac104930 0x7f38ac197d20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f389800d740 tx=0x7f389800dc00 comp rx=0 tx=0).stop 2024-03-26T08:31:49.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.684+0000 7f38b1000640 1 -- 172.21.15.43:0/3478845662 shutdown_connections 2024-03-26T08:31:49.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.684+0000 7f38b1000640 1 --2- 172.21.15.43:0/3478845662 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f38900a2800 0x7f38900a4cc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:49.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.684+0000 7f38b1000640 1 --2- 172.21.15.43:0/3478845662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38ac107260 0x7f38ac19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:49.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.684+0000 7f38b1000640 1 --2- 172.21.15.43:0/3478845662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38ac104930 0x7f38ac197d20 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:49.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.684+0000 7f38b1000640 1 --2- 172.21.15.43:0/3478845662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38ac102000 0x7f38ac1977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:49.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.684+0000 7f38b1000640 1 -- 172.21.15.43:0/3478845662 >> 172.21.15.43:0/3478845662 conn(0x7f38ac0fb9f0 msgr2=0x7f38ac102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:49.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.684+0000 7f38b1000640 1 -- 172.21.15.43:0/3478845662 shutdown_connections 2024-03-26T08:31:49.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.684+0000 7f38b1000640 1 -- 172.21.15.43:0/3478845662 wait complete. 2024-03-26T08:31:49.699 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-26T08:31:49.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:49 smithi052 ceph-mon[35071]: pgmap v388: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 94 B/s rd, 471 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-03-26T08:31:49.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:49 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1186877598' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-26T08:31:49.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:49 smithi052 ceph-mon[35071]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-03-26T08:31:49.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:49 smithi052 ceph-mon[35071]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:31:49.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:49 smithi043 ceph-mon[31929]: pgmap v388: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 94 B/s rd, 471 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-03-26T08:31:49.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:49 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1186877598' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-26T08:31:49.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:49 smithi043 ceph-mon[31929]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-03-26T08:31:49.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:49 smithi043 ceph-mon[31929]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:31:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.941+0000 7fec67200640 1 -- 172.21.15.43:0/3533585050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec68105350 msgr2=0x7fec6810bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.941+0000 7fec67200640 1 --2- 172.21.15.43:0/3533585050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec68105350 0x7fec6810bbe0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fec60009900 tx=0x7fec6002bc30 comp rx=0 tx=0).stop 2024-03-26T08:31:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.942+0000 7fec67200640 1 -- 172.21.15.43:0/3533585050 shutdown_connections 2024-03-26T08:31:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.942+0000 7fec67200640 1 --2- 172.21.15.43:0/3533585050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec68105350 0x7fec6810bbe0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.942+0000 7fec67200640 1 --2- 172.21.15.43:0/3533585050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec68104990 0x7fec68104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.942+0000 7fec67200640 1 --2- 172.21.15.43:0/3533585050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec681037a0 0x7fec68103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.942+0000 7fec67200640 1 -- 172.21.15.43:0/3533585050 >> 172.21.15.43:0/3533585050 conn(0x7fec680fed10 msgr2=0x7fec68101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.942+0000 7fec67200640 1 -- 172.21.15.43:0/3533585050 shutdown_connections 2024-03-26T08:31:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.942+0000 7fec67200640 1 -- 172.21.15.43:0/3533585050 wait complete. 2024-03-26T08:31:49.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.943+0000 7fec67200640 1 Processor -- start 2024-03-26T08:31:49.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.943+0000 7fec67200640 1 -- start start 2024-03-26T08:31:49.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.943+0000 7fec67200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec681037a0 0x7fec68199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:49.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.943+0000 7fec67200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec68104990 0x7fec68199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:49.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.943+0000 7fec67200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec68105350 0x7fec681a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:49.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.943+0000 7fec67200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6810f2d0 con 0x7fec681037a0 2024-03-26T08:31:49.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.943+0000 7fec67200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6810f150 con 0x7fec68104990 2024-03-26T08:31:49.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.943+0000 7fec67200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6810f450 con 0x7fec68105350 2024-03-26T08:31:49.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.943+0000 7fec65e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec681037a0 0x7fec68199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:49.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.944+0000 7fec65e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec681037a0 0x7fec68199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57404/0 (socket says 172.21.15.43:57404) 2024-03-26T08:31:49.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.944+0000 7fec65e00640 1 -- 172.21.15.43:0/255802770 learned_addr learned my addr 172.21.15.43:0/255802770 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:49.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.944+0000 7fec65e00640 1 -- 172.21.15.43:0/255802770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec68105350 msgr2=0x7fec681a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:49.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.944+0000 7fec66800640 1 --2- 172.21.15.43:0/255802770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec68105350 0x7fec681a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:49.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.944+0000 7fec65400640 1 --2- 172.21.15.43:0/255802770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec68104990 0x7fec68199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:49.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.944+0000 7fec65e00640 1 --2- 172.21.15.43:0/255802770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec68105350 0x7fec681a0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:49.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.944+0000 7fec65e00640 1 -- 172.21.15.43:0/255802770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec68104990 msgr2=0x7fec68199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:49.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.944+0000 7fec65e00640 1 --2- 172.21.15.43:0/255802770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec68104990 0x7fec68199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:49.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.944+0000 7fec65e00640 1 -- 172.21.15.43:0/255802770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec681a1560 con 0x7fec681037a0 2024-03-26T08:31:49.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.944+0000 7fec65e00640 1 --2- 172.21.15.43:0/255802770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec681037a0 0x7fec68199940 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fec5800eaa0 tx=0x7fec5800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:49.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.944+0000 7fec5ea00640 1 -- 172.21.15.43:0/255802770 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec58016070 con 0x7fec681037a0 2024-03-26T08:31:49.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.945+0000 7fec5ea00640 1 -- 172.21.15.43:0/255802770 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fec58003e20 con 0x7fec681037a0 2024-03-26T08:31:49.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.945+0000 7fec67200640 1 -- 172.21.15.43:0/255802770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec681a1850 con 0x7fec681037a0 2024-03-26T08:31:49.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.945+0000 7fec5ea00640 1 -- 172.21.15.43:0/255802770 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec580116d0 con 0x7fec681037a0 2024-03-26T08:31:49.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.945+0000 7fec67200640 1 -- 172.21.15.43:0/255802770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec681a1d50 con 0x7fec681037a0 2024-03-26T08:31:49.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.946+0000 7fec67200640 1 -- 172.21.15.43:0/255802770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec6810aac0 con 0x7fec681037a0 2024-03-26T08:31:49.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.948+0000 7fec5ea00640 1 -- 172.21.15.43:0/255802770 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fec58019070 con 0x7fec681037a0 2024-03-26T08:31:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.949+0000 7fec5ea00640 1 --2- 172.21.15.43:0/255802770 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fec4c0a2800 0x7fec4c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.949+0000 7fec5ea00640 1 -- 172.21.15.43:0/255802770 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7fec580cef40 con 0x7fec681037a0 2024-03-26T08:31:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.950+0000 7fec65400640 1 --2- 172.21.15.43:0/255802770 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fec4c0a2800 0x7fec4c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:49.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.950+0000 7fec65400640 1 --2- 172.21.15.43:0/255802770 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fec4c0a2800 0x7fec4c0a4cc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fec50005fd0 tx=0x7fec50009470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:49.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:49.954+0000 7fec5ea00640 1 -- 172.21.15.43:0/255802770 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fec5809bca0 con 0x7fec681037a0 2024-03-26T08:31:50.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:49 smithi183 ceph-mon[35818]: pgmap v388: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 94 B/s rd, 471 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-03-26T08:31:50.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:49 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1186877598' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-26T08:31:50.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:49 smithi183 ceph-mon[35818]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-03-26T08:31:50.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:49 smithi183 ceph-mon[35818]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:31:50.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.185+0000 7fec67200640 1 -- 172.21.15.43:0/255802770 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fec68061ae0 con 0x7fec4c0a2800 2024-03-26T08:31:50.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.200+0000 7fec5ea00640 1 -- 172.21.15.43:0/255802770 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fec68061ae0 con 0x7fec4c0a2800 2024-03-26T08:31:50.193 INFO:teuthology.orchestra.run.smithi043.stdout:/volumes/qa/sv_0/32b40a1c-a81f-4074-a091-7196526b98fa 2024-03-26T08:31:50.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.202+0000 7fec67200640 1 -- 172.21.15.43:0/255802770 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fec4c0a2800 msgr2=0x7fec4c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:50.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.202+0000 7fec67200640 1 --2- 172.21.15.43:0/255802770 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fec4c0a2800 0x7fec4c0a4cc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fec50005fd0 tx=0x7fec50009470 comp rx=0 tx=0).stop 2024-03-26T08:31:50.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.202+0000 7fec67200640 1 -- 172.21.15.43:0/255802770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec681037a0 msgr2=0x7fec68199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:50.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.202+0000 7fec67200640 1 --2- 172.21.15.43:0/255802770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec681037a0 0x7fec68199940 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fec5800eaa0 tx=0x7fec5800ef60 comp rx=0 tx=0).stop 2024-03-26T08:31:50.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.202+0000 7fec67200640 1 -- 172.21.15.43:0/255802770 shutdown_connections 2024-03-26T08:31:50.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.202+0000 7fec67200640 1 --2- 172.21.15.43:0/255802770 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fec4c0a2800 0x7fec4c0a4cc0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:50.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.202+0000 7fec67200640 1 --2- 172.21.15.43:0/255802770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec68105350 0x7fec681a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:50.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.202+0000 7fec67200640 1 --2- 172.21.15.43:0/255802770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec68104990 0x7fec68199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:50.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.202+0000 7fec67200640 1 --2- 172.21.15.43:0/255802770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec681037a0 0x7fec68199940 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:50.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.202+0000 7fec67200640 1 -- 172.21.15.43:0/255802770 >> 172.21.15.43:0/255802770 conn(0x7fec680fed10 msgr2=0x7fec68107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:50.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.203+0000 7fec67200640 1 -- 172.21.15.43:0/255802770 shutdown_connections 2024-03-26T08:31:50.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.203+0000 7fec67200640 1 -- 172.21.15.43:0/255802770 wait complete. 2024-03-26T08:31:50.215 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-26T08:31:50.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.461+0000 7fd82a400640 1 -- 172.21.15.43:0/1577307150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd824104990 msgr2=0x7fd824106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:50.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.461+0000 7fd82a400640 1 --2- 172.21.15.43:0/1577307150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd824104990 0x7fd824106d80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd814009ca0 tx=0x7fd81402c0f0 comp rx=0 tx=0).stop 2024-03-26T08:31:50.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.461+0000 7fd82a400640 1 -- 172.21.15.43:0/1577307150 shutdown_connections 2024-03-26T08:31:50.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.461+0000 7fd82a400640 1 --2- 172.21.15.43:0/1577307150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8241072c0 0x7fd8241096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:50.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.461+0000 7fd82a400640 1 --2- 172.21.15.43:0/1577307150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd824104990 0x7fd824106d80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:50.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.461+0000 7fd82a400640 1 --2- 172.21.15.43:0/1577307150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd824102060 0x7fd824104450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:50.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.461+0000 7fd82a400640 1 -- 172.21.15.43:0/1577307150 >> 172.21.15.43:0/1577307150 conn(0x7fd8240fba10 msgr2=0x7fd8240fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:50.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.461+0000 7fd82a400640 1 -- 172.21.15.43:0/1577307150 shutdown_connections 2024-03-26T08:31:50.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.461+0000 7fd82a400640 1 -- 172.21.15.43:0/1577307150 wait complete. 2024-03-26T08:31:50.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.462+0000 7fd82a400640 1 Processor -- start 2024-03-26T08:31:50.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.462+0000 7fd82a400640 1 -- start start 2024-03-26T08:31:50.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.462+0000 7fd82a400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd824102060 0x7fd824197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:50.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.462+0000 7fd82a400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd824104990 0x7fd824197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:50.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.462+0000 7fd82a400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8241072c0 0x7fd82419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:50.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.462+0000 7fd82a400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82410d0c0 con 0x7fd824102060 2024-03-26T08:31:50.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.462+0000 7fd82a400640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82410cf40 con 0x7fd824104990 2024-03-26T08:31:50.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.462+0000 7fd82a400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82410d240 con 0x7fd8241072c0 2024-03-26T08:31:50.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.463+0000 7fd829000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd824102060 0x7fd824197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:50.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.463+0000 7fd829000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd824102060 0x7fd824197810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57426/0 (socket says 172.21.15.43:57426) 2024-03-26T08:31:50.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.463+0000 7fd829000640 1 -- 172.21.15.43:0/3694158035 learned_addr learned my addr 172.21.15.43:0/3694158035 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:50.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.463+0000 7fd829a00640 1 --2- 172.21.15.43:0/3694158035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8241072c0 0x7fd82419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:50.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.463+0000 7fd829000640 1 -- 172.21.15.43:0/3694158035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8241072c0 msgr2=0x7fd82419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:50.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.463+0000 7fd829000640 1 --2- 172.21.15.43:0/3694158035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8241072c0 0x7fd82419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:50.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.463+0000 7fd829000640 1 -- 172.21.15.43:0/3694158035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd824104990 msgr2=0x7fd824197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:50.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.463+0000 7fd829000640 1 --2- 172.21.15.43:0/3694158035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd824104990 0x7fd824197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:50.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.463+0000 7fd829000640 1 -- 172.21.15.43:0/3694158035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd82419f430 con 0x7fd824102060 2024-03-26T08:31:50.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.463+0000 7fd829a00640 1 --2- 172.21.15.43:0/3694158035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8241072c0 0x7fd82419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:31:50.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.464+0000 7fd829000640 1 --2- 172.21.15.43:0/3694158035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd824102060 0x7fd824197810 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd80c00d770 tx=0x7fd80c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:50.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.464+0000 7fd821600640 1 -- 172.21.15.43:0/3694158035 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c017070 con 0x7fd824102060 2024-03-26T08:31:50.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.464+0000 7fd821600640 1 -- 172.21.15.43:0/3694158035 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd80c0054d0 con 0x7fd824102060 2024-03-26T08:31:50.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.464+0000 7fd821600640 1 -- 172.21.15.43:0/3694158035 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c004cf0 con 0x7fd824102060 2024-03-26T08:31:50.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.464+0000 7fd82a400640 1 -- 172.21.15.43:0/3694158035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd82419f720 con 0x7fd824102060 2024-03-26T08:31:50.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.464+0000 7fd82a400640 1 -- 172.21.15.43:0/3694158035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd82419fc20 con 0x7fd824102060 2024-03-26T08:31:50.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.465+0000 7fd82a400640 1 -- 172.21.15.43:0/3694158035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8240fd500 con 0x7fd824102060 2024-03-26T08:31:50.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.467+0000 7fd821600640 1 -- 172.21.15.43:0/3694158035 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd80c004e90 con 0x7fd824102060 2024-03-26T08:31:50.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.469+0000 7fd821600640 1 --2- 172.21.15.43:0/3694158035 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fd8080a2880 0x7fd8080a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:50.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.469+0000 7fd821600640 1 -- 172.21.15.43:0/3694158035 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7fd80c0ce830 con 0x7fd824102060 2024-03-26T08:31:50.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.469+0000 7fd823e00640 1 --2- 172.21.15.43:0/3694158035 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fd8080a2880 0x7fd8080a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:50.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.470+0000 7fd823e00640 1 --2- 172.21.15.43:0/3694158035 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fd8080a2880 0x7fd8080a4d40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd81402c700 tx=0x7fd814032340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:50.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.474+0000 7fd821600640 1 -- 172.21.15.43:0/3694158035 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd80c09b590 con 0x7fd824102060 2024-03-26T08:31:50.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.607+0000 7fd821600640 1 -- 172.21.15.43:0/3694158035 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fd80c097500 con 0x7fd824102060 2024-03-26T08:31:50.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.719+0000 7fd82a400640 1 -- 172.21.15.43:0/3694158035 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fd824061b70 con 0x7fd8080a2880 2024-03-26T08:31:50.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.792+0000 7fd821600640 1 -- 172.21.15.43:0/3694158035 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd824061b70 con 0x7fd8080a2880 2024-03-26T08:31:50.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.795+0000 7fd82a400640 1 -- 172.21.15.43:0/3694158035 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fd8080a2880 msgr2=0x7fd8080a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:50.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.795+0000 7fd82a400640 1 --2- 172.21.15.43:0/3694158035 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fd8080a2880 0x7fd8080a4d40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd81402c700 tx=0x7fd814032340 comp rx=0 tx=0).stop 2024-03-26T08:31:50.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.795+0000 7fd82a400640 1 -- 172.21.15.43:0/3694158035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd824102060 msgr2=0x7fd824197810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:50.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.795+0000 7fd82a400640 1 --2- 172.21.15.43:0/3694158035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd824102060 0x7fd824197810 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd80c00d770 tx=0x7fd80c00dc30 comp rx=0 tx=0).stop 2024-03-26T08:31:50.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.795+0000 7fd82a400640 1 -- 172.21.15.43:0/3694158035 shutdown_connections 2024-03-26T08:31:50.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.795+0000 7fd82a400640 1 --2- 172.21.15.43:0/3694158035 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fd8080a2880 0x7fd8080a4d40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:50.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.795+0000 7fd82a400640 1 --2- 172.21.15.43:0/3694158035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8241072c0 0x7fd82419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:50.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.795+0000 7fd82a400640 1 --2- 172.21.15.43:0/3694158035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd824104990 0x7fd824197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:50.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.795+0000 7fd82a400640 1 --2- 172.21.15.43:0/3694158035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd824102060 0x7fd824197810 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:50.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.795+0000 7fd82a400640 1 -- 172.21.15.43:0/3694158035 >> 172.21.15.43:0/3694158035 conn(0x7fd8240fba10 msgr2=0x7fd824102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:50.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.795+0000 7fd82a400640 1 -- 172.21.15.43:0/3694158035 shutdown_connections 2024-03-26T08:31:50.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:50.796+0000 7fd82a400640 1 -- 172.21.15.43:0/3694158035 wait complete. 2024-03-26T08:31:50.812 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-26T08:31:50.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:50 smithi043 ceph-mon[31929]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:31:50.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:50 smithi043 ceph-mon[31929]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:31:51.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.051+0000 7f12aca00640 1 -- 172.21.15.43:0/144919825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12a8109460 msgr2=0x7f12a810b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:51.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.051+0000 7f12aca00640 1 --2- 172.21.15.43:0/144919825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12a8109460 0x7f12a810b8f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009900 tx=0x7f12a002bc30 comp rx=0 tx=0).stop 2024-03-26T08:31:51.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.051+0000 7f12aca00640 1 -- 172.21.15.43:0/144919825 shutdown_connections 2024-03-26T08:31:51.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.051+0000 7f12aca00640 1 --2- 172.21.15.43:0/144919825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12a8109460 0x7f12a810b8f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:51.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.051+0000 7f12aca00640 1 --2- 172.21.15.43:0/144919825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12a8074100 0x7f12a8074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:51.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.051+0000 7f12aca00640 1 --2- 172.21.15.43:0/144919825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12a80756e0 0x7f12a8073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:51.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.051+0000 7f12aca00640 1 -- 172.21.15.43:0/144919825 >> 172.21.15.43:0/144919825 conn(0x7f12a80fcb10 msgr2=0x7f12a80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:51.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.051+0000 7f12aca00640 1 -- 172.21.15.43:0/144919825 shutdown_connections 2024-03-26T08:31:51.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.051+0000 7f12aca00640 1 -- 172.21.15.43:0/144919825 wait complete. 2024-03-26T08:31:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.052+0000 7f12aca00640 1 Processor -- start 2024-03-26T08:31:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.053+0000 7f12aca00640 1 -- start start 2024-03-26T08:31:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.053+0000 7f12aca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12a8074100 0x7f12a8199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.053+0000 7f12aca00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12a80756e0 0x7f12a8199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.053+0000 7f12aca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12a8109460 0x7f12a81a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.053+0000 7f12aca00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a810f300 con 0x7f12a8109460 2024-03-26T08:31:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.053+0000 7f12aca00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a810f180 con 0x7f12a80756e0 2024-03-26T08:31:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.053+0000 7f12aca00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a810f480 con 0x7f12a8074100 2024-03-26T08:31:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.053+0000 7f12a7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12a8109460 0x7f12a81a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.053+0000 7f12a7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12a8109460 0x7f12a81a0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57452/0 (socket says 172.21.15.43:57452) 2024-03-26T08:31:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.053+0000 7f12a7200640 1 -- 172.21.15.43:0/1037445745 learned_addr learned my addr 172.21.15.43:0/1037445745 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.053+0000 7f12a6800640 1 --2- 172.21.15.43:0/1037445745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12a8074100 0x7f12a8199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.053+0000 7f129de00640 1 --2- 172.21.15.43:0/1037445745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12a80756e0 0x7f12a8199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.054+0000 7f12a7200640 1 -- 172.21.15.43:0/1037445745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12a8074100 msgr2=0x7f12a8199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.054+0000 7f12a7200640 1 --2- 172.21.15.43:0/1037445745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12a8074100 0x7f12a8199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.054+0000 7f12a7200640 1 -- 172.21.15.43:0/1037445745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12a80756e0 msgr2=0x7f12a8199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.054+0000 7f12a7200640 1 --2- 172.21.15.43:0/1037445745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12a80756e0 0x7f12a8199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.054+0000 7f12a7200640 1 -- 172.21.15.43:0/1037445745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a81a1570 con 0x7f12a8109460 2024-03-26T08:31:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.054+0000 7f12a6800640 1 --2- 172.21.15.43:0/1037445745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12a8074100 0x7f12a8199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:31:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.054+0000 7f12a7200640 1 --2- 172.21.15.43:0/1037445745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12a8109460 0x7f12a81a0f00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f12a0036040 tx=0x7f12a0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.054+0000 7f129fe00640 1 -- 172.21.15.43:0/1037445745 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a0043020 con 0x7f12a8109460 2024-03-26T08:31:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.054+0000 7f12aca00640 1 -- 172.21.15.43:0/1037445745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a81a1800 con 0x7f12a8109460 2024-03-26T08:31:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.054+0000 7f129fe00640 1 -- 172.21.15.43:0/1037445745 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f12a0004490 con 0x7f12a8109460 2024-03-26T08:31:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.055+0000 7f129fe00640 1 -- 172.21.15.43:0/1037445745 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a0003d10 con 0x7f12a8109460 2024-03-26T08:31:51.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.055+0000 7f12aca00640 1 -- 172.21.15.43:0/1037445745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a81a1d50 con 0x7f12a8109460 2024-03-26T08:31:51.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:50 smithi183 ceph-mon[35818]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:31:51.050 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:50 smithi183 ceph-mon[35818]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:31:51.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.056+0000 7f12aca00640 1 -- 172.21.15.43:0/1037445745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a8108190 con 0x7f12a8109460 2024-03-26T08:31:51.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.058+0000 7f129fe00640 1 -- 172.21.15.43:0/1037445745 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f12a0046020 con 0x7f12a8109460 2024-03-26T08:31:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.059+0000 7f129fe00640 1 --2- 172.21.15.43:0/1037445745 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f128c0a2890 0x7f128c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.059+0000 7f129fe00640 1 -- 172.21.15.43:0/1037445745 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f12a00f16b0 con 0x7f12a8109460 2024-03-26T08:31:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.059+0000 7f12a6800640 1 --2- 172.21.15.43:0/1037445745 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f128c0a2890 0x7f128c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.060+0000 7f12a6800640 1 --2- 172.21.15.43:0/1037445745 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f128c0a2890 0x7f128c0a4d50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1298005fd0 tx=0x7f1298009520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.062+0000 7f129fe00640 1 -- 172.21.15.43:0/1037445745 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f12a004d020 con 0x7f12a8109460 2024-03-26T08:31:51.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:50 smithi052 ceph-mon[35071]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:31:51.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:50 smithi052 ceph-mon[35071]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:31:51.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.298+0000 7f12aca00640 1 -- 172.21.15.43:0/1037445745 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f12a8061ae0 con 0x7f128c0a2890 2024-03-26T08:31:51.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.315+0000 7f129fe00640 1 -- 172.21.15.43:0/1037445745 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f12a8061ae0 con 0x7f128c0a2890 2024-03-26T08:31:51.308 INFO:teuthology.orchestra.run.smithi043.stdout:/volumes/qa/sv_1/45367009-de98-4c54-9aef-3ff3a7e7e9d6 2024-03-26T08:31:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.318+0000 7f12aca00640 1 -- 172.21.15.43:0/1037445745 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f128c0a2890 msgr2=0x7f128c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.318+0000 7f12aca00640 1 --2- 172.21.15.43:0/1037445745 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f128c0a2890 0x7f128c0a4d50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1298005fd0 tx=0x7f1298009520 comp rx=0 tx=0).stop 2024-03-26T08:31:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.318+0000 7f12aca00640 1 -- 172.21.15.43:0/1037445745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12a8109460 msgr2=0x7f12a81a0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.318+0000 7f12aca00640 1 --2- 172.21.15.43:0/1037445745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12a8109460 0x7f12a81a0f00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f12a0036040 tx=0x7f12a0004270 comp rx=0 tx=0).stop 2024-03-26T08:31:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.318+0000 7f12aca00640 1 -- 172.21.15.43:0/1037445745 shutdown_connections 2024-03-26T08:31:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.318+0000 7f12aca00640 1 --2- 172.21.15.43:0/1037445745 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f128c0a2890 0x7f128c0a4d50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.318+0000 7f12aca00640 1 --2- 172.21.15.43:0/1037445745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12a8109460 0x7f12a81a0f00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.318+0000 7f12aca00640 1 --2- 172.21.15.43:0/1037445745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12a80756e0 0x7f12a8199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.318+0000 7f12aca00640 1 --2- 172.21.15.43:0/1037445745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12a8074100 0x7f12a8199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.318+0000 7f12aca00640 1 -- 172.21.15.43:0/1037445745 >> 172.21.15.43:0/1037445745 conn(0x7f12a80fcb10 msgr2=0x7f12a8105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.319+0000 7f12aca00640 1 -- 172.21.15.43:0/1037445745 shutdown_connections 2024-03-26T08:31:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.319+0000 7f12aca00640 1 -- 172.21.15.43:0/1037445745 wait complete. 2024-03-26T08:31:51.332 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:31:51.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.562+0000 7f3ee7200640 1 -- 172.21.15.43:0/4005167498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee01048e0 msgr2=0x7f3ee0104d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:51.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.562+0000 7f3ee7200640 1 --2- 172.21.15.43:0/4005167498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee01048e0 0x7f3ee0104d60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0009900 tx=0x7f3ed002bc30 comp rx=0 tx=0).stop 2024-03-26T08:31:51.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.562+0000 7f3ee7200640 1 -- 172.21.15.43:0/4005167498 shutdown_connections 2024-03-26T08:31:51.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.562+0000 7f3ee7200640 1 --2- 172.21.15.43:0/4005167498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ee01052a0 0x7f3ee010bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:51.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.562+0000 7f3ee7200640 1 --2- 172.21.15.43:0/4005167498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee01048e0 0x7f3ee0104d60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:51.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.562+0000 7f3ee7200640 1 --2- 172.21.15.43:0/4005167498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ee01036f0 0x7f3ee0103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:51.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.562+0000 7f3ee7200640 1 -- 172.21.15.43:0/4005167498 >> 172.21.15.43:0/4005167498 conn(0x7f3ee00fec80 msgr2=0x7f3ee01010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:51.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.563+0000 7f3ee7200640 1 -- 172.21.15.43:0/4005167498 shutdown_connections 2024-03-26T08:31:51.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.563+0000 7f3ee7200640 1 -- 172.21.15.43:0/4005167498 wait complete. 2024-03-26T08:31:51.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.564+0000 7f3ee7200640 1 Processor -- start 2024-03-26T08:31:51.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.564+0000 7f3ee7200640 1 -- start start 2024-03-26T08:31:51.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.564+0000 7f3ee7200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee01036f0 0x7f3ee01998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:51.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.565+0000 7f3ee7200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ee01048e0 0x7f3ee0199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:51.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.565+0000 7f3ee7200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ee01052a0 0x7f3ee01a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:51.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.565+0000 7f3ee7200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee010f270 con 0x7f3ee01036f0 2024-03-26T08:31:51.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.565+0000 7f3ee7200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee010f0f0 con 0x7f3ee01048e0 2024-03-26T08:31:51.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.565+0000 7f3ee7200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee010f3f0 con 0x7f3ee01052a0 2024-03-26T08:31:51.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.565+0000 7f3ee5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee01036f0 0x7f3ee01998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:51.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.565+0000 7f3ee5e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee01036f0 0x7f3ee01998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57474/0 (socket says 172.21.15.43:57474) 2024-03-26T08:31:51.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.565+0000 7f3ee5e00640 1 -- 172.21.15.43:0/987212315 learned_addr learned my addr 172.21.15.43:0/987212315 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:51.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.565+0000 7f3ee5400640 1 --2- 172.21.15.43:0/987212315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ee01048e0 0x7f3ee0199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:51.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.565+0000 7f3ee5e00640 1 -- 172.21.15.43:0/987212315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ee01052a0 msgr2=0x7f3ee01a0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:51.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.565+0000 7f3ee5e00640 1 --2- 172.21.15.43:0/987212315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ee01052a0 0x7f3ee01a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:51.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.565+0000 7f3ee5e00640 1 -- 172.21.15.43:0/987212315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ee01048e0 msgr2=0x7f3ee0199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:51.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.565+0000 7f3ee5e00640 1 --2- 172.21.15.43:0/987212315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ee01048e0 0x7f3ee0199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:51.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.565+0000 7f3ee5e00640 1 -- 172.21.15.43:0/987212315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee01a14e0 con 0x7f3ee01036f0 2024-03-26T08:31:51.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.566+0000 7f3ee5e00640 1 --2- 172.21.15.43:0/987212315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee01036f0 0x7f3ee01998c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3ed400d770 tx=0x7f3ed400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:51.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.566+0000 7f3edaa00640 1 -- 172.21.15.43:0/987212315 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4017070 con 0x7f3ee01036f0 2024-03-26T08:31:51.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.566+0000 7f3edaa00640 1 -- 172.21.15.43:0/987212315 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3ed4009c30 con 0x7f3ee01036f0 2024-03-26T08:31:51.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.566+0000 7f3ee7200640 1 -- 172.21.15.43:0/987212315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee01a17d0 con 0x7f3ee01036f0 2024-03-26T08:31:51.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.566+0000 7f3edaa00640 1 -- 172.21.15.43:0/987212315 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4003d00 con 0x7f3ee01036f0 2024-03-26T08:31:51.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.566+0000 7f3ee7200640 1 -- 172.21.15.43:0/987212315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee01a1d40 con 0x7f3ee01036f0 2024-03-26T08:31:51.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.567+0000 7f3ee7200640 1 -- 172.21.15.43:0/987212315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee010aa10 con 0x7f3ee01036f0 2024-03-26T08:31:51.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.569+0000 7f3edaa00640 1 -- 172.21.15.43:0/987212315 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f3ed4003ea0 con 0x7f3ee01036f0 2024-03-26T08:31:51.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.570+0000 7f3edaa00640 1 --2- 172.21.15.43:0/987212315 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3ec80a2910 0x7f3ec80a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:51.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.571+0000 7f3edaa00640 1 -- 172.21.15.43:0/987212315 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f3ed40cecb0 con 0x7f3ee01036f0 2024-03-26T08:31:51.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.571+0000 7f3ee5400640 1 --2- 172.21.15.43:0/987212315 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3ec80a2910 0x7f3ec80a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:51.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.572+0000 7f3ee5400640 1 --2- 172.21.15.43:0/987212315 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3ec80a2910 0x7f3ec80a4dd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0006010 tx=0x7f3ed0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:51.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.575+0000 7f3edaa00640 1 -- 172.21.15.43:0/987212315 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3ed409b8b0 con 0x7f3ee01036f0 2024-03-26T08:31:51.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.825+0000 7f3ee7200640 1 -- 172.21.15.43:0/987212315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ee002d2d0 con 0x7f3ee01036f0 2024-03-26T08:31:51.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.827+0000 7f3edaa00640 1 -- 172.21.15.43:0/987212315 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11331 (secure 0 0 0) 0x7f3ed4009de0 con 0x7f3ee01036f0 2024-03-26T08:31:51.820 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:31:51.820 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1452206807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1452206807},{"type":"v1","addr":"172.21.15.43:6835","nonce":1452206807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3151775467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3151775467},{"type":"v1","addr":"172.21.15.43:6837","nonce":3151775467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/1284655682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":1284655682},{"type":"v1","addr":"172.21.15.43:6839","nonce":1284655682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3081085787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3081085787},{"type":"v1","addr":"172.21.15.43:6841","nonce":3081085787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1842295306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1842295306},{"type":"v1","addr":"172.21.15.183:6835","nonce":1842295306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2267781666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2267781666},{"type":"v1","addr":"172.21.15.183:6837","nonce":2267781666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3572429948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3572429948},{"type":"v1","addr":"172.21.15.183:6839","nonce":3572429948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34448,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/1108327179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108327179},{"type":"v1","addr":"172.21.15.52:6833","nonce":1108327179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34487,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/742057410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":742057410},{"type":"v1","addr":"172.21.15.52:6835","nonce":742057410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:31:21.819689+0000","modified":"2024-03-26T08:31:48.080233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34517":{"gid":34517,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/748239918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":748239918},{"type":"v1","addr":"172.21.15.52:6837","nonce":748239918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34559":{"gid":34559,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.52:6839/2551965101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2551965101},{"type":"v1","addr":"172.21.15.52:6839","nonce":2551965101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34559,"qdb_cluster":[34559]},"id":1}]} 2024-03-26T08:31:51.820 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 21 2024-03-26T08:31:51.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.829+0000 7f3ee7200640 1 -- 172.21.15.43:0/987212315 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3ec80a2910 msgr2=0x7f3ec80a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:51.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.829+0000 7f3ee7200640 1 --2- 172.21.15.43:0/987212315 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3ec80a2910 0x7f3ec80a4dd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0006010 tx=0x7f3ed0000bc0 comp rx=0 tx=0).stop 2024-03-26T08:31:51.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.829+0000 7f3ee7200640 1 -- 172.21.15.43:0/987212315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee01036f0 msgr2=0x7f3ee01998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:51.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.829+0000 7f3ee7200640 1 --2- 172.21.15.43:0/987212315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee01036f0 0x7f3ee01998c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3ed400d770 tx=0x7f3ed400dc30 comp rx=0 tx=0).stop 2024-03-26T08:31:51.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.830+0000 7f3ee7200640 1 -- 172.21.15.43:0/987212315 shutdown_connections 2024-03-26T08:31:51.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.830+0000 7f3ee7200640 1 --2- 172.21.15.43:0/987212315 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3ec80a2910 0x7f3ec80a4dd0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:51.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.830+0000 7f3ee7200640 1 --2- 172.21.15.43:0/987212315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ee01052a0 0x7f3ee01a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:51.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.830+0000 7f3ee7200640 1 --2- 172.21.15.43:0/987212315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ee01048e0 0x7f3ee0199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:51.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.830+0000 7f3ee7200640 1 --2- 172.21.15.43:0/987212315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ee01036f0 0x7f3ee01998c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:51.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.830+0000 7f3ee7200640 1 -- 172.21.15.43:0/987212315 >> 172.21.15.43:0/987212315 conn(0x7f3ee00fec80 msgr2=0x7f3ee0107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:51.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.830+0000 7f3ee7200640 1 -- 172.21.15.43:0/987212315 shutdown_connections 2024-03-26T08:31:51.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:51.830+0000 7f3ee7200640 1 -- 172.21.15.43:0/987212315 wait complete. 2024-03-26T08:31:51.844 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:31:51.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:51 smithi043 ceph-mon[31929]: pgmap v389: 129 pgs: 129 active+clean; 600 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.3 KiB/s wr, 2 op/s; 1 B/s, 0 objects/s recovering 2024-03-26T08:31:51.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:51 smithi043 ceph-mon[31929]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-03-26T08:31:51.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:51 smithi043 ceph-mon[31929]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:31:52.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:51 smithi183 ceph-mon[35818]: pgmap v389: 129 pgs: 129 active+clean; 600 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.3 KiB/s wr, 2 op/s; 1 B/s, 0 objects/s recovering 2024-03-26T08:31:52.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:51 smithi183 ceph-mon[35818]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-03-26T08:31:52.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:51 smithi183 ceph-mon[35818]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:31:52.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.078+0000 7f9d92600640 1 -- 172.21.15.43:0/3610812993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d8c1052a0 msgr2=0x7f9d8c10bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:52.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.078+0000 7f9d92600640 1 --2- 172.21.15.43:0/3610812993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d8c1052a0 0x7f9d8c10bb30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9d84009ca0 tx=0x7f9d8402c0f0 comp rx=0 tx=0).stop 2024-03-26T08:31:52.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.078+0000 7f9d92600640 1 -- 172.21.15.43:0/3610812993 shutdown_connections 2024-03-26T08:31:52.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.078+0000 7f9d92600640 1 --2- 172.21.15.43:0/3610812993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d8c1052a0 0x7f9d8c10bb30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:52.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.078+0000 7f9d92600640 1 --2- 172.21.15.43:0/3610812993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d8c1048e0 0x7f9d8c104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:52.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.078+0000 7f9d92600640 1 --2- 172.21.15.43:0/3610812993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d8c1036f0 0x7f9d8c103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:52.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.078+0000 7f9d92600640 1 -- 172.21.15.43:0/3610812993 >> 172.21.15.43:0/3610812993 conn(0x7f9d8c0fec80 msgr2=0x7f9d8c1010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:52.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.079+0000 7f9d92600640 1 -- 172.21.15.43:0/3610812993 shutdown_connections 2024-03-26T08:31:52.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.079+0000 7f9d92600640 1 -- 172.21.15.43:0/3610812993 wait complete. 2024-03-26T08:31:52.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:51 smithi052 ceph-mon[35071]: pgmap v389: 129 pgs: 129 active+clean; 600 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.3 KiB/s wr, 2 op/s; 1 B/s, 0 objects/s recovering 2024-03-26T08:31:52.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:51 smithi052 ceph-mon[35071]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-03-26T08:31:52.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:51 smithi052 ceph-mon[35071]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:31:52.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.080+0000 7f9d92600640 1 Processor -- start 2024-03-26T08:31:52.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.080+0000 7f9d92600640 1 -- start start 2024-03-26T08:31:52.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.080+0000 7f9d92600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d8c1036f0 0x7f9d8c1998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:52.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.080+0000 7f9d92600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d8c1048e0 0x7f9d8c199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:52.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.080+0000 7f9d92600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d8c1052a0 0x7f9d8c1a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:52.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.080+0000 7f9d92600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c10f220 con 0x7f9d8c1052a0 2024-03-26T08:31:52.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.080+0000 7f9d92600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c10f0a0 con 0x7f9d8c1036f0 2024-03-26T08:31:52.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.080+0000 7f9d92600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c10f3a0 con 0x7f9d8c1048e0 2024-03-26T08:31:52.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.081+0000 7f9d91c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d8c1052a0 0x7f9d8c1a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:52.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.081+0000 7f9d91c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d8c1052a0 0x7f9d8c1a0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57490/0 (socket says 172.21.15.43:57490) 2024-03-26T08:31:52.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.081+0000 7f9d91c00640 1 -- 172.21.15.43:0/1920952881 learned_addr learned my addr 172.21.15.43:0/1920952881 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:52.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.081+0000 7f9d91200640 1 --2- 172.21.15.43:0/1920952881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d8c1036f0 0x7f9d8c1998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:52.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.081+0000 7f9d8be00640 1 --2- 172.21.15.43:0/1920952881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d8c1048e0 0x7f9d8c199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:52.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.081+0000 7f9d91c00640 1 -- 172.21.15.43:0/1920952881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d8c1048e0 msgr2=0x7f9d8c199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:52.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.081+0000 7f9d91c00640 1 --2- 172.21.15.43:0/1920952881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d8c1048e0 0x7f9d8c199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:52.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.081+0000 7f9d91c00640 1 -- 172.21.15.43:0/1920952881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d8c1036f0 msgr2=0x7f9d8c1998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:52.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.081+0000 7f9d91c00640 1 --2- 172.21.15.43:0/1920952881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d8c1036f0 0x7f9d8c1998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:52.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.081+0000 7f9d91c00640 1 -- 172.21.15.43:0/1920952881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8c1a14e0 con 0x7f9d8c1052a0 2024-03-26T08:31:52.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.082+0000 7f9d91c00640 1 --2- 172.21.15.43:0/1920952881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d8c1052a0 0x7f9d8c1a0e70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9d84032380 tx=0x7f9d84032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:52.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.082+0000 7f9d89600640 1 -- 172.21.15.43:0/1920952881 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d84041020 con 0x7f9d8c1052a0 2024-03-26T08:31:52.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.082+0000 7f9d89600640 1 -- 172.21.15.43:0/1920952881 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9d84003e20 con 0x7f9d8c1052a0 2024-03-26T08:31:52.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.082+0000 7f9d92600640 1 -- 172.21.15.43:0/1920952881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d8c1a1770 con 0x7f9d8c1052a0 2024-03-26T08:31:52.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.082+0000 7f9d89600640 1 -- 172.21.15.43:0/1920952881 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d84039040 con 0x7f9d8c1052a0 2024-03-26T08:31:52.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.082+0000 7f9d92600640 1 -- 172.21.15.43:0/1920952881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d8c1a1cb0 con 0x7f9d8c1052a0 2024-03-26T08:31:52.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.083+0000 7f9d92600640 1 -- 172.21.15.43:0/1920952881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d8c10aa10 con 0x7f9d8c1052a0 2024-03-26T08:31:52.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.086+0000 7f9d89600640 1 -- 172.21.15.43:0/1920952881 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f9d84044020 con 0x7f9d8c1052a0 2024-03-26T08:31:52.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.088+0000 7f9d89600640 1 --2- 172.21.15.43:0/1920952881 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f9d740a2910 0x7f9d740a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:52.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.088+0000 7f9d89600640 1 -- 172.21.15.43:0/1920952881 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f9d840f1060 con 0x7f9d8c1052a0 2024-03-26T08:31:52.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.088+0000 7f9d91200640 1 --2- 172.21.15.43:0/1920952881 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f9d740a2910 0x7f9d740a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:52.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.089+0000 7f9d91200640 1 --2- 172.21.15.43:0/1920952881 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f9d740a2910 0x7f9d740a4dd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9d80005fd0 tx=0x7f9d8000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:52.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.092+0000 7f9d89600640 1 -- 172.21.15.43:0/1920952881 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9d8404b020 con 0x7f9d8c1052a0 2024-03-26T08:31:52.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.299+0000 7f9d92600640 1 -- 172.21.15.43:0/1920952881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d8c0622a0 con 0x7f9d8c1052a0 2024-03-26T08:31:52.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.303+0000 7f9d89600640 1 -- 172.21.15.43:0/1920952881 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24576 (secure 0 0 0) 0x7f9d840bdd10 con 0x7f9d8c1052a0 2024-03-26T08:31:52.297 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:31:52.297 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":92,"fsid":"88759420-eb49-11ee-95d0-87774f69a715","created":"2024-03-26T08:20:27.817107+0000","modified":"2024-03-26T08:31:37.455519+0000","last_up_change":"2024-03-26T08:30:10.520976+0000","last_in_change":"2024-03-26T08:29:46.394360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:24:54.852427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:31:19.924902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:31:21.323881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T08:31:24.951015+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"28670673-9193-4fa3-9cc0-c1a5a6b27284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6803","nonce":936601146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6805","nonce":936601146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6809","nonce":936601146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6807","nonce":936601146}]},"public_addr":"172.21.15.43:6803/936601146","cluster_addr":"172.21.15.43:6805/936601146","heartbeat_back_addr":"172.21.15.43:6809/936601146","heartbeat_front_addr":"172.21.15.43:6807/936601146","state":["exists","up"]},{"osd":1,"uuid":"fa8ad6b6-277a-42b8-b0f8-b36fc219de2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6801","nonce":3268412328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6803","nonce":3268412328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6807","nonce":3268412328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6805","nonce":3268412328}]},"public_addr":"172.21.15.52:6801/3268412328","cluster_addr":"172.21.15.52:6803/3268412328","heartbeat_back_addr":"172.21.15.52:6807/3268412328","heartbeat_front_addr":"172.21.15.52:6805/3268412328","state":["exists","up"]},{"osd":2,"uuid":"ec0039c3-f9bd-419a-81de-fa4418359808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6801","nonce":2103180908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6803","nonce":2103180908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6807","nonce":2103180908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6805","nonce":2103180908}]},"public_addr":"172.21.15.183:6801/2103180908","cluster_addr":"172.21.15.183:6803/2103180908","heartbeat_back_addr":"172.21.15.183:6807/2103180908","heartbeat_front_addr":"172.21.15.183:6805/2103180908","state":["exists","up"]},{"osd":3,"uuid":"e7dcbf00-6586-4664-a69e-3cb0df8dfd30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6811","nonce":3722846033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6813","nonce":3722846033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6817","nonce":3722846033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6815","nonce":3722846033}]},"public_addr":"172.21.15.43:6811/3722846033","cluster_addr":"172.21.15.43:6813/3722846033","heartbeat_back_addr":"172.21.15.43:6817/3722846033","heartbeat_front_addr":"172.21.15.43:6815/3722846033","state":["exists","up"]},{"osd":4,"uuid":"db089a49-5e98-432d-9687-cdc044114c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6809","nonce":4028436523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6811","nonce":4028436523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6815","nonce":4028436523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6813","nonce":4028436523}]},"public_addr":"172.21.15.52:6809/4028436523","cluster_addr":"172.21.15.52:6811/4028436523","heartbeat_back_addr":"172.21.15.52:6815/4028436523","heartbeat_front_addr":"172.21.15.52:6813/4028436523","state":["exists","up"]},{"osd":5,"uuid":"556ac804-02ea-423b-a599-55baf5be4ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6809","nonce":787132252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6811","nonce":787132252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6815","nonce":787132252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6813","nonce":787132252}]},"public_addr":"172.21.15.183:6809/787132252","cluster_addr":"172.21.15.183:6811/787132252","heartbeat_back_addr":"172.21.15.183:6815/787132252","heartbeat_front_addr":"172.21.15.183:6813/787132252","state":["exists","up"]},{"osd":6,"uuid":"3de2a49d-6cef-4580-9d8a-f90cdfcecd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6819","nonce":4206861158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6821","nonce":4206861158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6825","nonce":4206861158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6823","nonce":4206861158}]},"public_addr":"172.21.15.43:6819/4206861158","cluster_addr":"172.21.15.43:6821/4206861158","heartbeat_back_addr":"172.21.15.43:6825/4206861158","heartbeat_front_addr":"172.21.15.43:6823/4206861158","state":["exists","up"]},{"osd":7,"uuid":"a3b832f7-7d7d-4a62-b679-db6da98275d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6817","nonce":3452064146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6819","nonce":3452064146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6823","nonce":3452064146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6821","nonce":3452064146}]},"public_addr":"172.21.15.52:6817/3452064146","cluster_addr":"172.21.15.52:6819/3452064146","heartbeat_back_addr":"172.21.15.52:6823/3452064146","heartbeat_front_addr":"172.21.15.52:6821/3452064146","state":["exists","up"]},{"osd":8,"uuid":"ddc6b07f-d703-414d-a59a-02316376b13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6817","nonce":1023131725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6819","nonce":1023131725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6823","nonce":1023131725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6821","nonce":1023131725}]},"public_addr":"172.21.15.183:6817/1023131725","cluster_addr":"172.21.15.183:6819/1023131725","heartbeat_back_addr":"172.21.15.183:6823/1023131725","heartbeat_front_addr":"172.21.15.183:6821/1023131725","state":["exists","up"]},{"osd":9,"uuid":"ac16bb0e-6232-45c8-b16a-4d878874c588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6827","nonce":3935195657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6829","nonce":3935195657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6833","nonce":3935195657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6831","nonce":3935195657}]},"public_addr":"172.21.15.43:6827/3935195657","cluster_addr":"172.21.15.43:6829/3935195657","heartbeat_back_addr":"172.21.15.43:6833/3935195657","heartbeat_front_addr":"172.21.15.43:6831/3935195657","state":["exists","up"]},{"osd":10,"uuid":"fd839188-f398-4cff-aa7b-36522ff1a8a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6825","nonce":377628320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6827","nonce":377628320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6831","nonce":377628320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6829","nonce":377628320}]},"public_addr":"172.21.15.52:6825/377628320","cluster_addr":"172.21.15.52:6827/377628320","heartbeat_back_addr":"172.21.15.52:6831/377628320","heartbeat_front_addr":"172.21.15.52:6829/377628320","state":["exists","up"]},{"osd":11,"uuid":"a40282a0-0c64-49ee-b64e-6f3f8119e827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6825","nonce":1373086433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6827","nonce":1373086433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6831","nonce":1373086433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6829","nonce":1373086433}]},"public_addr":"172.21.15.183:6825/1373086433","cluster_addr":"172.21.15.183:6827/1373086433","heartbeat_back_addr":"172.21.15.183:6831/1373086433","heartbeat_front_addr":"172.21.15.183:6829/1373086433","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:23:43.478905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:17.573532+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:50.286610+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:25:27.407663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:03.081294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:36.057416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:12.062013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:47.032271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:21.023201+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:57.364739+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:29:34.039995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:30:08.827577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6801/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/4240775295":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/510203140":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/4251957687":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:6801/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1885244665":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/298320867":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1301092625":"2024-03-27T08:21:36.149092+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:31:52.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.306+0000 7f9d92600640 1 -- 172.21.15.43:0/1920952881 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f9d740a2910 msgr2=0x7f9d740a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:52.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.306+0000 7f9d92600640 1 --2- 172.21.15.43:0/1920952881 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f9d740a2910 0x7f9d740a4dd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9d80005fd0 tx=0x7f9d8000b390 comp rx=0 tx=0).stop 2024-03-26T08:31:52.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.306+0000 7f9d92600640 1 -- 172.21.15.43:0/1920952881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d8c1052a0 msgr2=0x7f9d8c1a0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:52.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.306+0000 7f9d92600640 1 --2- 172.21.15.43:0/1920952881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d8c1052a0 0x7f9d8c1a0e70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9d84032380 tx=0x7f9d84032820 comp rx=0 tx=0).stop 2024-03-26T08:31:52.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.306+0000 7f9d92600640 1 -- 172.21.15.43:0/1920952881 shutdown_connections 2024-03-26T08:31:52.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.306+0000 7f9d92600640 1 --2- 172.21.15.43:0/1920952881 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f9d740a2910 0x7f9d740a4dd0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:52.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.307+0000 7f9d92600640 1 --2- 172.21.15.43:0/1920952881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d8c1052a0 0x7f9d8c1a0e70 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:52.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.307+0000 7f9d92600640 1 --2- 172.21.15.43:0/1920952881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d8c1048e0 0x7f9d8c199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:52.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.307+0000 7f9d92600640 1 --2- 172.21.15.43:0/1920952881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d8c1036f0 0x7f9d8c1998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:52.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.307+0000 7f9d92600640 1 -- 172.21.15.43:0/1920952881 >> 172.21.15.43:0/1920952881 conn(0x7f9d8c0fec80 msgr2=0x7f9d8c107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:52.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.307+0000 7f9d92600640 1 -- 172.21.15.43:0/1920952881 shutdown_connections 2024-03-26T08:31:52.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.307+0000 7f9d92600640 1 -- 172.21.15.43:0/1920952881 wait complete. 2024-03-26T08:31:52.319 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-26T08:31:52.320 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T08:31:52.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.553+0000 7fb04d400640 1 -- 172.21.15.43:0/2925019222 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03802cbe0 con 0x7fb0501026f0 2024-03-26T08:31:52.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.554+0000 7fb054a00640 1 -- 172.21.15.43:0/2925019222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0501026f0 msgr2=0x7fb050102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:52.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.554+0000 7fb054a00640 1 --2- 172.21.15.43:0/2925019222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0501026f0 0x7fb050102b70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb038009900 tx=0x7fb03802bc30 comp rx=0 tx=0).stop 2024-03-26T08:31:52.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.554+0000 7fb054a00640 1 -- 172.21.15.43:0/2925019222 shutdown_connections 2024-03-26T08:31:52.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.554+0000 7fb054a00640 1 --2- 172.21.15.43:0/2925019222 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0501030b0 0x7fb050109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:52.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.554+0000 7fb054a00640 1 --2- 172.21.15.43:0/2925019222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0501026f0 0x7fb050102b70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:52.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.554+0000 7fb054a00640 1 --2- 172.21.15.43:0/2925019222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb050101500 0x7fb050101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:52.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.554+0000 7fb054a00640 1 -- 172.21.15.43:0/2925019222 >> 172.21.15.43:0/2925019222 conn(0x7fb0500fca70 msgr2=0x7fb0500feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:52.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.554+0000 7fb054a00640 1 -- 172.21.15.43:0/2925019222 shutdown_connections 2024-03-26T08:31:52.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.554+0000 7fb054a00640 1 -- 172.21.15.43:0/2925019222 wait complete. 2024-03-26T08:31:52.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.555+0000 7fb054a00640 1 Processor -- start 2024-03-26T08:31:52.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.555+0000 7fb054a00640 1 -- start start 2024-03-26T08:31:52.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.556+0000 7fb054a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb050101500 0x7fb050075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:52.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.556+0000 7fb054a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0501026f0 0x7fb050073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:52.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.556+0000 7fb054a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0501030b0 0x7fb050073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:52.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.556+0000 7fb054a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05010d040 con 0x7fb0501026f0 2024-03-26T08:31:52.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.556+0000 7fb054a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05010cec0 con 0x7fb0501030b0 2024-03-26T08:31:52.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.556+0000 7fb054a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05010d1c0 con 0x7fb050101500 2024-03-26T08:31:52.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.556+0000 7fb04de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0501026f0 0x7fb050073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:52.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.556+0000 7fb04de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0501026f0 0x7fb050073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57502/0 (socket says 172.21.15.43:57502) 2024-03-26T08:31:52.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.556+0000 7fb04de00640 1 -- 172.21.15.43:0/3481356286 learned_addr learned my addr 172.21.15.43:0/3481356286 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:52.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.556+0000 7fb04e800640 1 --2- 172.21.15.43:0/3481356286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb050101500 0x7fb050075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:52.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.556+0000 7fb04f200640 1 --2- 172.21.15.43:0/3481356286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0501030b0 0x7fb050073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:52.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.557+0000 7fb04de00640 1 -- 172.21.15.43:0/3481356286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb050101500 msgr2=0x7fb050075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:52.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.557+0000 7fb04de00640 1 --2- 172.21.15.43:0/3481356286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb050101500 0x7fb050075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:52.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.557+0000 7fb04de00640 1 -- 172.21.15.43:0/3481356286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0501030b0 msgr2=0x7fb050073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:52.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.557+0000 7fb04de00640 1 --2- 172.21.15.43:0/3481356286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0501030b0 0x7fb050073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:52.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.557+0000 7fb04de00640 1 -- 172.21.15.43:0/3481356286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb050074860 con 0x7fb0501026f0 2024-03-26T08:31:52.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.557+0000 7fb04f200640 1 --2- 172.21.15.43:0/3481356286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0501030b0 0x7fb050073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:31:52.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.557+0000 7fb04de00640 1 --2- 172.21.15.43:0/3481356286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0501026f0 0x7fb050073a90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb038006010 tx=0x7fb0380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:52.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.557+0000 7fb047400640 1 -- 172.21.15.43:0/3481356286 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb038043020 con 0x7fb0501026f0 2024-03-26T08:31:52.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.557+0000 7fb047400640 1 -- 172.21.15.43:0/3481356286 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb038004aa0 con 0x7fb0501026f0 2024-03-26T08:31:52.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.557+0000 7fb054a00640 1 -- 172.21.15.43:0/3481356286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0501a17c0 con 0x7fb0501026f0 2024-03-26T08:31:52.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.557+0000 7fb047400640 1 -- 172.21.15.43:0/3481356286 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb038003c60 con 0x7fb0501026f0 2024-03-26T08:31:52.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.558+0000 7fb054a00640 1 -- 172.21.15.43:0/3481356286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0501a1d60 con 0x7fb0501026f0 2024-03-26T08:31:52.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.559+0000 7fb054a00640 1 -- 172.21.15.43:0/3481356286 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb050108820 con 0x7fb0501026f0 2024-03-26T08:31:52.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.561+0000 7fb047400640 1 -- 172.21.15.43:0/3481356286 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fb038046020 con 0x7fb0501026f0 2024-03-26T08:31:52.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.562+0000 7fb047400640 1 --2- 172.21.15.43:0/3481356286 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb0340a2910 0x7fb0340a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:52.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.562+0000 7fb047400640 1 -- 172.21.15.43:0/3481356286 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7fb0380f12c0 con 0x7fb0501026f0 2024-03-26T08:31:52.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.562+0000 7fb04e800640 1 --2- 172.21.15.43:0/3481356286 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb0340a2910 0x7fb0340a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:52.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.563+0000 7fb04e800640 1 --2- 172.21.15.43:0/3481356286 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb0340a2910 0x7fb0340a4dd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb0400058c0 tx=0x7fb040005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:52.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.566+0000 7fb047400640 1 -- 172.21.15.43:0/3481356286 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb03804d020 con 0x7fb0501026f0 2024-03-26T08:31:52.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.769+0000 7fb054a00640 1 -- 172.21.15.43:0/3481356286 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb050061ae0 con 0x7fb0340a2910 2024-03-26T08:31:52.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.805+0000 7fb047400640 1 -- 172.21.15.43:0/3481356286 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340575 (secure 0 0 0) 0x7fb050061ae0 con 0x7fb0340a2910 2024-03-26T08:31:52.803 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:31:52.805 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-03-26T08:31:52.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.811+0000 7fb054a00640 1 -- 172.21.15.43:0/3481356286 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb0340a2910 msgr2=0x7fb0340a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:52.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.811+0000 7fb054a00640 1 --2- 172.21.15.43:0/3481356286 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb0340a2910 0x7fb0340a4dd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb0400058c0 tx=0x7fb040005850 comp rx=0 tx=0).stop 2024-03-26T08:31:52.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.811+0000 7fb054a00640 1 -- 172.21.15.43:0/3481356286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0501026f0 msgr2=0x7fb050073a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:52.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.811+0000 7fb054a00640 1 --2- 172.21.15.43:0/3481356286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0501026f0 0x7fb050073a90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb038006010 tx=0x7fb0380042f0 comp rx=0 tx=0).stop 2024-03-26T08:31:52.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.812+0000 7fb054a00640 1 -- 172.21.15.43:0/3481356286 shutdown_connections 2024-03-26T08:31:52.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.812+0000 7fb054a00640 1 --2- 172.21.15.43:0/3481356286 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb0340a2910 0x7fb0340a4dd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:52.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.812+0000 7fb054a00640 1 --2- 172.21.15.43:0/3481356286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0501030b0 0x7fb050073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:52.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.812+0000 7fb054a00640 1 --2- 172.21.15.43:0/3481356286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0501026f0 0x7fb050073a90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:52.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.812+0000 7fb054a00640 1 --2- 172.21.15.43:0/3481356286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb050101500 0x7fb050075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:52.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.812+0000 7fb054a00640 1 -- 172.21.15.43:0/3481356286 >> 172.21.15.43:0/3481356286 conn(0x7fb0500fca70 msgr2=0x7fb0501059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:52.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.812+0000 7fb054a00640 1 -- 172.21.15.43:0/3481356286 shutdown_connections 2024-03-26T08:31:52.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:52.812+0000 7fb054a00640 1 -- 172.21.15.43:0/3481356286 wait complete. 2024-03-26T08:31:52.826 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":390,"stamp":"2024-03-26T08:31:52.339305+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":614307,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":141,"num_read_kb":130,"num_write":239,"num_write_kb":2980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":188,"ondisk_log_size":188,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369472,"kb_used_data":11932,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572352,"statfs":{"total":1151940427776,"available":1151562088448,"internally_reserved":0,"allocated":12218368,"data_stored":7802111,"data_compressed":355755,"data_compressed_allocated":479232,"data_compressed_original":2457600,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19440,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":21,"num_write":10,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005196"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.489348+0000","last_change":"2024-03-26T08:31:22.394256+0000","last_active":"2024-03-26T08:31:37.489348+0000","last_peered":"2024-03-26T08:31:37.489348+0000","last_clean":"2024-03-26T08:31:37.489348+0000","last_became_active":"2024-03-26T08:31:22.393947+0000","last_became_peered":"2024-03-26T08:31:22.393947+0000","last_unstale":"2024-03-26T08:31:37.489348+0000","last_undegraded":"2024-03-26T08:31:37.489348+0000","last_fullsized":"2024-03-26T08:31:37.489348+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:00:50.769853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.484219+0000","last_change":"2024-03-26T08:31:22.387549+0000","last_active":"2024-03-26T08:31:37.484219+0000","last_peered":"2024-03-26T08:31:37.484219+0000","last_clean":"2024-03-26T08:31:37.484219+0000","last_became_active":"2024-03-26T08:31:22.387127+0000","last_became_peered":"2024-03-26T08:31:22.387127+0000","last_unstale":"2024-03-26T08:31:37.484219+0000","last_undegraded":"2024-03-26T08:31:37.484219+0000","last_fullsized":"2024-03-26T08:31:37.484219+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:52:36.156193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.484048+0000","last_change":"2024-03-26T08:31:22.404275+0000","last_active":"2024-03-26T08:31:37.484048+0000","last_peered":"2024-03-26T08:31:37.484048+0000","last_clean":"2024-03-26T08:31:37.484048+0000","last_became_active":"2024-03-26T08:31:22.403909+0000","last_became_peered":"2024-03-26T08:31:22.403909+0000","last_unstale":"2024-03-26T08:31:37.484048+0000","last_undegraded":"2024-03-26T08:31:37.484048+0000","last_fullsized":"2024-03-26T08:31:37.484048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:55:56.295719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.481567+0000","last_change":"2024-03-26T08:31:22.381310+0000","last_active":"2024-03-26T08:31:37.481567+0000","last_peered":"2024-03-26T08:31:37.481567+0000","last_clean":"2024-03-26T08:31:37.481567+0000","last_became_active":"2024-03-26T08:31:22.380721+0000","last_became_peered":"2024-03-26T08:31:22.380721+0000","last_unstale":"2024-03-26T08:31:37.481567+0000","last_undegraded":"2024-03-26T08:31:37.481567+0000","last_fullsized":"2024-03-26T08:31:37.481567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:42:30.325409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.482566+0000","last_change":"2024-03-26T08:31:22.396906+0000","last_active":"2024-03-26T08:31:37.482566+0000","last_peered":"2024-03-26T08:31:37.482566+0000","last_clean":"2024-03-26T08:31:37.482566+0000","last_became_active":"2024-03-26T08:31:22.392815+0000","last_became_peered":"2024-03-26T08:31:22.392815+0000","last_unstale":"2024-03-26T08:31:37.482566+0000","last_undegraded":"2024-03-26T08:31:37.482566+0000","last_fullsized":"2024-03-26T08:31:37.482566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:04:25.761459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.489012+0000","last_change":"2024-03-26T08:31:22.389374+0000","last_active":"2024-03-26T08:31:37.489012+0000","last_peered":"2024-03-26T08:31:37.489012+0000","last_clean":"2024-03-26T08:31:37.489012+0000","last_became_active":"2024-03-26T08:31:22.389012+0000","last_became_peered":"2024-03-26T08:31:22.389012+0000","last_unstale":"2024-03-26T08:31:37.489012+0000","last_undegraded":"2024-03-26T08:31:37.489012+0000","last_fullsized":"2024-03-26T08:31:37.489012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:23:35.696335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.488660+0000","last_change":"2024-03-26T08:31:22.378805+0000","last_active":"2024-03-26T08:31:37.488660+0000","last_peered":"2024-03-26T08:31:37.488660+0000","last_clean":"2024-03-26T08:31:37.488660+0000","last_became_active":"2024-03-26T08:31:22.378313+0000","last_became_peered":"2024-03-26T08:31:22.378313+0000","last_unstale":"2024-03-26T08:31:37.488660+0000","last_undegraded":"2024-03-26T08:31:37.488660+0000","last_fullsized":"2024-03-26T08:31:37.488660+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:10:03.816988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.497874+0000","last_change":"2024-03-26T08:31:37.495699+0000","last_active":"2024-03-26T08:31:37.497874+0000","last_peered":"2024-03-26T08:31:37.497874+0000","last_clean":"2024-03-26T08:31:37.497874+0000","last_became_active":"2024-03-26T08:31:37.494458+0000","last_became_peered":"2024-03-26T08:31:37.494458+0000","last_unstale":"2024-03-26T08:31:37.497874+0000","last_undegraded":"2024-03-26T08:31:37.497874+0000","last_fullsized":"2024-03-26T08:31:37.497874+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:32:36.973898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,10],"acting":[9,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.479928+0000","last_change":"2024-03-26T08:31:22.383958+0000","last_active":"2024-03-26T08:31:37.479928+0000","last_peered":"2024-03-26T08:31:37.479928+0000","last_clean":"2024-03-26T08:31:37.479928+0000","last_became_active":"2024-03-26T08:31:22.383167+0000","last_became_peered":"2024-03-26T08:31:22.383167+0000","last_unstale":"2024-03-26T08:31:37.479928+0000","last_undegraded":"2024-03-26T08:31:37.479928+0000","last_fullsized":"2024-03-26T08:31:37.479928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:38:24.265054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.493950+0000","last_change":"2024-03-26T08:31:22.394564+0000","last_active":"2024-03-26T08:31:37.493950+0000","last_peered":"2024-03-26T08:31:37.493950+0000","last_clean":"2024-03-26T08:31:37.493950+0000","last_became_active":"2024-03-26T08:31:22.394196+0000","last_became_peered":"2024-03-26T08:31:22.394196+0000","last_unstale":"2024-03-26T08:31:37.493950+0000","last_undegraded":"2024-03-26T08:31:37.493950+0000","last_fullsized":"2024-03-26T08:31:37.493950+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:35:47.429246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.494708+0000","last_change":"2024-03-26T08:31:22.388214+0000","last_active":"2024-03-26T08:31:37.494708+0000","last_peered":"2024-03-26T08:31:37.494708+0000","last_clean":"2024-03-26T08:31:37.494708+0000","last_became_active":"2024-03-26T08:31:22.387760+0000","last_became_peered":"2024-03-26T08:31:22.387760+0000","last_unstale":"2024-03-26T08:31:37.494708+0000","last_undegraded":"2024-03-26T08:31:37.494708+0000","last_fullsized":"2024-03-26T08:31:37.494708+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:25:22.498259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.481292+0000","last_change":"2024-03-26T08:31:22.390892+0000","last_active":"2024-03-26T08:31:37.481292+0000","last_peered":"2024-03-26T08:31:37.481292+0000","last_clean":"2024-03-26T08:31:37.481292+0000","last_became_active":"2024-03-26T08:31:22.390500+0000","last_became_peered":"2024-03-26T08:31:22.390500+0000","last_unstale":"2024-03-26T08:31:37.481292+0000","last_undegraded":"2024-03-26T08:31:37.481292+0000","last_fullsized":"2024-03-26T08:31:37.481292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:29:01.832399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.490525+0000","last_change":"2024-03-26T08:31:22.382442+0000","last_active":"2024-03-26T08:31:37.490525+0000","last_peered":"2024-03-26T08:31:37.490525+0000","last_clean":"2024-03-26T08:31:37.490525+0000","last_became_active":"2024-03-26T08:31:22.380346+0000","last_became_peered":"2024-03-26T08:31:22.380346+0000","last_unstale":"2024-03-26T08:31:37.490525+0000","last_undegraded":"2024-03-26T08:31:37.490525+0000","last_fullsized":"2024-03-26T08:31:37.490525+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:36:36.451504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.483321+0000","last_change":"2024-03-26T08:31:22.387401+0000","last_active":"2024-03-26T08:31:37.483321+0000","last_peered":"2024-03-26T08:31:37.483321+0000","last_clean":"2024-03-26T08:31:37.483321+0000","last_became_active":"2024-03-26T08:31:22.387020+0000","last_became_peered":"2024-03-26T08:31:22.387020+0000","last_unstale":"2024-03-26T08:31:37.483321+0000","last_undegraded":"2024-03-26T08:31:37.483321+0000","last_fullsized":"2024-03-26T08:31:37.483321+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:50:15.073663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.483648+0000","last_change":"2024-03-26T08:31:22.397971+0000","last_active":"2024-03-26T08:31:37.483648+0000","last_peered":"2024-03-26T08:31:37.483648+0000","last_clean":"2024-03-26T08:31:37.483648+0000","last_became_active":"2024-03-26T08:31:22.395371+0000","last_became_peered":"2024-03-26T08:31:22.395371+0000","last_unstale":"2024-03-26T08:31:37.483648+0000","last_undegraded":"2024-03-26T08:31:37.483648+0000","last_fullsized":"2024-03-26T08:31:37.483648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:59:51.053712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.477150+0000","last_change":"2024-03-26T08:31:22.394437+0000","last_active":"2024-03-26T08:31:37.477150+0000","last_peered":"2024-03-26T08:31:37.477150+0000","last_clean":"2024-03-26T08:31:37.477150+0000","last_became_active":"2024-03-26T08:31:22.386794+0000","last_became_peered":"2024-03-26T08:31:22.386794+0000","last_unstale":"2024-03-26T08:31:37.477150+0000","last_undegraded":"2024-03-26T08:31:37.477150+0000","last_fullsized":"2024-03-26T08:31:37.477150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:56:01.686361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.488931+0000","last_change":"2024-03-26T08:31:22.397887+0000","last_active":"2024-03-26T08:31:37.488931+0000","last_peered":"2024-03-26T08:31:37.488931+0000","last_clean":"2024-03-26T08:31:37.488931+0000","last_became_active":"2024-03-26T08:31:22.397086+0000","last_became_peered":"2024-03-26T08:31:22.397086+0000","last_unstale":"2024-03-26T08:31:37.488931+0000","last_undegraded":"2024-03-26T08:31:37.488931+0000","last_fullsized":"2024-03-26T08:31:37.488931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:41:47.512904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.478384+0000","last_change":"2024-03-26T08:31:22.398868+0000","last_active":"2024-03-26T08:31:37.478384+0000","last_peered":"2024-03-26T08:31:37.478384+0000","last_clean":"2024-03-26T08:31:37.478384+0000","last_became_active":"2024-03-26T08:31:22.398536+0000","last_became_peered":"2024-03-26T08:31:22.398536+0000","last_unstale":"2024-03-26T08:31:37.478384+0000","last_undegraded":"2024-03-26T08:31:37.478384+0000","last_fullsized":"2024-03-26T08:31:37.478384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:59:07.931120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.619260+0000","last_change":"2024-03-26T08:31:22.389805+0000","last_active":"2024-03-26T08:31:37.619260+0000","last_peered":"2024-03-26T08:31:37.619260+0000","last_clean":"2024-03-26T08:31:37.619260+0000","last_became_active":"2024-03-26T08:31:22.389410+0000","last_became_peered":"2024-03-26T08:31:22.389410+0000","last_unstale":"2024-03-26T08:31:37.619260+0000","last_undegraded":"2024-03-26T08:31:37.619260+0000","last_fullsized":"2024-03-26T08:31:37.619260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:36:29.934029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.618816+0000","last_change":"2024-03-26T08:31:26.437909+0000","last_active":"2024-03-26T08:31:37.618816+0000","last_peered":"2024-03-26T08:31:37.618816+0000","last_clean":"2024-03-26T08:31:37.618816+0000","last_became_active":"2024-03-26T08:31:26.437337+0000","last_became_peered":"2024-03-26T08:31:26.437337+0000","last_unstale":"2024-03-26T08:31:37.618816+0000","last_undegraded":"2024-03-26T08:31:37.618816+0000","last_fullsized":"2024-03-26T08:31:37.618816+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:01:15.856194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.476962+0000","last_change":"2024-03-26T08:31:21.367133+0000","last_active":"2024-03-26T08:31:37.476962+0000","last_peered":"2024-03-26T08:31:37.476962+0000","last_clean":"2024-03-26T08:31:37.476962+0000","last_became_active":"2024-03-26T08:31:21.366206+0000","last_became_peered":"2024-03-26T08:31:21.366206+0000","last_unstale":"2024-03-26T08:31:37.476962+0000","last_undegraded":"2024-03-26T08:31:37.476962+0000","last_fullsized":"2024-03-26T08:31:37.476962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:11:16.614464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.497113+0000","last_change":"2024-03-26T08:31:22.385984+0000","last_active":"2024-03-26T08:31:37.497113+0000","last_peered":"2024-03-26T08:31:37.497113+0000","last_clean":"2024-03-26T08:31:37.497113+0000","last_became_active":"2024-03-26T08:31:22.385512+0000","last_became_peered":"2024-03-26T08:31:22.385512+0000","last_unstale":"2024-03-26T08:31:37.497113+0000","last_undegraded":"2024-03-26T08:31:37.497113+0000","last_fullsized":"2024-03-26T08:31:37.497113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:46:24.087975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.477452+0000","last_change":"2024-03-26T08:31:26.454001+0000","last_active":"2024-03-26T08:31:37.477452+0000","last_peered":"2024-03-26T08:31:37.477452+0000","last_clean":"2024-03-26T08:31:37.477452+0000","last_became_active":"2024-03-26T08:31:26.453474+0000","last_became_peered":"2024-03-26T08:31:26.453474+0000","last_unstale":"2024-03-26T08:31:37.477452+0000","last_undegraded":"2024-03-26T08:31:37.477452+0000","last_fullsized":"2024-03-26T08:31:37.477452+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:09:42.197500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.497998+0000","last_change":"2024-03-26T08:31:21.365012+0000","last_active":"2024-03-26T08:31:37.497998+0000","last_peered":"2024-03-26T08:31:37.497998+0000","last_clean":"2024-03-26T08:31:37.497998+0000","last_became_active":"2024-03-26T08:31:21.363981+0000","last_became_peered":"2024-03-26T08:31:21.363981+0000","last_unstale":"2024-03-26T08:31:37.497998+0000","last_undegraded":"2024-03-26T08:31:37.497998+0000","last_fullsized":"2024-03-26T08:31:37.497998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:12:36.581648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.483616+0000","last_change":"2024-03-26T08:31:22.387960+0000","last_active":"2024-03-26T08:31:37.483616+0000","last_peered":"2024-03-26T08:31:37.483616+0000","last_clean":"2024-03-26T08:31:37.483616+0000","last_became_active":"2024-03-26T08:31:22.387627+0000","last_became_peered":"2024-03-26T08:31:22.387627+0000","last_unstale":"2024-03-26T08:31:37.483616+0000","last_undegraded":"2024-03-26T08:31:37.483616+0000","last_fullsized":"2024-03-26T08:31:37.483616+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:20:09.265075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.485506+0000","last_change":"2024-03-26T08:31:26.450387+0000","last_active":"2024-03-26T08:31:37.485506+0000","last_peered":"2024-03-26T08:31:37.485506+0000","last_clean":"2024-03-26T08:31:37.485506+0000","last_became_active":"2024-03-26T08:31:26.449418+0000","last_became_peered":"2024-03-26T08:31:26.449418+0000","last_unstale":"2024-03-26T08:31:37.485506+0000","last_undegraded":"2024-03-26T08:31:37.485506+0000","last_fullsized":"2024-03-26T08:31:37.485506+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:51:59.022851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.483493+0000","last_change":"2024-03-26T08:31:21.373422+0000","last_active":"2024-03-26T08:31:37.483493+0000","last_peered":"2024-03-26T08:31:37.483493+0000","last_clean":"2024-03-26T08:31:37.483493+0000","last_became_active":"2024-03-26T08:31:21.372916+0000","last_became_peered":"2024-03-26T08:31:21.372916+0000","last_unstale":"2024-03-26T08:31:37.483493+0000","last_undegraded":"2024-03-26T08:31:37.483493+0000","last_fullsized":"2024-03-26T08:31:37.483493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:09:08.313365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.478286+0000","last_change":"2024-03-26T08:31:22.403385+0000","last_active":"2024-03-26T08:31:37.478286+0000","last_peered":"2024-03-26T08:31:37.478286+0000","last_clean":"2024-03-26T08:31:37.478286+0000","last_became_active":"2024-03-26T08:31:22.398021+0000","last_became_peered":"2024-03-26T08:31:22.398021+0000","last_unstale":"2024-03-26T08:31:37.478286+0000","last_undegraded":"2024-03-26T08:31:37.478286+0000","last_fullsized":"2024-03-26T08:31:37.478286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:56:06.691167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.479767+0000","last_change":"2024-03-26T08:31:26.449049+0000","last_active":"2024-03-26T08:31:37.479767+0000","last_peered":"2024-03-26T08:31:37.479767+0000","last_clean":"2024-03-26T08:31:37.479767+0000","last_became_active":"2024-03-26T08:31:26.448510+0000","last_became_peered":"2024-03-26T08:31:26.448510+0000","last_unstale":"2024-03-26T08:31:37.479767+0000","last_undegraded":"2024-03-26T08:31:37.479767+0000","last_fullsized":"2024-03-26T08:31:37.479767+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:24:58.699699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.482690+0000","last_change":"2024-03-26T08:31:21.367289+0000","last_active":"2024-03-26T08:31:37.482690+0000","last_peered":"2024-03-26T08:31:37.482690+0000","last_clean":"2024-03-26T08:31:37.482690+0000","last_became_active":"2024-03-26T08:31:21.366960+0000","last_became_peered":"2024-03-26T08:31:21.366960+0000","last_unstale":"2024-03-26T08:31:37.482690+0000","last_undegraded":"2024-03-26T08:31:37.482690+0000","last_fullsized":"2024-03-26T08:31:37.482690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:22:25.479383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.495835+0000","last_change":"2024-03-26T08:31:22.385815+0000","last_active":"2024-03-26T08:31:37.495835+0000","last_peered":"2024-03-26T08:31:37.495835+0000","last_clean":"2024-03-26T08:31:37.495835+0000","last_became_active":"2024-03-26T08:31:22.385503+0000","last_became_peered":"2024-03-26T08:31:22.385503+0000","last_unstale":"2024-03-26T08:31:37.495835+0000","last_undegraded":"2024-03-26T08:31:37.495835+0000","last_fullsized":"2024-03-26T08:31:37.495835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:33:35.768902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.618892+0000","last_change":"2024-03-26T08:31:26.449656+0000","last_active":"2024-03-26T08:31:37.618892+0000","last_peered":"2024-03-26T08:31:37.618892+0000","last_clean":"2024-03-26T08:31:37.618892+0000","last_became_active":"2024-03-26T08:31:26.448843+0000","last_became_peered":"2024-03-26T08:31:26.448843+0000","last_unstale":"2024-03-26T08:31:37.618892+0000","last_undegraded":"2024-03-26T08:31:37.618892+0000","last_fullsized":"2024-03-26T08:31:37.618892+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:11:28.135551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.499950+0000","last_change":"2024-03-26T08:31:21.373384+0000","last_active":"2024-03-26T08:31:37.499950+0000","last_peered":"2024-03-26T08:31:37.499950+0000","last_clean":"2024-03-26T08:31:37.499950+0000","last_became_active":"2024-03-26T08:31:21.371708+0000","last_became_peered":"2024-03-26T08:31:21.371708+0000","last_unstale":"2024-03-26T08:31:37.499950+0000","last_undegraded":"2024-03-26T08:31:37.499950+0000","last_fullsized":"2024-03-26T08:31:37.499950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:46:09.745738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.482429+0000","last_change":"2024-03-26T08:31:22.383990+0000","last_active":"2024-03-26T08:31:37.482429+0000","last_peered":"2024-03-26T08:31:37.482429+0000","last_clean":"2024-03-26T08:31:37.482429+0000","last_became_active":"2024-03-26T08:31:22.383580+0000","last_became_peered":"2024-03-26T08:31:22.383580+0000","last_unstale":"2024-03-26T08:31:37.482429+0000","last_undegraded":"2024-03-26T08:31:37.482429+0000","last_fullsized":"2024-03-26T08:31:37.482429+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:32:36.037407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.618204+0000","last_change":"2024-03-26T08:31:26.445710+0000","last_active":"2024-03-26T08:31:37.618204+0000","last_peered":"2024-03-26T08:31:37.618204+0000","last_clean":"2024-03-26T08:31:37.618204+0000","last_became_active":"2024-03-26T08:31:26.445189+0000","last_became_peered":"2024-03-26T08:31:26.445189+0000","last_unstale":"2024-03-26T08:31:37.618204+0000","last_undegraded":"2024-03-26T08:31:37.618204+0000","last_fullsized":"2024-03-26T08:31:37.618204+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:31:23.033192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.488809+0000","last_change":"2024-03-26T08:31:21.362175+0000","last_active":"2024-03-26T08:31:37.488809+0000","last_peered":"2024-03-26T08:31:37.488809+0000","last_clean":"2024-03-26T08:31:37.488809+0000","last_became_active":"2024-03-26T08:31:21.361805+0000","last_became_peered":"2024-03-26T08:31:21.361805+0000","last_unstale":"2024-03-26T08:31:37.488809+0000","last_undegraded":"2024-03-26T08:31:37.488809+0000","last_fullsized":"2024-03-26T08:31:37.488809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:38:26.537955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.500216+0000","last_change":"2024-03-26T08:31:22.380489+0000","last_active":"2024-03-26T08:31:37.500216+0000","last_peered":"2024-03-26T08:31:37.500216+0000","last_clean":"2024-03-26T08:31:37.500216+0000","last_became_active":"2024-03-26T08:31:22.380139+0000","last_became_peered":"2024-03-26T08:31:22.380139+0000","last_unstale":"2024-03-26T08:31:37.500216+0000","last_undegraded":"2024-03-26T08:31:37.500216+0000","last_fullsized":"2024-03-26T08:31:37.500216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:09:18.550790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.488626+0000","last_change":"2024-03-26T08:31:26.454535+0000","last_active":"2024-03-26T08:31:37.488626+0000","last_peered":"2024-03-26T08:31:37.488626+0000","last_clean":"2024-03-26T08:31:37.488626+0000","last_became_active":"2024-03-26T08:31:26.454106+0000","last_became_peered":"2024-03-26T08:31:26.454106+0000","last_unstale":"2024-03-26T08:31:37.488626+0000","last_undegraded":"2024-03-26T08:31:37.488626+0000","last_fullsized":"2024-03-26T08:31:37.488626+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:57:28.870653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.489004+0000","last_change":"2024-03-26T08:31:21.347236+0000","last_active":"2024-03-26T08:31:37.489004+0000","last_peered":"2024-03-26T08:31:37.489004+0000","last_clean":"2024-03-26T08:31:37.489004+0000","last_became_active":"2024-03-26T08:31:21.346770+0000","last_became_peered":"2024-03-26T08:31:21.346770+0000","last_unstale":"2024-03-26T08:31:37.489004+0000","last_undegraded":"2024-03-26T08:31:37.489004+0000","last_fullsized":"2024-03-26T08:31:37.489004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:17:22.927636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.491908+0000","last_change":"2024-03-26T08:31:22.390623+0000","last_active":"2024-03-26T08:31:37.491908+0000","last_peered":"2024-03-26T08:31:37.491908+0000","last_clean":"2024-03-26T08:31:37.491908+0000","last_became_active":"2024-03-26T08:31:22.389512+0000","last_became_peered":"2024-03-26T08:31:22.389512+0000","last_unstale":"2024-03-26T08:31:37.491908+0000","last_undegraded":"2024-03-26T08:31:37.491908+0000","last_fullsized":"2024-03-26T08:31:37.491908+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:04:31.317386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.483881+0000","last_change":"2024-03-26T08:31:26.445277+0000","last_active":"2024-03-26T08:31:37.483881+0000","last_peered":"2024-03-26T08:31:37.483881+0000","last_clean":"2024-03-26T08:31:37.483881+0000","last_became_active":"2024-03-26T08:31:26.443558+0000","last_became_peered":"2024-03-26T08:31:26.443558+0000","last_unstale":"2024-03-26T08:31:37.483881+0000","last_undegraded":"2024-03-26T08:31:37.483881+0000","last_fullsized":"2024-03-26T08:31:37.483881+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:20:16.688699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.496031+0000","last_change":"2024-03-26T08:31:21.371523+0000","last_active":"2024-03-26T08:31:37.496031+0000","last_peered":"2024-03-26T08:31:37.496031+0000","last_clean":"2024-03-26T08:31:37.496031+0000","last_became_active":"2024-03-26T08:31:21.371136+0000","last_became_peered":"2024-03-26T08:31:21.371136+0000","last_unstale":"2024-03-26T08:31:37.496031+0000","last_undegraded":"2024-03-26T08:31:37.496031+0000","last_fullsized":"2024-03-26T08:31:37.496031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:06:55.083579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.475639+0000","last_change":"2024-03-26T08:31:22.386237+0000","last_active":"2024-03-26T08:31:37.475639+0000","last_peered":"2024-03-26T08:31:37.475639+0000","last_clean":"2024-03-26T08:31:37.475639+0000","last_became_active":"2024-03-26T08:31:22.385874+0000","last_became_peered":"2024-03-26T08:31:22.385874+0000","last_unstale":"2024-03-26T08:31:37.475639+0000","last_undegraded":"2024-03-26T08:31:37.475639+0000","last_fullsized":"2024-03-26T08:31:37.475639+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:14:58.278611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.492261+0000","last_change":"2024-03-26T08:31:26.452464+0000","last_active":"2024-03-26T08:31:37.492261+0000","last_peered":"2024-03-26T08:31:37.492261+0000","last_clean":"2024-03-26T08:31:37.492261+0000","last_became_active":"2024-03-26T08:31:26.451862+0000","last_became_peered":"2024-03-26T08:31:26.451862+0000","last_unstale":"2024-03-26T08:31:37.492261+0000","last_undegraded":"2024-03-26T08:31:37.492261+0000","last_fullsized":"2024-03-26T08:31:37.492261+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:48:29.616247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.489892+0000","last_change":"2024-03-26T08:31:21.365283+0000","last_active":"2024-03-26T08:31:37.489892+0000","last_peered":"2024-03-26T08:31:37.489892+0000","last_clean":"2024-03-26T08:31:37.489892+0000","last_became_active":"2024-03-26T08:31:21.364902+0000","last_became_peered":"2024-03-26T08:31:21.364902+0000","last_unstale":"2024-03-26T08:31:37.489892+0000","last_undegraded":"2024-03-26T08:31:37.489892+0000","last_fullsized":"2024-03-26T08:31:37.489892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:55:39.394355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.479955+0000","last_change":"2024-03-26T08:31:22.391645+0000","last_active":"2024-03-26T08:31:37.479955+0000","last_peered":"2024-03-26T08:31:37.479955+0000","last_clean":"2024-03-26T08:31:37.479955+0000","last_became_active":"2024-03-26T08:31:22.391280+0000","last_became_peered":"2024-03-26T08:31:22.391280+0000","last_unstale":"2024-03-26T08:31:37.479955+0000","last_undegraded":"2024-03-26T08:31:37.479955+0000","last_fullsized":"2024-03-26T08:31:37.479955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:31:48.963287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.475260+0000","last_change":"2024-03-26T08:31:26.446215+0000","last_active":"2024-03-26T08:31:37.475260+0000","last_peered":"2024-03-26T08:31:37.475260+0000","last_clean":"2024-03-26T08:31:37.475260+0000","last_became_active":"2024-03-26T08:31:26.445765+0000","last_became_peered":"2024-03-26T08:31:26.445765+0000","last_unstale":"2024-03-26T08:31:37.475260+0000","last_undegraded":"2024-03-26T08:31:37.475260+0000","last_fullsized":"2024-03-26T08:31:37.475260+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:57:14.809610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.495756+0000","last_change":"2024-03-26T08:31:21.373345+0000","last_active":"2024-03-26T08:31:37.495756+0000","last_peered":"2024-03-26T08:31:37.495756+0000","last_clean":"2024-03-26T08:31:37.495756+0000","last_became_active":"2024-03-26T08:31:21.372906+0000","last_became_peered":"2024-03-26T08:31:21.372906+0000","last_unstale":"2024-03-26T08:31:37.495756+0000","last_undegraded":"2024-03-26T08:31:37.495756+0000","last_fullsized":"2024-03-26T08:31:37.495756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:20:00.518815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.484356+0000","last_change":"2024-03-26T08:31:22.393433+0000","last_active":"2024-03-26T08:31:37.484356+0000","last_peered":"2024-03-26T08:31:37.484356+0000","last_clean":"2024-03-26T08:31:37.484356+0000","last_became_active":"2024-03-26T08:31:22.393100+0000","last_became_peered":"2024-03-26T08:31:22.393100+0000","last_unstale":"2024-03-26T08:31:37.484356+0000","last_undegraded":"2024-03-26T08:31:37.484356+0000","last_fullsized":"2024-03-26T08:31:37.484356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:33:30.418492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.482525+0000","last_change":"2024-03-26T08:31:26.449975+0000","last_active":"2024-03-26T08:31:37.482525+0000","last_peered":"2024-03-26T08:31:37.482525+0000","last_clean":"2024-03-26T08:31:37.482525+0000","last_became_active":"2024-03-26T08:31:26.448345+0000","last_became_peered":"2024-03-26T08:31:26.448345+0000","last_unstale":"2024-03-26T08:31:37.482525+0000","last_undegraded":"2024-03-26T08:31:37.482525+0000","last_fullsized":"2024-03-26T08:31:37.482525+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:01:24.765939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.495512+0000","last_change":"2024-03-26T08:31:21.364195+0000","last_active":"2024-03-26T08:31:37.495512+0000","last_peered":"2024-03-26T08:31:37.495512+0000","last_clean":"2024-03-26T08:31:37.495512+0000","last_became_active":"2024-03-26T08:31:21.363887+0000","last_became_peered":"2024-03-26T08:31:21.363887+0000","last_unstale":"2024-03-26T08:31:37.495512+0000","last_undegraded":"2024-03-26T08:31:37.495512+0000","last_fullsized":"2024-03-26T08:31:37.495512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:03:03.015581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.493245+0000","last_change":"2024-03-26T08:31:37.493245+0000","last_active":"2024-03-26T08:31:37.493245+0000","last_peered":"2024-03-26T08:31:37.493245+0000","last_clean":"2024-03-26T08:31:37.493245+0000","last_became_active":"2024-03-26T08:31:37.492509+0000","last_became_peered":"2024-03-26T08:31:37.492509+0000","last_unstale":"2024-03-26T08:31:37.493245+0000","last_undegraded":"2024-03-26T08:31:37.493245+0000","last_fullsized":"2024-03-26T08:31:37.493245+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:28:56.355010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.475182+0000","last_change":"2024-03-26T08:31:26.449405+0000","last_active":"2024-03-26T08:31:37.475182+0000","last_peered":"2024-03-26T08:31:37.475182+0000","last_clean":"2024-03-26T08:31:37.475182+0000","last_became_active":"2024-03-26T08:31:26.448934+0000","last_became_peered":"2024-03-26T08:31:26.448934+0000","last_unstale":"2024-03-26T08:31:37.475182+0000","last_undegraded":"2024-03-26T08:31:37.475182+0000","last_fullsized":"2024-03-26T08:31:37.475182+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:12:22.797816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'17","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:49.655720+0000","last_change":"2024-03-26T08:31:21.372300+0000","last_active":"2024-03-26T08:31:49.655720+0000","last_peered":"2024-03-26T08:31:49.655720+0000","last_clean":"2024-03-26T08:31:49.655720+0000","last_became_active":"2024-03-26T08:31:21.370103+0000","last_became_peered":"2024-03-26T08:31:21.370103+0000","last_unstale":"2024-03-26T08:31:49.655720+0000","last_undegraded":"2024-03-26T08:31:49.655720+0000","last_fullsized":"2024-03-26T08:31:49.655720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:54:52.657768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23141,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":15,"num_write":17,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.483057+0000","last_change":"2024-03-26T08:31:22.373102+0000","last_active":"2024-03-26T08:31:37.483057+0000","last_peered":"2024-03-26T08:31:37.483057+0000","last_clean":"2024-03-26T08:31:37.483057+0000","last_became_active":"2024-03-26T08:31:22.372792+0000","last_became_peered":"2024-03-26T08:31:22.372792+0000","last_unstale":"2024-03-26T08:31:37.483057+0000","last_undegraded":"2024-03-26T08:31:37.483057+0000","last_fullsized":"2024-03-26T08:31:37.483057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:58:58.274941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.496650+0000","last_change":"2024-03-26T08:31:26.448172+0000","last_active":"2024-03-26T08:31:37.496650+0000","last_peered":"2024-03-26T08:31:37.496650+0000","last_clean":"2024-03-26T08:31:37.496650+0000","last_became_active":"2024-03-26T08:31:26.447550+0000","last_became_peered":"2024-03-26T08:31:26.447550+0000","last_unstale":"2024-03-26T08:31:37.496650+0000","last_undegraded":"2024-03-26T08:31:37.496650+0000","last_fullsized":"2024-03-26T08:31:37.496650+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:56:19.806076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.618648+0000","last_change":"2024-03-26T08:31:21.363946+0000","last_active":"2024-03-26T08:31:37.618648+0000","last_peered":"2024-03-26T08:31:37.618648+0000","last_clean":"2024-03-26T08:31:37.618648+0000","last_became_active":"2024-03-26T08:31:21.363566+0000","last_became_peered":"2024-03-26T08:31:21.363566+0000","last_unstale":"2024-03-26T08:31:37.618648+0000","last_undegraded":"2024-03-26T08:31:37.618648+0000","last_fullsized":"2024-03-26T08:31:37.618648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:32:51.714454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.489387+0000","last_change":"2024-03-26T08:31:22.375813+0000","last_active":"2024-03-26T08:31:37.489387+0000","last_peered":"2024-03-26T08:31:37.489387+0000","last_clean":"2024-03-26T08:31:37.489387+0000","last_became_active":"2024-03-26T08:31:22.375437+0000","last_became_peered":"2024-03-26T08:31:22.375437+0000","last_unstale":"2024-03-26T08:31:37.489387+0000","last_undegraded":"2024-03-26T08:31:37.489387+0000","last_fullsized":"2024-03-26T08:31:37.489387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:48:25.167143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.488397+0000","last_change":"2024-03-26T08:31:26.452079+0000","last_active":"2024-03-26T08:31:37.488397+0000","last_peered":"2024-03-26T08:31:37.488397+0000","last_clean":"2024-03-26T08:31:37.488397+0000","last_became_active":"2024-03-26T08:31:26.451146+0000","last_became_peered":"2024-03-26T08:31:26.451146+0000","last_unstale":"2024-03-26T08:31:37.488397+0000","last_undegraded":"2024-03-26T08:31:37.488397+0000","last_fullsized":"2024-03-26T08:31:37.488397+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:08:57.803180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.499295+0000","last_change":"2024-03-26T08:31:21.371222+0000","last_active":"2024-03-26T08:31:37.499295+0000","last_peered":"2024-03-26T08:31:37.499295+0000","last_clean":"2024-03-26T08:31:37.499295+0000","last_became_active":"2024-03-26T08:31:21.370843+0000","last_became_peered":"2024-03-26T08:31:21.370843+0000","last_unstale":"2024-03-26T08:31:37.499295+0000","last_undegraded":"2024-03-26T08:31:37.499295+0000","last_fullsized":"2024-03-26T08:31:37.499295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:32:00.518713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.492125+0000","last_change":"2024-03-26T08:31:22.379781+0000","last_active":"2024-03-26T08:31:37.492125+0000","last_peered":"2024-03-26T08:31:37.492125+0000","last_clean":"2024-03-26T08:31:37.492125+0000","last_became_active":"2024-03-26T08:31:22.379357+0000","last_became_peered":"2024-03-26T08:31:22.379357+0000","last_unstale":"2024-03-26T08:31:37.492125+0000","last_undegraded":"2024-03-26T08:31:37.492125+0000","last_fullsized":"2024-03-26T08:31:37.492125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:21:52.306343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.500702+0000","last_change":"2024-03-26T08:31:37.498490+0000","last_active":"2024-03-26T08:31:37.500702+0000","last_peered":"2024-03-26T08:31:37.500702+0000","last_clean":"2024-03-26T08:31:37.500702+0000","last_became_active":"2024-03-26T08:31:37.496608+0000","last_became_peered":"2024-03-26T08:31:37.496608+0000","last_unstale":"2024-03-26T08:31:37.500702+0000","last_undegraded":"2024-03-26T08:31:37.500702+0000","last_fullsized":"2024-03-26T08:31:37.500702+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:32:11.653087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.485003+0000","last_change":"2024-03-26T08:31:21.366945+0000","last_active":"2024-03-26T08:31:37.485003+0000","last_peered":"2024-03-26T08:31:37.485003+0000","last_clean":"2024-03-26T08:31:37.485003+0000","last_became_active":"2024-03-26T08:31:21.366548+0000","last_became_peered":"2024-03-26T08:31:21.366548+0000","last_unstale":"2024-03-26T08:31:37.485003+0000","last_undegraded":"2024-03-26T08:31:37.485003+0000","last_fullsized":"2024-03-26T08:31:37.485003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:13:52.037163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.490058+0000","last_change":"2024-03-26T08:31:22.387575+0000","last_active":"2024-03-26T08:31:37.490058+0000","last_peered":"2024-03-26T08:31:37.490058+0000","last_clean":"2024-03-26T08:31:37.490058+0000","last_became_active":"2024-03-26T08:31:22.383815+0000","last_became_peered":"2024-03-26T08:31:22.383815+0000","last_unstale":"2024-03-26T08:31:37.490058+0000","last_undegraded":"2024-03-26T08:31:37.490058+0000","last_fullsized":"2024-03-26T08:31:37.490058+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:10:20.318597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.490627+0000","last_change":"2024-03-26T08:31:26.440745+0000","last_active":"2024-03-26T08:31:37.490627+0000","last_peered":"2024-03-26T08:31:37.490627+0000","last_clean":"2024-03-26T08:31:37.490627+0000","last_became_active":"2024-03-26T08:31:26.440199+0000","last_became_peered":"2024-03-26T08:31:26.440199+0000","last_unstale":"2024-03-26T08:31:37.490627+0000","last_undegraded":"2024-03-26T08:31:37.490627+0000","last_fullsized":"2024-03-26T08:31:37.490627+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:25:11.407274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.481224+0000","last_change":"2024-03-26T08:31:21.369227+0000","last_active":"2024-03-26T08:31:37.481224+0000","last_peered":"2024-03-26T08:31:37.481224+0000","last_clean":"2024-03-26T08:31:37.481224+0000","last_became_active":"2024-03-26T08:31:21.368709+0000","last_became_peered":"2024-03-26T08:31:21.368709+0000","last_unstale":"2024-03-26T08:31:37.481224+0000","last_undegraded":"2024-03-26T08:31:37.481224+0000","last_fullsized":"2024-03-26T08:31:37.481224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:30:04.714715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.498997+0000","last_change":"2024-03-26T08:31:22.384199+0000","last_active":"2024-03-26T08:31:37.498997+0000","last_peered":"2024-03-26T08:31:37.498997+0000","last_clean":"2024-03-26T08:31:37.498997+0000","last_became_active":"2024-03-26T08:31:22.383854+0000","last_became_peered":"2024-03-26T08:31:22.383854+0000","last_unstale":"2024-03-26T08:31:37.498997+0000","last_undegraded":"2024-03-26T08:31:37.498997+0000","last_fullsized":"2024-03-26T08:31:37.498997+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:33:24.240505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.498650+0000","last_change":"2024-03-26T08:31:26.447657+0000","last_active":"2024-03-26T08:31:37.498650+0000","last_peered":"2024-03-26T08:31:37.498650+0000","last_clean":"2024-03-26T08:31:37.498650+0000","last_became_active":"2024-03-26T08:31:26.444812+0000","last_became_peered":"2024-03-26T08:31:26.444812+0000","last_unstale":"2024-03-26T08:31:37.498650+0000","last_undegraded":"2024-03-26T08:31:37.498650+0000","last_fullsized":"2024-03-26T08:31:37.498650+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:58:21.422318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:40.553106+0000","last_change":"2024-03-26T08:31:37.510665+0000","last_active":"2024-03-26T08:31:40.553106+0000","last_peered":"2024-03-26T08:31:40.553106+0000","last_clean":"2024-03-26T08:31:40.553106+0000","last_became_active":"2024-03-26T08:31:37.504109+0000","last_became_peered":"2024-03-26T08:31:37.504109+0000","last_unstale":"2024-03-26T08:31:40.553106+0000","last_undegraded":"2024-03-26T08:31:40.553106+0000","last_fullsized":"2024-03-26T08:31:40.553106+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:38:21.446881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.502088+0000","last_change":"2024-03-26T08:31:37.502088+0000","last_active":"2024-03-26T08:31:37.502088+0000","last_peered":"2024-03-26T08:31:37.502088+0000","last_clean":"2024-03-26T08:31:37.502088+0000","last_became_active":"2024-03-26T08:31:37.499767+0000","last_became_peered":"2024-03-26T08:31:37.499767+0000","last_unstale":"2024-03-26T08:31:37.502088+0000","last_undegraded":"2024-03-26T08:31:37.502088+0000","last_fullsized":"2024-03-26T08:31:37.502088+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:55:32.652231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.482121+0000","last_change":"2024-03-26T08:31:26.479654+0000","last_active":"2024-03-26T08:31:37.482121+0000","last_peered":"2024-03-26T08:31:37.482121+0000","last_clean":"2024-03-26T08:31:37.482121+0000","last_became_active":"2024-03-26T08:31:26.478948+0000","last_became_peered":"2024-03-26T08:31:26.478948+0000","last_unstale":"2024-03-26T08:31:37.482121+0000","last_undegraded":"2024-03-26T08:31:37.482121+0000","last_fullsized":"2024-03-26T08:31:37.482121+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:33:58.445531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.476079+0000","last_change":"2024-03-26T08:31:21.365203+0000","last_active":"2024-03-26T08:31:37.476079+0000","last_peered":"2024-03-26T08:31:37.476079+0000","last_clean":"2024-03-26T08:31:37.476079+0000","last_became_active":"2024-03-26T08:31:21.364816+0000","last_became_peered":"2024-03-26T08:31:21.364816+0000","last_unstale":"2024-03-26T08:31:37.476079+0000","last_undegraded":"2024-03-26T08:31:37.476079+0000","last_fullsized":"2024-03-26T08:31:37.476079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:59:50.157011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.492710+0000","last_change":"2024-03-26T08:31:22.392014+0000","last_active":"2024-03-26T08:31:37.492710+0000","last_peered":"2024-03-26T08:31:37.492710+0000","last_clean":"2024-03-26T08:31:37.492710+0000","last_became_active":"2024-03-26T08:31:22.391627+0000","last_became_peered":"2024-03-26T08:31:22.391627+0000","last_unstale":"2024-03-26T08:31:37.492710+0000","last_undegraded":"2024-03-26T08:31:37.492710+0000","last_fullsized":"2024-03-26T08:31:37.492710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:59:12.544248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.490621+0000","last_change":"2024-03-26T08:31:26.444754+0000","last_active":"2024-03-26T08:31:37.490621+0000","last_peered":"2024-03-26T08:31:37.490621+0000","last_clean":"2024-03-26T08:31:37.490621+0000","last_became_active":"2024-03-26T08:31:26.444331+0000","last_became_peered":"2024-03-26T08:31:26.444331+0000","last_unstale":"2024-03-26T08:31:37.490621+0000","last_undegraded":"2024-03-26T08:31:37.490621+0000","last_fullsized":"2024-03-26T08:31:37.490621+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:27:58.430695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.490937+0000","last_change":"2024-03-26T08:31:21.371028+0000","last_active":"2024-03-26T08:31:37.490937+0000","last_peered":"2024-03-26T08:31:37.490937+0000","last_clean":"2024-03-26T08:31:37.490937+0000","last_became_active":"2024-03-26T08:31:21.370631+0000","last_became_peered":"2024-03-26T08:31:21.370631+0000","last_unstale":"2024-03-26T08:31:37.490937+0000","last_undegraded":"2024-03-26T08:31:37.490937+0000","last_fullsized":"2024-03-26T08:31:37.490937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:33:25.546255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.490360+0000","last_change":"2024-03-26T08:31:22.388791+0000","last_active":"2024-03-26T08:31:37.490360+0000","last_peered":"2024-03-26T08:31:37.490360+0000","last_clean":"2024-03-26T08:31:37.490360+0000","last_became_active":"2024-03-26T08:31:22.388342+0000","last_became_peered":"2024-03-26T08:31:22.388342+0000","last_unstale":"2024-03-26T08:31:37.490360+0000","last_undegraded":"2024-03-26T08:31:37.490360+0000","last_fullsized":"2024-03-26T08:31:37.490360+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:26:30.537880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.489879+0000","last_change":"2024-03-26T08:31:26.478919+0000","last_active":"2024-03-26T08:31:37.489879+0000","last_peered":"2024-03-26T08:31:37.489879+0000","last_clean":"2024-03-26T08:31:37.489879+0000","last_became_active":"2024-03-26T08:31:26.478330+0000","last_became_peered":"2024-03-26T08:31:26.478330+0000","last_unstale":"2024-03-26T08:31:37.489879+0000","last_undegraded":"2024-03-26T08:31:37.489879+0000","last_fullsized":"2024-03-26T08:31:37.489879+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:37:36.421982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.497216+0000","last_change":"2024-03-26T08:31:22.384299+0000","last_active":"2024-03-26T08:31:37.497216+0000","last_peered":"2024-03-26T08:31:37.497216+0000","last_clean":"2024-03-26T08:31:37.497216+0000","last_became_active":"2024-03-26T08:31:22.383955+0000","last_became_peered":"2024-03-26T08:31:22.383955+0000","last_unstale":"2024-03-26T08:31:37.497216+0000","last_undegraded":"2024-03-26T08:31:37.497216+0000","last_fullsized":"2024-03-26T08:31:37.497216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:08:53.065524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.488692+0000","last_change":"2024-03-26T08:31:21.358464+0000","last_active":"2024-03-26T08:31:37.488692+0000","last_peered":"2024-03-26T08:31:37.488692+0000","last_clean":"2024-03-26T08:31:37.488692+0000","last_became_active":"2024-03-26T08:31:21.356853+0000","last_became_peered":"2024-03-26T08:31:21.356853+0000","last_unstale":"2024-03-26T08:31:37.488692+0000","last_undegraded":"2024-03-26T08:31:37.488692+0000","last_fullsized":"2024-03-26T08:31:37.488692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:08:21.395306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.502144+0000","last_change":"2024-03-26T08:31:37.502144+0000","last_active":"2024-03-26T08:31:37.502144+0000","last_peered":"2024-03-26T08:31:37.502144+0000","last_clean":"2024-03-26T08:31:37.502144+0000","last_became_active":"2024-03-26T08:31:37.501465+0000","last_became_peered":"2024-03-26T08:31:37.501465+0000","last_unstale":"2024-03-26T08:31:37.502144+0000","last_undegraded":"2024-03-26T08:31:37.502144+0000","last_fullsized":"2024-03-26T08:31:37.502144+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:30:18.771898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.495955+0000","last_change":"2024-03-26T08:31:22.403380+0000","last_active":"2024-03-26T08:31:37.495955+0000","last_peered":"2024-03-26T08:31:37.495955+0000","last_clean":"2024-03-26T08:31:37.495955+0000","last_became_active":"2024-03-26T08:31:22.400991+0000","last_became_peered":"2024-03-26T08:31:22.400991+0000","last_unstale":"2024-03-26T08:31:37.495955+0000","last_undegraded":"2024-03-26T08:31:37.495955+0000","last_fullsized":"2024-03-26T08:31:37.495955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:04:38.842919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.482253+0000","last_change":"2024-03-26T08:31:21.358570+0000","last_active":"2024-03-26T08:31:37.482253+0000","last_peered":"2024-03-26T08:31:37.482253+0000","last_clean":"2024-03-26T08:31:37.482253+0000","last_became_active":"2024-03-26T08:31:21.356603+0000","last_became_peered":"2024-03-26T08:31:21.356603+0000","last_unstale":"2024-03-26T08:31:37.482253+0000","last_undegraded":"2024-03-26T08:31:37.482253+0000","last_fullsized":"2024-03-26T08:31:37.482253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:10:28.829223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":351,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.491987+0000","last_change":"2024-03-26T08:29:39.048593+0000","last_active":"2024-03-26T08:31:37.491987+0000","last_peered":"2024-03-26T08:31:37.491987+0000","last_clean":"2024-03-26T08:31:37.491987+0000","last_became_active":"2024-03-26T08:29:39.047909+0000","last_became_peered":"2024-03-26T08:29:39.047909+0000","last_unstale":"2024-03-26T08:31:37.491987+0000","last_undegraded":"2024-03-26T08:31:37.491987+0000","last_fullsized":"2024-03-26T08:31:37.491987+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:24:55.011726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:24:55.011726+0000","last_clean_scrub_stamp":"2024-03-26T08:24:55.011726+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:46:55.510857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.495687+0000","last_change":"2024-03-26T08:31:26.478740+0000","last_active":"2024-03-26T08:31:37.495687+0000","last_peered":"2024-03-26T08:31:37.495687+0000","last_clean":"2024-03-26T08:31:37.495687+0000","last_became_active":"2024-03-26T08:31:26.478218+0000","last_became_peered":"2024-03-26T08:31:26.478218+0000","last_unstale":"2024-03-26T08:31:37.495687+0000","last_undegraded":"2024-03-26T08:31:37.495687+0000","last_fullsized":"2024-03-26T08:31:37.495687+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:02:54.506805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.488379+0000","last_change":"2024-03-26T08:31:22.381931+0000","last_active":"2024-03-26T08:31:37.488379+0000","last_peered":"2024-03-26T08:31:37.488379+0000","last_clean":"2024-03-26T08:31:37.488379+0000","last_became_active":"2024-03-26T08:31:22.379021+0000","last_became_peered":"2024-03-26T08:31:22.379021+0000","last_unstale":"2024-03-26T08:31:37.488379+0000","last_undegraded":"2024-03-26T08:31:37.488379+0000","last_fullsized":"2024-03-26T08:31:37.488379+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:07:31.555154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.489823+0000","last_change":"2024-03-26T08:31:21.375753+0000","last_active":"2024-03-26T08:31:37.489823+0000","last_peered":"2024-03-26T08:31:37.489823+0000","last_clean":"2024-03-26T08:31:37.489823+0000","last_became_active":"2024-03-26T08:31:21.375340+0000","last_became_peered":"2024-03-26T08:31:21.375340+0000","last_unstale":"2024-03-26T08:31:37.489823+0000","last_undegraded":"2024-03-26T08:31:37.489823+0000","last_fullsized":"2024-03-26T08:31:37.489823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:21:14.241774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.488046+0000","last_change":"2024-03-26T08:31:26.454026+0000","last_active":"2024-03-26T08:31:37.488046+0000","last_peered":"2024-03-26T08:31:37.488046+0000","last_clean":"2024-03-26T08:31:37.488046+0000","last_became_active":"2024-03-26T08:31:26.453494+0000","last_became_peered":"2024-03-26T08:31:26.453494+0000","last_unstale":"2024-03-26T08:31:37.488046+0000","last_undegraded":"2024-03-26T08:31:37.488046+0000","last_fullsized":"2024-03-26T08:31:37.488046+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:30:10.388280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.483076+0000","last_change":"2024-03-26T08:31:21.358930+0000","last_active":"2024-03-26T08:31:37.483076+0000","last_peered":"2024-03-26T08:31:37.483076+0000","last_clean":"2024-03-26T08:31:37.483076+0000","last_became_active":"2024-03-26T08:31:21.357573+0000","last_became_peered":"2024-03-26T08:31:21.357573+0000","last_unstale":"2024-03-26T08:31:37.483076+0000","last_undegraded":"2024-03-26T08:31:37.483076+0000","last_fullsized":"2024-03-26T08:31:37.483076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:36:41.572993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.490376+0000","last_change":"2024-03-26T08:31:22.378520+0000","last_active":"2024-03-26T08:31:37.490376+0000","last_peered":"2024-03-26T08:31:37.490376+0000","last_clean":"2024-03-26T08:31:37.490376+0000","last_became_active":"2024-03-26T08:31:22.378081+0000","last_became_peered":"2024-03-26T08:31:22.378081+0000","last_unstale":"2024-03-26T08:31:37.490376+0000","last_undegraded":"2024-03-26T08:31:37.490376+0000","last_fullsized":"2024-03-26T08:31:37.490376+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:09:34.748182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.491619+0000","last_change":"2024-03-26T08:31:26.479545+0000","last_active":"2024-03-26T08:31:37.491619+0000","last_peered":"2024-03-26T08:31:37.491619+0000","last_clean":"2024-03-26T08:31:37.491619+0000","last_became_active":"2024-03-26T08:31:26.479029+0000","last_became_peered":"2024-03-26T08:31:26.479029+0000","last_unstale":"2024-03-26T08:31:37.491619+0000","last_undegraded":"2024-03-26T08:31:37.491619+0000","last_fullsized":"2024-03-26T08:31:37.491619+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:49:03.795752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.478175+0000","last_change":"2024-03-26T08:31:21.360510+0000","last_active":"2024-03-26T08:31:37.478175+0000","last_peered":"2024-03-26T08:31:37.478175+0000","last_clean":"2024-03-26T08:31:37.478175+0000","last_became_active":"2024-03-26T08:31:21.359593+0000","last_became_peered":"2024-03-26T08:31:21.359593+0000","last_unstale":"2024-03-26T08:31:37.478175+0000","last_undegraded":"2024-03-26T08:31:37.478175+0000","last_fullsized":"2024-03-26T08:31:37.478175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:37:33.752689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.496466+0000","last_change":"2024-03-26T08:31:22.390181+0000","last_active":"2024-03-26T08:31:37.496466+0000","last_peered":"2024-03-26T08:31:37.496466+0000","last_clean":"2024-03-26T08:31:37.496466+0000","last_became_active":"2024-03-26T08:31:22.389686+0000","last_became_peered":"2024-03-26T08:31:22.389686+0000","last_unstale":"2024-03-26T08:31:37.496466+0000","last_undegraded":"2024-03-26T08:31:37.496466+0000","last_fullsized":"2024-03-26T08:31:37.496466+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:48:09.913134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.617390+0000","last_change":"2024-03-26T08:31:26.449160+0000","last_active":"2024-03-26T08:31:37.617390+0000","last_peered":"2024-03-26T08:31:37.617390+0000","last_clean":"2024-03-26T08:31:37.617390+0000","last_became_active":"2024-03-26T08:31:26.448570+0000","last_became_peered":"2024-03-26T08:31:26.448570+0000","last_unstale":"2024-03-26T08:31:37.617390+0000","last_undegraded":"2024-03-26T08:31:37.617390+0000","last_fullsized":"2024-03-26T08:31:37.617390+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:05:21.258982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.482445+0000","last_change":"2024-03-26T08:31:21.365723+0000","last_active":"2024-03-26T08:31:37.482445+0000","last_peered":"2024-03-26T08:31:37.482445+0000","last_clean":"2024-03-26T08:31:37.482445+0000","last_became_active":"2024-03-26T08:31:21.364893+0000","last_became_peered":"2024-03-26T08:31:21.364893+0000","last_unstale":"2024-03-26T08:31:37.482445+0000","last_undegraded":"2024-03-26T08:31:37.482445+0000","last_fullsized":"2024-03-26T08:31:37.482445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:24:17.210093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.484283+0000","last_change":"2024-03-26T08:31:22.396341+0000","last_active":"2024-03-26T08:31:37.484283+0000","last_peered":"2024-03-26T08:31:37.484283+0000","last_clean":"2024-03-26T08:31:37.484283+0000","last_became_active":"2024-03-26T08:31:22.395956+0000","last_became_peered":"2024-03-26T08:31:22.395956+0000","last_unstale":"2024-03-26T08:31:37.484283+0000","last_undegraded":"2024-03-26T08:31:37.484283+0000","last_fullsized":"2024-03-26T08:31:37.484283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:08:51.194370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.497971+0000","last_change":"2024-03-26T08:31:37.497971+0000","last_active":"2024-03-26T08:31:37.497971+0000","last_peered":"2024-03-26T08:31:37.497971+0000","last_clean":"2024-03-26T08:31:37.497971+0000","last_became_active":"2024-03-26T08:31:37.497379+0000","last_became_peered":"2024-03-26T08:31:37.497379+0000","last_unstale":"2024-03-26T08:31:37.497971+0000","last_undegraded":"2024-03-26T08:31:37.497971+0000","last_fullsized":"2024-03-26T08:31:37.497971+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:35:55.266756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.493616+0000","last_change":"2024-03-26T08:31:37.493616+0000","last_active":"2024-03-26T08:31:37.493616+0000","last_peered":"2024-03-26T08:31:37.493616+0000","last_clean":"2024-03-26T08:31:37.493616+0000","last_became_active":"2024-03-26T08:31:37.492288+0000","last_became_peered":"2024-03-26T08:31:37.492288+0000","last_unstale":"2024-03-26T08:31:37.493616+0000","last_undegraded":"2024-03-26T08:31:37.493616+0000","last_fullsized":"2024-03-26T08:31:37.493616+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:49:10.413751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.482550+0000","last_change":"2024-03-26T08:31:22.384347+0000","last_active":"2024-03-26T08:31:37.482550+0000","last_peered":"2024-03-26T08:31:37.482550+0000","last_clean":"2024-03-26T08:31:37.482550+0000","last_became_active":"2024-03-26T08:31:22.380639+0000","last_became_peered":"2024-03-26T08:31:22.380639+0000","last_unstale":"2024-03-26T08:31:37.482550+0000","last_undegraded":"2024-03-26T08:31:37.482550+0000","last_fullsized":"2024-03-26T08:31:37.482550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:39:07.778005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.476638+0000","last_change":"2024-03-26T08:31:26.455706+0000","last_active":"2024-03-26T08:31:37.476638+0000","last_peered":"2024-03-26T08:31:37.476638+0000","last_clean":"2024-03-26T08:31:37.476638+0000","last_became_active":"2024-03-26T08:31:26.454591+0000","last_became_peered":"2024-03-26T08:31:26.454591+0000","last_unstale":"2024-03-26T08:31:37.476638+0000","last_undegraded":"2024-03-26T08:31:37.476638+0000","last_fullsized":"2024-03-26T08:31:37.476638+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:07:30.612220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.492584+0000","last_change":"2024-03-26T08:31:21.357734+0000","last_active":"2024-03-26T08:31:37.492584+0000","last_peered":"2024-03-26T08:31:37.492584+0000","last_clean":"2024-03-26T08:31:37.492584+0000","last_became_active":"2024-03-26T08:31:21.357361+0000","last_became_peered":"2024-03-26T08:31:21.357361+0000","last_unstale":"2024-03-26T08:31:37.492584+0000","last_undegraded":"2024-03-26T08:31:37.492584+0000","last_fullsized":"2024-03-26T08:31:37.492584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:26:57.454577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.488392+0000","last_change":"2024-03-26T08:31:22.383118+0000","last_active":"2024-03-26T08:31:37.488392+0000","last_peered":"2024-03-26T08:31:37.488392+0000","last_clean":"2024-03-26T08:31:37.488392+0000","last_became_active":"2024-03-26T08:31:22.382773+0000","last_became_peered":"2024-03-26T08:31:22.382773+0000","last_unstale":"2024-03-26T08:31:37.488392+0000","last_undegraded":"2024-03-26T08:31:37.488392+0000","last_fullsized":"2024-03-26T08:31:37.488392+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:50:34.889914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.478266+0000","last_change":"2024-03-26T08:31:26.451467+0000","last_active":"2024-03-26T08:31:37.478266+0000","last_peered":"2024-03-26T08:31:37.478266+0000","last_clean":"2024-03-26T08:31:37.478266+0000","last_became_active":"2024-03-26T08:31:26.451022+0000","last_became_peered":"2024-03-26T08:31:26.451022+0000","last_unstale":"2024-03-26T08:31:37.478266+0000","last_undegraded":"2024-03-26T08:31:37.478266+0000","last_fullsized":"2024-03-26T08:31:37.478266+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:19:50.442779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.617859+0000","last_change":"2024-03-26T08:31:21.361596+0000","last_active":"2024-03-26T08:31:37.617859+0000","last_peered":"2024-03-26T08:31:37.617859+0000","last_clean":"2024-03-26T08:31:37.617859+0000","last_became_active":"2024-03-26T08:31:21.357502+0000","last_became_peered":"2024-03-26T08:31:21.357502+0000","last_unstale":"2024-03-26T08:31:37.617859+0000","last_undegraded":"2024-03-26T08:31:37.617859+0000","last_fullsized":"2024-03-26T08:31:37.617859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:05:33.951346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.496907+0000","last_change":"2024-03-26T08:31:22.397344+0000","last_active":"2024-03-26T08:31:37.496907+0000","last_peered":"2024-03-26T08:31:37.496907+0000","last_clean":"2024-03-26T08:31:37.496907+0000","last_became_active":"2024-03-26T08:31:22.396936+0000","last_became_peered":"2024-03-26T08:31:22.396936+0000","last_unstale":"2024-03-26T08:31:37.496907+0000","last_undegraded":"2024-03-26T08:31:37.496907+0000","last_fullsized":"2024-03-26T08:31:37.496907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:07:49.412098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.479959+0000","last_change":"2024-03-26T08:31:26.479060+0000","last_active":"2024-03-26T08:31:37.479959+0000","last_peered":"2024-03-26T08:31:37.479959+0000","last_clean":"2024-03-26T08:31:37.479959+0000","last_became_active":"2024-03-26T08:31:26.478519+0000","last_became_peered":"2024-03-26T08:31:26.478519+0000","last_unstale":"2024-03-26T08:31:37.479959+0000","last_undegraded":"2024-03-26T08:31:37.479959+0000","last_fullsized":"2024-03-26T08:31:37.479959+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:37:20.528845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.482635+0000","last_change":"2024-03-26T08:31:21.374128+0000","last_active":"2024-03-26T08:31:37.482635+0000","last_peered":"2024-03-26T08:31:37.482635+0000","last_clean":"2024-03-26T08:31:37.482635+0000","last_became_active":"2024-03-26T08:31:21.372934+0000","last_became_peered":"2024-03-26T08:31:21.372934+0000","last_unstale":"2024-03-26T08:31:37.482635+0000","last_undegraded":"2024-03-26T08:31:37.482635+0000","last_fullsized":"2024-03-26T08:31:37.482635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:25:15.919855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.491242+0000","last_change":"2024-03-26T08:31:22.381847+0000","last_active":"2024-03-26T08:31:37.491242+0000","last_peered":"2024-03-26T08:31:37.491242+0000","last_clean":"2024-03-26T08:31:37.491242+0000","last_became_active":"2024-03-26T08:31:22.379956+0000","last_became_peered":"2024-03-26T08:31:22.379956+0000","last_unstale":"2024-03-26T08:31:37.491242+0000","last_undegraded":"2024-03-26T08:31:37.491242+0000","last_fullsized":"2024-03-26T08:31:37.491242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:16:38.048495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.477598+0000","last_change":"2024-03-26T08:31:26.455503+0000","last_active":"2024-03-26T08:31:37.477598+0000","last_peered":"2024-03-26T08:31:37.477598+0000","last_clean":"2024-03-26T08:31:37.477598+0000","last_became_active":"2024-03-26T08:31:26.455023+0000","last_became_peered":"2024-03-26T08:31:26.455023+0000","last_unstale":"2024-03-26T08:31:37.477598+0000","last_undegraded":"2024-03-26T08:31:37.477598+0000","last_fullsized":"2024-03-26T08:31:37.477598+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:06:47.565407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.618849+0000","last_change":"2024-03-26T08:31:21.362832+0000","last_active":"2024-03-26T08:31:37.618849+0000","last_peered":"2024-03-26T08:31:37.618849+0000","last_clean":"2024-03-26T08:31:37.618849+0000","last_became_active":"2024-03-26T08:31:21.362411+0000","last_became_peered":"2024-03-26T08:31:21.362411+0000","last_unstale":"2024-03-26T08:31:37.618849+0000","last_undegraded":"2024-03-26T08:31:37.618849+0000","last_fullsized":"2024-03-26T08:31:37.618849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:34:07.646011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.494839+0000","last_change":"2024-03-26T08:31:22.403106+0000","last_active":"2024-03-26T08:31:37.494839+0000","last_peered":"2024-03-26T08:31:37.494839+0000","last_clean":"2024-03-26T08:31:37.494839+0000","last_became_active":"2024-03-26T08:31:22.401172+0000","last_became_peered":"2024-03-26T08:31:22.401172+0000","last_unstale":"2024-03-26T08:31:37.494839+0000","last_undegraded":"2024-03-26T08:31:37.494839+0000","last_fullsized":"2024-03-26T08:31:37.494839+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:31:42.654817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.495759+0000","last_change":"2024-03-26T08:31:37.495759+0000","last_active":"2024-03-26T08:31:37.495759+0000","last_peered":"2024-03-26T08:31:37.495759+0000","last_clean":"2024-03-26T08:31:37.495759+0000","last_became_active":"2024-03-26T08:31:37.493610+0000","last_became_peered":"2024-03-26T08:31:37.493610+0000","last_unstale":"2024-03-26T08:31:37.495759+0000","last_undegraded":"2024-03-26T08:31:37.495759+0000","last_fullsized":"2024-03-26T08:31:37.495759+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:17:05.345033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.496272+0000","last_change":"2024-03-26T08:31:21.369396+0000","last_active":"2024-03-26T08:31:37.496272+0000","last_peered":"2024-03-26T08:31:37.496272+0000","last_clean":"2024-03-26T08:31:37.496272+0000","last_became_active":"2024-03-26T08:31:21.364011+0000","last_became_peered":"2024-03-26T08:31:21.364011+0000","last_unstale":"2024-03-26T08:31:37.496272+0000","last_undegraded":"2024-03-26T08:31:37.496272+0000","last_fullsized":"2024-03-26T08:31:37.496272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:23:47.347158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.494166+0000","last_change":"2024-03-26T08:31:22.389586+0000","last_active":"2024-03-26T08:31:37.494166+0000","last_peered":"2024-03-26T08:31:37.494166+0000","last_clean":"2024-03-26T08:31:37.494166+0000","last_became_active":"2024-03-26T08:31:22.389200+0000","last_became_peered":"2024-03-26T08:31:22.389200+0000","last_unstale":"2024-03-26T08:31:37.494166+0000","last_undegraded":"2024-03-26T08:31:37.494166+0000","last_fullsized":"2024-03-26T08:31:37.494166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:54:30.076744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.494970+0000","last_change":"2024-03-26T08:31:26.452274+0000","last_active":"2024-03-26T08:31:37.494970+0000","last_peered":"2024-03-26T08:31:37.494970+0000","last_clean":"2024-03-26T08:31:37.494970+0000","last_became_active":"2024-03-26T08:31:26.451670+0000","last_became_peered":"2024-03-26T08:31:26.451670+0000","last_unstale":"2024-03-26T08:31:37.494970+0000","last_undegraded":"2024-03-26T08:31:37.494970+0000","last_fullsized":"2024-03-26T08:31:37.494970+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:48:36.028810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.490244+0000","last_change":"2024-03-26T08:31:21.356259+0000","last_active":"2024-03-26T08:31:37.490244+0000","last_peered":"2024-03-26T08:31:37.490244+0000","last_clean":"2024-03-26T08:31:37.490244+0000","last_became_active":"2024-03-26T08:31:21.355724+0000","last_became_peered":"2024-03-26T08:31:21.355724+0000","last_unstale":"2024-03-26T08:31:37.490244+0000","last_undegraded":"2024-03-26T08:31:37.490244+0000","last_fullsized":"2024-03-26T08:31:37.490244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:30:05.621417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.495928+0000","last_change":"2024-03-26T08:31:22.370869+0000","last_active":"2024-03-26T08:31:37.495928+0000","last_peered":"2024-03-26T08:31:37.495928+0000","last_clean":"2024-03-26T08:31:37.495928+0000","last_became_active":"2024-03-26T08:31:22.369797+0000","last_became_peered":"2024-03-26T08:31:22.369797+0000","last_unstale":"2024-03-26T08:31:37.495928+0000","last_undegraded":"2024-03-26T08:31:37.495928+0000","last_fullsized":"2024-03-26T08:31:37.495928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:59:33.064619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.488869+0000","last_change":"2024-03-26T08:31:22.377927+0000","last_active":"2024-03-26T08:31:37.488869+0000","last_peered":"2024-03-26T08:31:37.488869+0000","last_clean":"2024-03-26T08:31:37.488869+0000","last_became_active":"2024-03-26T08:31:22.377559+0000","last_became_peered":"2024-03-26T08:31:22.377559+0000","last_unstale":"2024-03-26T08:31:37.488869+0000","last_undegraded":"2024-03-26T08:31:37.488869+0000","last_fullsized":"2024-03-26T08:31:37.488869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:06:05.544179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.617757+0000","last_change":"2024-03-26T08:31:22.380287+0000","last_active":"2024-03-26T08:31:37.617757+0000","last_peered":"2024-03-26T08:31:37.617757+0000","last_clean":"2024-03-26T08:31:37.617757+0000","last_became_active":"2024-03-26T08:31:22.379948+0000","last_became_peered":"2024-03-26T08:31:22.379948+0000","last_unstale":"2024-03-26T08:31:37.617757+0000","last_undegraded":"2024-03-26T08:31:37.617757+0000","last_fullsized":"2024-03-26T08:31:37.617757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:57:12.636151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.617508+0000","last_change":"2024-03-26T08:31:22.390109+0000","last_active":"2024-03-26T08:31:37.617508+0000","last_peered":"2024-03-26T08:31:37.617508+0000","last_clean":"2024-03-26T08:31:37.617508+0000","last_became_active":"2024-03-26T08:31:22.389766+0000","last_became_peered":"2024-03-26T08:31:22.389766+0000","last_unstale":"2024-03-26T08:31:37.617508+0000","last_undegraded":"2024-03-26T08:31:37.617508+0000","last_fullsized":"2024-03-26T08:31:37.617508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:24:27.454598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.485963+0000","last_change":"2024-03-26T08:31:22.391882+0000","last_active":"2024-03-26T08:31:37.485963+0000","last_peered":"2024-03-26T08:31:37.485963+0000","last_clean":"2024-03-26T08:31:37.485963+0000","last_became_active":"2024-03-26T08:31:22.391395+0000","last_became_peered":"2024-03-26T08:31:22.391395+0000","last_unstale":"2024-03-26T08:31:37.485963+0000","last_undegraded":"2024-03-26T08:31:37.485963+0000","last_fullsized":"2024-03-26T08:31:37.485963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:45:06.685287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.489387+0000","last_change":"2024-03-26T08:31:22.385993+0000","last_active":"2024-03-26T08:31:37.489387+0000","last_peered":"2024-03-26T08:31:37.489387+0000","last_clean":"2024-03-26T08:31:37.489387+0000","last_became_active":"2024-03-26T08:31:22.385625+0000","last_became_peered":"2024-03-26T08:31:22.385625+0000","last_unstale":"2024-03-26T08:31:37.489387+0000","last_undegraded":"2024-03-26T08:31:37.489387+0000","last_fullsized":"2024-03-26T08:31:37.489387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:16:20.731718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.490485+0000","last_change":"2024-03-26T08:31:22.379588+0000","last_active":"2024-03-26T08:31:37.490485+0000","last_peered":"2024-03-26T08:31:37.490485+0000","last_clean":"2024-03-26T08:31:37.490485+0000","last_became_active":"2024-03-26T08:31:22.379215+0000","last_became_peered":"2024-03-26T08:31:22.379215+0000","last_unstale":"2024-03-26T08:31:37.490485+0000","last_undegraded":"2024-03-26T08:31:37.490485+0000","last_fullsized":"2024-03-26T08:31:37.490485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:23:20.339737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.618737+0000","last_change":"2024-03-26T08:31:22.404354+0000","last_active":"2024-03-26T08:31:37.618737+0000","last_peered":"2024-03-26T08:31:37.618737+0000","last_clean":"2024-03-26T08:31:37.618737+0000","last_became_active":"2024-03-26T08:31:22.403973+0000","last_became_peered":"2024-03-26T08:31:22.403973+0000","last_unstale":"2024-03-26T08:31:37.618737+0000","last_undegraded":"2024-03-26T08:31:37.618737+0000","last_fullsized":"2024-03-26T08:31:37.618737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:36:54.360147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.496860+0000","last_change":"2024-03-26T08:31:22.379642+0000","last_active":"2024-03-26T08:31:37.496860+0000","last_peered":"2024-03-26T08:31:37.496860+0000","last_clean":"2024-03-26T08:31:37.496860+0000","last_became_active":"2024-03-26T08:31:22.379241+0000","last_became_peered":"2024-03-26T08:31:22.379241+0000","last_unstale":"2024-03-26T08:31:37.496860+0000","last_undegraded":"2024-03-26T08:31:37.496860+0000","last_fullsized":"2024-03-26T08:31:37.496860+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:21:51.117616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.496330+0000","last_change":"2024-03-26T08:31:22.378809+0000","last_active":"2024-03-26T08:31:37.496330+0000","last_peered":"2024-03-26T08:31:37.496330+0000","last_clean":"2024-03-26T08:31:37.496330+0000","last_became_active":"2024-03-26T08:31:22.378388+0000","last_became_peered":"2024-03-26T08:31:22.378388+0000","last_unstale":"2024-03-26T08:31:37.496330+0000","last_undegraded":"2024-03-26T08:31:37.496330+0000","last_fullsized":"2024-03-26T08:31:37.496330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:33:21.583423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.492875+0000","last_change":"2024-03-26T08:31:22.389230+0000","last_active":"2024-03-26T08:31:37.492875+0000","last_peered":"2024-03-26T08:31:37.492875+0000","last_clean":"2024-03-26T08:31:37.492875+0000","last_became_active":"2024-03-26T08:31:22.388870+0000","last_became_peered":"2024-03-26T08:31:22.388870+0000","last_unstale":"2024-03-26T08:31:37.492875+0000","last_undegraded":"2024-03-26T08:31:37.492875+0000","last_fullsized":"2024-03-26T08:31:37.492875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:40:20.377122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.478732+0000","last_change":"2024-03-26T08:31:22.396232+0000","last_active":"2024-03-26T08:31:37.478732+0000","last_peered":"2024-03-26T08:31:37.478732+0000","last_clean":"2024-03-26T08:31:37.478732+0000","last_became_active":"2024-03-26T08:31:22.392940+0000","last_became_peered":"2024-03-26T08:31:22.392940+0000","last_unstale":"2024-03-26T08:31:37.478732+0000","last_undegraded":"2024-03-26T08:31:37.478732+0000","last_fullsized":"2024-03-26T08:31:37.478732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:01:15.460526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.490207+0000","last_change":"2024-03-26T08:31:22.370880+0000","last_active":"2024-03-26T08:31:37.490207+0000","last_peered":"2024-03-26T08:31:37.490207+0000","last_clean":"2024-03-26T08:31:37.490207+0000","last_became_active":"2024-03-26T08:31:22.370546+0000","last_became_peered":"2024-03-26T08:31:22.370546+0000","last_unstale":"2024-03-26T08:31:37.490207+0000","last_undegraded":"2024-03-26T08:31:37.490207+0000","last_fullsized":"2024-03-26T08:31:37.490207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:46:11.657763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.488781+0000","last_change":"2024-03-26T08:31:22.387573+0000","last_active":"2024-03-26T08:31:37.488781+0000","last_peered":"2024-03-26T08:31:37.488781+0000","last_clean":"2024-03-26T08:31:37.488781+0000","last_became_active":"2024-03-26T08:31:22.387142+0000","last_became_peered":"2024-03-26T08:31:22.387142+0000","last_unstale":"2024-03-26T08:31:37.488781+0000","last_undegraded":"2024-03-26T08:31:37.488781+0000","last_fullsized":"2024-03-26T08:31:37.488781+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:13:32.307195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281284,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":23939,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":15,"num_read_kb":21,"num_write":58,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":39819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350998,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":503251,"data_compressed":20702,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612637,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1093632,"data_stored":1116932,"data_compressed":56479,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776165,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":499715,"data_compressed":20702,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037804,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":499715,"data_compressed":20702,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299442,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":1090083,"data_compressed":56479,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462969,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":499133,"data_compressed":20702,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":502975,"data_compressed":20702,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":499223,"data_compressed":20702,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":503009,"data_compressed":20702,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149775,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":499237,"data_compressed":20702,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888135,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":499155,"data_compressed":20702,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724608,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1056768,"data_stored":1089683,"data_compressed":56479,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34133,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34133,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34133,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35777,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35777,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35777,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35777,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35777,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":10442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":23335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T08:31:52.833 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T08:31:52.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:52 smithi043 ceph-mon[31929]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:31:52.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:52 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/987212315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:52.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:52 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1920952881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:31:53.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:52 smithi183 ceph-mon[35818]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:31:53.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:52 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/987212315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:53.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:52 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1920952881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:31:53.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:52 smithi052 ceph-mon[35071]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:31:53.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:52 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/987212315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:31:53.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:52 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1920952881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:31:53.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.106+0000 7fa7ed600640 1 -- 172.21.15.43:0/56589229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e81036f0 msgr2=0x7fa7e8103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:53.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.106+0000 7fa7ed600640 1 --2- 172.21.15.43:0/56589229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e81036f0 0x7fa7e8103b10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8009900 tx=0x7fa7d802bc30 comp rx=0 tx=0).stop 2024-03-26T08:31:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.106+0000 7fa7ed600640 1 -- 172.21.15.43:0/56589229 shutdown_connections 2024-03-26T08:31:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.106+0000 7fa7ed600640 1 --2- 172.21.15.43:0/56589229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7e81052a0 0x7fa7e810bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.106+0000 7fa7ed600640 1 --2- 172.21.15.43:0/56589229 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7e81048e0 0x7fa7e8104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.106+0000 7fa7ed600640 1 --2- 172.21.15.43:0/56589229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e81036f0 0x7fa7e8103b10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.106+0000 7fa7ed600640 1 -- 172.21.15.43:0/56589229 >> 172.21.15.43:0/56589229 conn(0x7fa7e80fec80 msgr2=0x7fa7e81010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.107+0000 7fa7ed600640 1 -- 172.21.15.43:0/56589229 shutdown_connections 2024-03-26T08:31:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.107+0000 7fa7ed600640 1 -- 172.21.15.43:0/56589229 wait complete. 2024-03-26T08:31:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.107+0000 7fa7ed600640 1 Processor -- start 2024-03-26T08:31:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.108+0000 7fa7ed600640 1 -- start start 2024-03-26T08:31:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.108+0000 7fa7ed600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7e81036f0 0x7fa7e8199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.108+0000 7fa7ed600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7e81048e0 0x7fa7e8199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.108+0000 7fa7ed600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e81052a0 0x7fa7e81a0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.108+0000 7fa7ed600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e810f290 con 0x7fa7e81052a0 2024-03-26T08:31:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.108+0000 7fa7ed600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e810f110 con 0x7fa7e81036f0 2024-03-26T08:31:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.108+0000 7fa7ed600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e810f410 con 0x7fa7e81048e0 2024-03-26T08:31:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.108+0000 7fa7ecc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e81052a0 0x7fa7e81a0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.108+0000 7fa7ecc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e81052a0 0x7fa7e81a0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57520/0 (socket says 172.21.15.43:57520) 2024-03-26T08:31:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.108+0000 7fa7ecc00640 1 -- 172.21.15.43:0/2266154235 learned_addr learned my addr 172.21.15.43:0/2266154235 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.109+0000 7fa7e7e00640 1 --2- 172.21.15.43:0/2266154235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7e81036f0 0x7fa7e8199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.109+0000 7fa7df400640 1 --2- 172.21.15.43:0/2266154235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7e81048e0 0x7fa7e8199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.109+0000 7fa7ecc00640 1 -- 172.21.15.43:0/2266154235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7e81048e0 msgr2=0x7fa7e8199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.109+0000 7fa7ecc00640 1 --2- 172.21.15.43:0/2266154235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7e81048e0 0x7fa7e8199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.109+0000 7fa7ecc00640 1 -- 172.21.15.43:0/2266154235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7e81036f0 msgr2=0x7fa7e8199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.109+0000 7fa7ecc00640 1 --2- 172.21.15.43:0/2266154235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7e81036f0 0x7fa7e8199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.109+0000 7fa7ecc00640 1 -- 172.21.15.43:0/2266154235 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e81a1590 con 0x7fa7e81052a0 2024-03-26T08:31:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.109+0000 7fa7ecc00640 1 --2- 172.21.15.43:0/2266154235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e81052a0 0x7fa7e81a0f20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000b970 tx=0x7fa7e000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.110+0000 7fa7e5600640 1 -- 172.21.15.43:0/2266154235 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0019070 con 0x7fa7e81052a0 2024-03-26T08:31:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.110+0000 7fa7e5600640 1 -- 172.21.15.43:0/2266154235 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa7e0014440 con 0x7fa7e81052a0 2024-03-26T08:31:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.110+0000 7fa7e5600640 1 -- 172.21.15.43:0/2266154235 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e00075d0 con 0x7fa7e81052a0 2024-03-26T08:31:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.110+0000 7fa7ed600640 1 -- 172.21.15.43:0/2266154235 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e81a1880 con 0x7fa7e81052a0 2024-03-26T08:31:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.110+0000 7fa7ed600640 1 -- 172.21.15.43:0/2266154235 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e81a1ea0 con 0x7fa7e81052a0 2024-03-26T08:31:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.111+0000 7fa7ed600640 1 -- 172.21.15.43:0/2266154235 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e810aa90 con 0x7fa7e81052a0 2024-03-26T08:31:53.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.113+0000 7fa7e5600640 1 -- 172.21.15.43:0/2266154235 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fa7e001c070 con 0x7fa7e81052a0 2024-03-26T08:31:53.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.114+0000 7fa7e5600640 1 --2- 172.21.15.43:0/2266154235 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa7cc0a2910 0x7fa7cc0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:53.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.114+0000 7fa7e5600640 1 -- 172.21.15.43:0/2266154235 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7fa7e00ce360 con 0x7fa7e81052a0 2024-03-26T08:31:53.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.114+0000 7fa7e7e00640 1 --2- 172.21.15.43:0/2266154235 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa7cc0a2910 0x7fa7cc0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:53.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.115+0000 7fa7e7e00640 1 --2- 172.21.15.43:0/2266154235 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa7cc0a2910 0x7fa7cc0a4dd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8009230 tx=0x7fa7d8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:53.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.119+0000 7fa7e5600640 1 -- 172.21.15.43:0/2266154235 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa7e009a9c0 con 0x7fa7e81052a0 2024-03-26T08:31:53.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.320+0000 7fa7ed600640 1 -- 172.21.15.43:0/2266154235 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa7e8061ae0 con 0x7fa7cc0a2910 2024-03-26T08:31:53.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.350+0000 7fa7e5600640 1 -- 172.21.15.43:0/2266154235 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340575 (secure 0 0 0) 0x7fa7e8061ae0 con 0x7fa7cc0a2910 2024-03-26T08:31:53.345 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:31:53.350 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-03-26T08:31:53.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.358+0000 7fa7ed600640 1 -- 172.21.15.43:0/2266154235 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa7cc0a2910 msgr2=0x7fa7cc0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:53.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.358+0000 7fa7ed600640 1 --2- 172.21.15.43:0/2266154235 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa7cc0a2910 0x7fa7cc0a4dd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8009230 tx=0x7fa7d8005fd0 comp rx=0 tx=0).stop 2024-03-26T08:31:53.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.358+0000 7fa7ed600640 1 -- 172.21.15.43:0/2266154235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e81052a0 msgr2=0x7fa7e81a0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:53.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.359+0000 7fa7ed600640 1 --2- 172.21.15.43:0/2266154235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e81052a0 0x7fa7e81a0f20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000b970 tx=0x7fa7e000be30 comp rx=0 tx=0).stop 2024-03-26T08:31:53.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.359+0000 7fa7ed600640 1 -- 172.21.15.43:0/2266154235 shutdown_connections 2024-03-26T08:31:53.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.359+0000 7fa7ed600640 1 --2- 172.21.15.43:0/2266154235 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa7cc0a2910 0x7fa7cc0a4dd0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:53.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.359+0000 7fa7ed600640 1 --2- 172.21.15.43:0/2266154235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e81052a0 0x7fa7e81a0f20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:53.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.359+0000 7fa7ed600640 1 --2- 172.21.15.43:0/2266154235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7e81048e0 0x7fa7e8199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:53.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.359+0000 7fa7ed600640 1 --2- 172.21.15.43:0/2266154235 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7e81036f0 0x7fa7e8199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:53.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.359+0000 7fa7ed600640 1 -- 172.21.15.43:0/2266154235 >> 172.21.15.43:0/2266154235 conn(0x7fa7e80fec80 msgr2=0x7fa7e8107c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:53.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.359+0000 7fa7ed600640 1 -- 172.21.15.43:0/2266154235 shutdown_connections 2024-03-26T08:31:53.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:53.359+0000 7fa7ed600640 1 -- 172.21.15.43:0/2266154235 wait complete. 2024-03-26T08:31:53.374 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":390,"stamp":"2024-03-26T08:31:52.339305+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":614307,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":141,"num_read_kb":130,"num_write":239,"num_write_kb":2980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":188,"ondisk_log_size":188,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369472,"kb_used_data":11932,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572352,"statfs":{"total":1151940427776,"available":1151562088448,"internally_reserved":0,"allocated":12218368,"data_stored":7802111,"data_compressed":355755,"data_compressed_allocated":479232,"data_compressed_original":2457600,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19440,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":21,"num_write":10,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005196"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.489348+0000","last_change":"2024-03-26T08:31:22.394256+0000","last_active":"2024-03-26T08:31:37.489348+0000","last_peered":"2024-03-26T08:31:37.489348+0000","last_clean":"2024-03-26T08:31:37.489348+0000","last_became_active":"2024-03-26T08:31:22.393947+0000","last_became_peered":"2024-03-26T08:31:22.393947+0000","last_unstale":"2024-03-26T08:31:37.489348+0000","last_undegraded":"2024-03-26T08:31:37.489348+0000","last_fullsized":"2024-03-26T08:31:37.489348+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:00:50.769853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.484219+0000","last_change":"2024-03-26T08:31:22.387549+0000","last_active":"2024-03-26T08:31:37.484219+0000","last_peered":"2024-03-26T08:31:37.484219+0000","last_clean":"2024-03-26T08:31:37.484219+0000","last_became_active":"2024-03-26T08:31:22.387127+0000","last_became_peered":"2024-03-26T08:31:22.387127+0000","last_unstale":"2024-03-26T08:31:37.484219+0000","last_undegraded":"2024-03-26T08:31:37.484219+0000","last_fullsized":"2024-03-26T08:31:37.484219+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:52:36.156193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.484048+0000","last_change":"2024-03-26T08:31:22.404275+0000","last_active":"2024-03-26T08:31:37.484048+0000","last_peered":"2024-03-26T08:31:37.484048+0000","last_clean":"2024-03-26T08:31:37.484048+0000","last_became_active":"2024-03-26T08:31:22.403909+0000","last_became_peered":"2024-03-26T08:31:22.403909+0000","last_unstale":"2024-03-26T08:31:37.484048+0000","last_undegraded":"2024-03-26T08:31:37.484048+0000","last_fullsized":"2024-03-26T08:31:37.484048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:55:56.295719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.481567+0000","last_change":"2024-03-26T08:31:22.381310+0000","last_active":"2024-03-26T08:31:37.481567+0000","last_peered":"2024-03-26T08:31:37.481567+0000","last_clean":"2024-03-26T08:31:37.481567+0000","last_became_active":"2024-03-26T08:31:22.380721+0000","last_became_peered":"2024-03-26T08:31:22.380721+0000","last_unstale":"2024-03-26T08:31:37.481567+0000","last_undegraded":"2024-03-26T08:31:37.481567+0000","last_fullsized":"2024-03-26T08:31:37.481567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:42:30.325409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.482566+0000","last_change":"2024-03-26T08:31:22.396906+0000","last_active":"2024-03-26T08:31:37.482566+0000","last_peered":"2024-03-26T08:31:37.482566+0000","last_clean":"2024-03-26T08:31:37.482566+0000","last_became_active":"2024-03-26T08:31:22.392815+0000","last_became_peered":"2024-03-26T08:31:22.392815+0000","last_unstale":"2024-03-26T08:31:37.482566+0000","last_undegraded":"2024-03-26T08:31:37.482566+0000","last_fullsized":"2024-03-26T08:31:37.482566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:04:25.761459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.489012+0000","last_change":"2024-03-26T08:31:22.389374+0000","last_active":"2024-03-26T08:31:37.489012+0000","last_peered":"2024-03-26T08:31:37.489012+0000","last_clean":"2024-03-26T08:31:37.489012+0000","last_became_active":"2024-03-26T08:31:22.389012+0000","last_became_peered":"2024-03-26T08:31:22.389012+0000","last_unstale":"2024-03-26T08:31:37.489012+0000","last_undegraded":"2024-03-26T08:31:37.489012+0000","last_fullsized":"2024-03-26T08:31:37.489012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:23:35.696335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.488660+0000","last_change":"2024-03-26T08:31:22.378805+0000","last_active":"2024-03-26T08:31:37.488660+0000","last_peered":"2024-03-26T08:31:37.488660+0000","last_clean":"2024-03-26T08:31:37.488660+0000","last_became_active":"2024-03-26T08:31:22.378313+0000","last_became_peered":"2024-03-26T08:31:22.378313+0000","last_unstale":"2024-03-26T08:31:37.488660+0000","last_undegraded":"2024-03-26T08:31:37.488660+0000","last_fullsized":"2024-03-26T08:31:37.488660+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:10:03.816988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.497874+0000","last_change":"2024-03-26T08:31:37.495699+0000","last_active":"2024-03-26T08:31:37.497874+0000","last_peered":"2024-03-26T08:31:37.497874+0000","last_clean":"2024-03-26T08:31:37.497874+0000","last_became_active":"2024-03-26T08:31:37.494458+0000","last_became_peered":"2024-03-26T08:31:37.494458+0000","last_unstale":"2024-03-26T08:31:37.497874+0000","last_undegraded":"2024-03-26T08:31:37.497874+0000","last_fullsized":"2024-03-26T08:31:37.497874+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:32:36.973898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,10],"acting":[9,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.479928+0000","last_change":"2024-03-26T08:31:22.383958+0000","last_active":"2024-03-26T08:31:37.479928+0000","last_peered":"2024-03-26T08:31:37.479928+0000","last_clean":"2024-03-26T08:31:37.479928+0000","last_became_active":"2024-03-26T08:31:22.383167+0000","last_became_peered":"2024-03-26T08:31:22.383167+0000","last_unstale":"2024-03-26T08:31:37.479928+0000","last_undegraded":"2024-03-26T08:31:37.479928+0000","last_fullsized":"2024-03-26T08:31:37.479928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:38:24.265054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.493950+0000","last_change":"2024-03-26T08:31:22.394564+0000","last_active":"2024-03-26T08:31:37.493950+0000","last_peered":"2024-03-26T08:31:37.493950+0000","last_clean":"2024-03-26T08:31:37.493950+0000","last_became_active":"2024-03-26T08:31:22.394196+0000","last_became_peered":"2024-03-26T08:31:22.394196+0000","last_unstale":"2024-03-26T08:31:37.493950+0000","last_undegraded":"2024-03-26T08:31:37.493950+0000","last_fullsized":"2024-03-26T08:31:37.493950+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:35:47.429246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.494708+0000","last_change":"2024-03-26T08:31:22.388214+0000","last_active":"2024-03-26T08:31:37.494708+0000","last_peered":"2024-03-26T08:31:37.494708+0000","last_clean":"2024-03-26T08:31:37.494708+0000","last_became_active":"2024-03-26T08:31:22.387760+0000","last_became_peered":"2024-03-26T08:31:22.387760+0000","last_unstale":"2024-03-26T08:31:37.494708+0000","last_undegraded":"2024-03-26T08:31:37.494708+0000","last_fullsized":"2024-03-26T08:31:37.494708+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:25:22.498259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.481292+0000","last_change":"2024-03-26T08:31:22.390892+0000","last_active":"2024-03-26T08:31:37.481292+0000","last_peered":"2024-03-26T08:31:37.481292+0000","last_clean":"2024-03-26T08:31:37.481292+0000","last_became_active":"2024-03-26T08:31:22.390500+0000","last_became_peered":"2024-03-26T08:31:22.390500+0000","last_unstale":"2024-03-26T08:31:37.481292+0000","last_undegraded":"2024-03-26T08:31:37.481292+0000","last_fullsized":"2024-03-26T08:31:37.481292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:29:01.832399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.490525+0000","last_change":"2024-03-26T08:31:22.382442+0000","last_active":"2024-03-26T08:31:37.490525+0000","last_peered":"2024-03-26T08:31:37.490525+0000","last_clean":"2024-03-26T08:31:37.490525+0000","last_became_active":"2024-03-26T08:31:22.380346+0000","last_became_peered":"2024-03-26T08:31:22.380346+0000","last_unstale":"2024-03-26T08:31:37.490525+0000","last_undegraded":"2024-03-26T08:31:37.490525+0000","last_fullsized":"2024-03-26T08:31:37.490525+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:36:36.451504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.483321+0000","last_change":"2024-03-26T08:31:22.387401+0000","last_active":"2024-03-26T08:31:37.483321+0000","last_peered":"2024-03-26T08:31:37.483321+0000","last_clean":"2024-03-26T08:31:37.483321+0000","last_became_active":"2024-03-26T08:31:22.387020+0000","last_became_peered":"2024-03-26T08:31:22.387020+0000","last_unstale":"2024-03-26T08:31:37.483321+0000","last_undegraded":"2024-03-26T08:31:37.483321+0000","last_fullsized":"2024-03-26T08:31:37.483321+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:50:15.073663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.483648+0000","last_change":"2024-03-26T08:31:22.397971+0000","last_active":"2024-03-26T08:31:37.483648+0000","last_peered":"2024-03-26T08:31:37.483648+0000","last_clean":"2024-03-26T08:31:37.483648+0000","last_became_active":"2024-03-26T08:31:22.395371+0000","last_became_peered":"2024-03-26T08:31:22.395371+0000","last_unstale":"2024-03-26T08:31:37.483648+0000","last_undegraded":"2024-03-26T08:31:37.483648+0000","last_fullsized":"2024-03-26T08:31:37.483648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:59:51.053712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.477150+0000","last_change":"2024-03-26T08:31:22.394437+0000","last_active":"2024-03-26T08:31:37.477150+0000","last_peered":"2024-03-26T08:31:37.477150+0000","last_clean":"2024-03-26T08:31:37.477150+0000","last_became_active":"2024-03-26T08:31:22.386794+0000","last_became_peered":"2024-03-26T08:31:22.386794+0000","last_unstale":"2024-03-26T08:31:37.477150+0000","last_undegraded":"2024-03-26T08:31:37.477150+0000","last_fullsized":"2024-03-26T08:31:37.477150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:56:01.686361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.488931+0000","last_change":"2024-03-26T08:31:22.397887+0000","last_active":"2024-03-26T08:31:37.488931+0000","last_peered":"2024-03-26T08:31:37.488931+0000","last_clean":"2024-03-26T08:31:37.488931+0000","last_became_active":"2024-03-26T08:31:22.397086+0000","last_became_peered":"2024-03-26T08:31:22.397086+0000","last_unstale":"2024-03-26T08:31:37.488931+0000","last_undegraded":"2024-03-26T08:31:37.488931+0000","last_fullsized":"2024-03-26T08:31:37.488931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:41:47.512904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.478384+0000","last_change":"2024-03-26T08:31:22.398868+0000","last_active":"2024-03-26T08:31:37.478384+0000","last_peered":"2024-03-26T08:31:37.478384+0000","last_clean":"2024-03-26T08:31:37.478384+0000","last_became_active":"2024-03-26T08:31:22.398536+0000","last_became_peered":"2024-03-26T08:31:22.398536+0000","last_unstale":"2024-03-26T08:31:37.478384+0000","last_undegraded":"2024-03-26T08:31:37.478384+0000","last_fullsized":"2024-03-26T08:31:37.478384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:59:07.931120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.619260+0000","last_change":"2024-03-26T08:31:22.389805+0000","last_active":"2024-03-26T08:31:37.619260+0000","last_peered":"2024-03-26T08:31:37.619260+0000","last_clean":"2024-03-26T08:31:37.619260+0000","last_became_active":"2024-03-26T08:31:22.389410+0000","last_became_peered":"2024-03-26T08:31:22.389410+0000","last_unstale":"2024-03-26T08:31:37.619260+0000","last_undegraded":"2024-03-26T08:31:37.619260+0000","last_fullsized":"2024-03-26T08:31:37.619260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:36:29.934029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.618816+0000","last_change":"2024-03-26T08:31:26.437909+0000","last_active":"2024-03-26T08:31:37.618816+0000","last_peered":"2024-03-26T08:31:37.618816+0000","last_clean":"2024-03-26T08:31:37.618816+0000","last_became_active":"2024-03-26T08:31:26.437337+0000","last_became_peered":"2024-03-26T08:31:26.437337+0000","last_unstale":"2024-03-26T08:31:37.618816+0000","last_undegraded":"2024-03-26T08:31:37.618816+0000","last_fullsized":"2024-03-26T08:31:37.618816+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:01:15.856194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.476962+0000","last_change":"2024-03-26T08:31:21.367133+0000","last_active":"2024-03-26T08:31:37.476962+0000","last_peered":"2024-03-26T08:31:37.476962+0000","last_clean":"2024-03-26T08:31:37.476962+0000","last_became_active":"2024-03-26T08:31:21.366206+0000","last_became_peered":"2024-03-26T08:31:21.366206+0000","last_unstale":"2024-03-26T08:31:37.476962+0000","last_undegraded":"2024-03-26T08:31:37.476962+0000","last_fullsized":"2024-03-26T08:31:37.476962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:11:16.614464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.497113+0000","last_change":"2024-03-26T08:31:22.385984+0000","last_active":"2024-03-26T08:31:37.497113+0000","last_peered":"2024-03-26T08:31:37.497113+0000","last_clean":"2024-03-26T08:31:37.497113+0000","last_became_active":"2024-03-26T08:31:22.385512+0000","last_became_peered":"2024-03-26T08:31:22.385512+0000","last_unstale":"2024-03-26T08:31:37.497113+0000","last_undegraded":"2024-03-26T08:31:37.497113+0000","last_fullsized":"2024-03-26T08:31:37.497113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:46:24.087975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.477452+0000","last_change":"2024-03-26T08:31:26.454001+0000","last_active":"2024-03-26T08:31:37.477452+0000","last_peered":"2024-03-26T08:31:37.477452+0000","last_clean":"2024-03-26T08:31:37.477452+0000","last_became_active":"2024-03-26T08:31:26.453474+0000","last_became_peered":"2024-03-26T08:31:26.453474+0000","last_unstale":"2024-03-26T08:31:37.477452+0000","last_undegraded":"2024-03-26T08:31:37.477452+0000","last_fullsized":"2024-03-26T08:31:37.477452+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:09:42.197500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.497998+0000","last_change":"2024-03-26T08:31:21.365012+0000","last_active":"2024-03-26T08:31:37.497998+0000","last_peered":"2024-03-26T08:31:37.497998+0000","last_clean":"2024-03-26T08:31:37.497998+0000","last_became_active":"2024-03-26T08:31:21.363981+0000","last_became_peered":"2024-03-26T08:31:21.363981+0000","last_unstale":"2024-03-26T08:31:37.497998+0000","last_undegraded":"2024-03-26T08:31:37.497998+0000","last_fullsized":"2024-03-26T08:31:37.497998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:12:36.581648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.483616+0000","last_change":"2024-03-26T08:31:22.387960+0000","last_active":"2024-03-26T08:31:37.483616+0000","last_peered":"2024-03-26T08:31:37.483616+0000","last_clean":"2024-03-26T08:31:37.483616+0000","last_became_active":"2024-03-26T08:31:22.387627+0000","last_became_peered":"2024-03-26T08:31:22.387627+0000","last_unstale":"2024-03-26T08:31:37.483616+0000","last_undegraded":"2024-03-26T08:31:37.483616+0000","last_fullsized":"2024-03-26T08:31:37.483616+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:20:09.265075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.485506+0000","last_change":"2024-03-26T08:31:26.450387+0000","last_active":"2024-03-26T08:31:37.485506+0000","last_peered":"2024-03-26T08:31:37.485506+0000","last_clean":"2024-03-26T08:31:37.485506+0000","last_became_active":"2024-03-26T08:31:26.449418+0000","last_became_peered":"2024-03-26T08:31:26.449418+0000","last_unstale":"2024-03-26T08:31:37.485506+0000","last_undegraded":"2024-03-26T08:31:37.485506+0000","last_fullsized":"2024-03-26T08:31:37.485506+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:51:59.022851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.483493+0000","last_change":"2024-03-26T08:31:21.373422+0000","last_active":"2024-03-26T08:31:37.483493+0000","last_peered":"2024-03-26T08:31:37.483493+0000","last_clean":"2024-03-26T08:31:37.483493+0000","last_became_active":"2024-03-26T08:31:21.372916+0000","last_became_peered":"2024-03-26T08:31:21.372916+0000","last_unstale":"2024-03-26T08:31:37.483493+0000","last_undegraded":"2024-03-26T08:31:37.483493+0000","last_fullsized":"2024-03-26T08:31:37.483493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:09:08.313365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.478286+0000","last_change":"2024-03-26T08:31:22.403385+0000","last_active":"2024-03-26T08:31:37.478286+0000","last_peered":"2024-03-26T08:31:37.478286+0000","last_clean":"2024-03-26T08:31:37.478286+0000","last_became_active":"2024-03-26T08:31:22.398021+0000","last_became_peered":"2024-03-26T08:31:22.398021+0000","last_unstale":"2024-03-26T08:31:37.478286+0000","last_undegraded":"2024-03-26T08:31:37.478286+0000","last_fullsized":"2024-03-26T08:31:37.478286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:56:06.691167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.479767+0000","last_change":"2024-03-26T08:31:26.449049+0000","last_active":"2024-03-26T08:31:37.479767+0000","last_peered":"2024-03-26T08:31:37.479767+0000","last_clean":"2024-03-26T08:31:37.479767+0000","last_became_active":"2024-03-26T08:31:26.448510+0000","last_became_peered":"2024-03-26T08:31:26.448510+0000","last_unstale":"2024-03-26T08:31:37.479767+0000","last_undegraded":"2024-03-26T08:31:37.479767+0000","last_fullsized":"2024-03-26T08:31:37.479767+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:24:58.699699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.482690+0000","last_change":"2024-03-26T08:31:21.367289+0000","last_active":"2024-03-26T08:31:37.482690+0000","last_peered":"2024-03-26T08:31:37.482690+0000","last_clean":"2024-03-26T08:31:37.482690+0000","last_became_active":"2024-03-26T08:31:21.366960+0000","last_became_peered":"2024-03-26T08:31:21.366960+0000","last_unstale":"2024-03-26T08:31:37.482690+0000","last_undegraded":"2024-03-26T08:31:37.482690+0000","last_fullsized":"2024-03-26T08:31:37.482690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:22:25.479383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.495835+0000","last_change":"2024-03-26T08:31:22.385815+0000","last_active":"2024-03-26T08:31:37.495835+0000","last_peered":"2024-03-26T08:31:37.495835+0000","last_clean":"2024-03-26T08:31:37.495835+0000","last_became_active":"2024-03-26T08:31:22.385503+0000","last_became_peered":"2024-03-26T08:31:22.385503+0000","last_unstale":"2024-03-26T08:31:37.495835+0000","last_undegraded":"2024-03-26T08:31:37.495835+0000","last_fullsized":"2024-03-26T08:31:37.495835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:33:35.768902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.618892+0000","last_change":"2024-03-26T08:31:26.449656+0000","last_active":"2024-03-26T08:31:37.618892+0000","last_peered":"2024-03-26T08:31:37.618892+0000","last_clean":"2024-03-26T08:31:37.618892+0000","last_became_active":"2024-03-26T08:31:26.448843+0000","last_became_peered":"2024-03-26T08:31:26.448843+0000","last_unstale":"2024-03-26T08:31:37.618892+0000","last_undegraded":"2024-03-26T08:31:37.618892+0000","last_fullsized":"2024-03-26T08:31:37.618892+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:11:28.135551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.499950+0000","last_change":"2024-03-26T08:31:21.373384+0000","last_active":"2024-03-26T08:31:37.499950+0000","last_peered":"2024-03-26T08:31:37.499950+0000","last_clean":"2024-03-26T08:31:37.499950+0000","last_became_active":"2024-03-26T08:31:21.371708+0000","last_became_peered":"2024-03-26T08:31:21.371708+0000","last_unstale":"2024-03-26T08:31:37.499950+0000","last_undegraded":"2024-03-26T08:31:37.499950+0000","last_fullsized":"2024-03-26T08:31:37.499950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:46:09.745738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.482429+0000","last_change":"2024-03-26T08:31:22.383990+0000","last_active":"2024-03-26T08:31:37.482429+0000","last_peered":"2024-03-26T08:31:37.482429+0000","last_clean":"2024-03-26T08:31:37.482429+0000","last_became_active":"2024-03-26T08:31:22.383580+0000","last_became_peered":"2024-03-26T08:31:22.383580+0000","last_unstale":"2024-03-26T08:31:37.482429+0000","last_undegraded":"2024-03-26T08:31:37.482429+0000","last_fullsized":"2024-03-26T08:31:37.482429+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:32:36.037407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.618204+0000","last_change":"2024-03-26T08:31:26.445710+0000","last_active":"2024-03-26T08:31:37.618204+0000","last_peered":"2024-03-26T08:31:37.618204+0000","last_clean":"2024-03-26T08:31:37.618204+0000","last_became_active":"2024-03-26T08:31:26.445189+0000","last_became_peered":"2024-03-26T08:31:26.445189+0000","last_unstale":"2024-03-26T08:31:37.618204+0000","last_undegraded":"2024-03-26T08:31:37.618204+0000","last_fullsized":"2024-03-26T08:31:37.618204+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:31:23.033192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.488809+0000","last_change":"2024-03-26T08:31:21.362175+0000","last_active":"2024-03-26T08:31:37.488809+0000","last_peered":"2024-03-26T08:31:37.488809+0000","last_clean":"2024-03-26T08:31:37.488809+0000","last_became_active":"2024-03-26T08:31:21.361805+0000","last_became_peered":"2024-03-26T08:31:21.361805+0000","last_unstale":"2024-03-26T08:31:37.488809+0000","last_undegraded":"2024-03-26T08:31:37.488809+0000","last_fullsized":"2024-03-26T08:31:37.488809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:38:26.537955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.500216+0000","last_change":"2024-03-26T08:31:22.380489+0000","last_active":"2024-03-26T08:31:37.500216+0000","last_peered":"2024-03-26T08:31:37.500216+0000","last_clean":"2024-03-26T08:31:37.500216+0000","last_became_active":"2024-03-26T08:31:22.380139+0000","last_became_peered":"2024-03-26T08:31:22.380139+0000","last_unstale":"2024-03-26T08:31:37.500216+0000","last_undegraded":"2024-03-26T08:31:37.500216+0000","last_fullsized":"2024-03-26T08:31:37.500216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:09:18.550790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.488626+0000","last_change":"2024-03-26T08:31:26.454535+0000","last_active":"2024-03-26T08:31:37.488626+0000","last_peered":"2024-03-26T08:31:37.488626+0000","last_clean":"2024-03-26T08:31:37.488626+0000","last_became_active":"2024-03-26T08:31:26.454106+0000","last_became_peered":"2024-03-26T08:31:26.454106+0000","last_unstale":"2024-03-26T08:31:37.488626+0000","last_undegraded":"2024-03-26T08:31:37.488626+0000","last_fullsized":"2024-03-26T08:31:37.488626+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:57:28.870653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.489004+0000","last_change":"2024-03-26T08:31:21.347236+0000","last_active":"2024-03-26T08:31:37.489004+0000","last_peered":"2024-03-26T08:31:37.489004+0000","last_clean":"2024-03-26T08:31:37.489004+0000","last_became_active":"2024-03-26T08:31:21.346770+0000","last_became_peered":"2024-03-26T08:31:21.346770+0000","last_unstale":"2024-03-26T08:31:37.489004+0000","last_undegraded":"2024-03-26T08:31:37.489004+0000","last_fullsized":"2024-03-26T08:31:37.489004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:17:22.927636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.491908+0000","last_change":"2024-03-26T08:31:22.390623+0000","last_active":"2024-03-26T08:31:37.491908+0000","last_peered":"2024-03-26T08:31:37.491908+0000","last_clean":"2024-03-26T08:31:37.491908+0000","last_became_active":"2024-03-26T08:31:22.389512+0000","last_became_peered":"2024-03-26T08:31:22.389512+0000","last_unstale":"2024-03-26T08:31:37.491908+0000","last_undegraded":"2024-03-26T08:31:37.491908+0000","last_fullsized":"2024-03-26T08:31:37.491908+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:04:31.317386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.483881+0000","last_change":"2024-03-26T08:31:26.445277+0000","last_active":"2024-03-26T08:31:37.483881+0000","last_peered":"2024-03-26T08:31:37.483881+0000","last_clean":"2024-03-26T08:31:37.483881+0000","last_became_active":"2024-03-26T08:31:26.443558+0000","last_became_peered":"2024-03-26T08:31:26.443558+0000","last_unstale":"2024-03-26T08:31:37.483881+0000","last_undegraded":"2024-03-26T08:31:37.483881+0000","last_fullsized":"2024-03-26T08:31:37.483881+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:20:16.688699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.496031+0000","last_change":"2024-03-26T08:31:21.371523+0000","last_active":"2024-03-26T08:31:37.496031+0000","last_peered":"2024-03-26T08:31:37.496031+0000","last_clean":"2024-03-26T08:31:37.496031+0000","last_became_active":"2024-03-26T08:31:21.371136+0000","last_became_peered":"2024-03-26T08:31:21.371136+0000","last_unstale":"2024-03-26T08:31:37.496031+0000","last_undegraded":"2024-03-26T08:31:37.496031+0000","last_fullsized":"2024-03-26T08:31:37.496031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:06:55.083579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.475639+0000","last_change":"2024-03-26T08:31:22.386237+0000","last_active":"2024-03-26T08:31:37.475639+0000","last_peered":"2024-03-26T08:31:37.475639+0000","last_clean":"2024-03-26T08:31:37.475639+0000","last_became_active":"2024-03-26T08:31:22.385874+0000","last_became_peered":"2024-03-26T08:31:22.385874+0000","last_unstale":"2024-03-26T08:31:37.475639+0000","last_undegraded":"2024-03-26T08:31:37.475639+0000","last_fullsized":"2024-03-26T08:31:37.475639+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:14:58.278611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.492261+0000","last_change":"2024-03-26T08:31:26.452464+0000","last_active":"2024-03-26T08:31:37.492261+0000","last_peered":"2024-03-26T08:31:37.492261+0000","last_clean":"2024-03-26T08:31:37.492261+0000","last_became_active":"2024-03-26T08:31:26.451862+0000","last_became_peered":"2024-03-26T08:31:26.451862+0000","last_unstale":"2024-03-26T08:31:37.492261+0000","last_undegraded":"2024-03-26T08:31:37.492261+0000","last_fullsized":"2024-03-26T08:31:37.492261+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:48:29.616247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.489892+0000","last_change":"2024-03-26T08:31:21.365283+0000","last_active":"2024-03-26T08:31:37.489892+0000","last_peered":"2024-03-26T08:31:37.489892+0000","last_clean":"2024-03-26T08:31:37.489892+0000","last_became_active":"2024-03-26T08:31:21.364902+0000","last_became_peered":"2024-03-26T08:31:21.364902+0000","last_unstale":"2024-03-26T08:31:37.489892+0000","last_undegraded":"2024-03-26T08:31:37.489892+0000","last_fullsized":"2024-03-26T08:31:37.489892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:55:39.394355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.479955+0000","last_change":"2024-03-26T08:31:22.391645+0000","last_active":"2024-03-26T08:31:37.479955+0000","last_peered":"2024-03-26T08:31:37.479955+0000","last_clean":"2024-03-26T08:31:37.479955+0000","last_became_active":"2024-03-26T08:31:22.391280+0000","last_became_peered":"2024-03-26T08:31:22.391280+0000","last_unstale":"2024-03-26T08:31:37.479955+0000","last_undegraded":"2024-03-26T08:31:37.479955+0000","last_fullsized":"2024-03-26T08:31:37.479955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:31:48.963287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.475260+0000","last_change":"2024-03-26T08:31:26.446215+0000","last_active":"2024-03-26T08:31:37.475260+0000","last_peered":"2024-03-26T08:31:37.475260+0000","last_clean":"2024-03-26T08:31:37.475260+0000","last_became_active":"2024-03-26T08:31:26.445765+0000","last_became_peered":"2024-03-26T08:31:26.445765+0000","last_unstale":"2024-03-26T08:31:37.475260+0000","last_undegraded":"2024-03-26T08:31:37.475260+0000","last_fullsized":"2024-03-26T08:31:37.475260+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:57:14.809610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.495756+0000","last_change":"2024-03-26T08:31:21.373345+0000","last_active":"2024-03-26T08:31:37.495756+0000","last_peered":"2024-03-26T08:31:37.495756+0000","last_clean":"2024-03-26T08:31:37.495756+0000","last_became_active":"2024-03-26T08:31:21.372906+0000","last_became_peered":"2024-03-26T08:31:21.372906+0000","last_unstale":"2024-03-26T08:31:37.495756+0000","last_undegraded":"2024-03-26T08:31:37.495756+0000","last_fullsized":"2024-03-26T08:31:37.495756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:20:00.518815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.484356+0000","last_change":"2024-03-26T08:31:22.393433+0000","last_active":"2024-03-26T08:31:37.484356+0000","last_peered":"2024-03-26T08:31:37.484356+0000","last_clean":"2024-03-26T08:31:37.484356+0000","last_became_active":"2024-03-26T08:31:22.393100+0000","last_became_peered":"2024-03-26T08:31:22.393100+0000","last_unstale":"2024-03-26T08:31:37.484356+0000","last_undegraded":"2024-03-26T08:31:37.484356+0000","last_fullsized":"2024-03-26T08:31:37.484356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:33:30.418492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.482525+0000","last_change":"2024-03-26T08:31:26.449975+0000","last_active":"2024-03-26T08:31:37.482525+0000","last_peered":"2024-03-26T08:31:37.482525+0000","last_clean":"2024-03-26T08:31:37.482525+0000","last_became_active":"2024-03-26T08:31:26.448345+0000","last_became_peered":"2024-03-26T08:31:26.448345+0000","last_unstale":"2024-03-26T08:31:37.482525+0000","last_undegraded":"2024-03-26T08:31:37.482525+0000","last_fullsized":"2024-03-26T08:31:37.482525+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:01:24.765939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.495512+0000","last_change":"2024-03-26T08:31:21.364195+0000","last_active":"2024-03-26T08:31:37.495512+0000","last_peered":"2024-03-26T08:31:37.495512+0000","last_clean":"2024-03-26T08:31:37.495512+0000","last_became_active":"2024-03-26T08:31:21.363887+0000","last_became_peered":"2024-03-26T08:31:21.363887+0000","last_unstale":"2024-03-26T08:31:37.495512+0000","last_undegraded":"2024-03-26T08:31:37.495512+0000","last_fullsized":"2024-03-26T08:31:37.495512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:03:03.015581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.493245+0000","last_change":"2024-03-26T08:31:37.493245+0000","last_active":"2024-03-26T08:31:37.493245+0000","last_peered":"2024-03-26T08:31:37.493245+0000","last_clean":"2024-03-26T08:31:37.493245+0000","last_became_active":"2024-03-26T08:31:37.492509+0000","last_became_peered":"2024-03-26T08:31:37.492509+0000","last_unstale":"2024-03-26T08:31:37.493245+0000","last_undegraded":"2024-03-26T08:31:37.493245+0000","last_fullsized":"2024-03-26T08:31:37.493245+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:28:56.355010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.475182+0000","last_change":"2024-03-26T08:31:26.449405+0000","last_active":"2024-03-26T08:31:37.475182+0000","last_peered":"2024-03-26T08:31:37.475182+0000","last_clean":"2024-03-26T08:31:37.475182+0000","last_became_active":"2024-03-26T08:31:26.448934+0000","last_became_peered":"2024-03-26T08:31:26.448934+0000","last_unstale":"2024-03-26T08:31:37.475182+0000","last_undegraded":"2024-03-26T08:31:37.475182+0000","last_fullsized":"2024-03-26T08:31:37.475182+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:12:22.797816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'17","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:49.655720+0000","last_change":"2024-03-26T08:31:21.372300+0000","last_active":"2024-03-26T08:31:49.655720+0000","last_peered":"2024-03-26T08:31:49.655720+0000","last_clean":"2024-03-26T08:31:49.655720+0000","last_became_active":"2024-03-26T08:31:21.370103+0000","last_became_peered":"2024-03-26T08:31:21.370103+0000","last_unstale":"2024-03-26T08:31:49.655720+0000","last_undegraded":"2024-03-26T08:31:49.655720+0000","last_fullsized":"2024-03-26T08:31:49.655720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:54:52.657768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23141,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":15,"num_write":17,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.483057+0000","last_change":"2024-03-26T08:31:22.373102+0000","last_active":"2024-03-26T08:31:37.483057+0000","last_peered":"2024-03-26T08:31:37.483057+0000","last_clean":"2024-03-26T08:31:37.483057+0000","last_became_active":"2024-03-26T08:31:22.372792+0000","last_became_peered":"2024-03-26T08:31:22.372792+0000","last_unstale":"2024-03-26T08:31:37.483057+0000","last_undegraded":"2024-03-26T08:31:37.483057+0000","last_fullsized":"2024-03-26T08:31:37.483057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:58:58.274941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.496650+0000","last_change":"2024-03-26T08:31:26.448172+0000","last_active":"2024-03-26T08:31:37.496650+0000","last_peered":"2024-03-26T08:31:37.496650+0000","last_clean":"2024-03-26T08:31:37.496650+0000","last_became_active":"2024-03-26T08:31:26.447550+0000","last_became_peered":"2024-03-26T08:31:26.447550+0000","last_unstale":"2024-03-26T08:31:37.496650+0000","last_undegraded":"2024-03-26T08:31:37.496650+0000","last_fullsized":"2024-03-26T08:31:37.496650+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:56:19.806076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.618648+0000","last_change":"2024-03-26T08:31:21.363946+0000","last_active":"2024-03-26T08:31:37.618648+0000","last_peered":"2024-03-26T08:31:37.618648+0000","last_clean":"2024-03-26T08:31:37.618648+0000","last_became_active":"2024-03-26T08:31:21.363566+0000","last_became_peered":"2024-03-26T08:31:21.363566+0000","last_unstale":"2024-03-26T08:31:37.618648+0000","last_undegraded":"2024-03-26T08:31:37.618648+0000","last_fullsized":"2024-03-26T08:31:37.618648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:32:51.714454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.489387+0000","last_change":"2024-03-26T08:31:22.375813+0000","last_active":"2024-03-26T08:31:37.489387+0000","last_peered":"2024-03-26T08:31:37.489387+0000","last_clean":"2024-03-26T08:31:37.489387+0000","last_became_active":"2024-03-26T08:31:22.375437+0000","last_became_peered":"2024-03-26T08:31:22.375437+0000","last_unstale":"2024-03-26T08:31:37.489387+0000","last_undegraded":"2024-03-26T08:31:37.489387+0000","last_fullsized":"2024-03-26T08:31:37.489387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:48:25.167143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.488397+0000","last_change":"2024-03-26T08:31:26.452079+0000","last_active":"2024-03-26T08:31:37.488397+0000","last_peered":"2024-03-26T08:31:37.488397+0000","last_clean":"2024-03-26T08:31:37.488397+0000","last_became_active":"2024-03-26T08:31:26.451146+0000","last_became_peered":"2024-03-26T08:31:26.451146+0000","last_unstale":"2024-03-26T08:31:37.488397+0000","last_undegraded":"2024-03-26T08:31:37.488397+0000","last_fullsized":"2024-03-26T08:31:37.488397+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:08:57.803180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.499295+0000","last_change":"2024-03-26T08:31:21.371222+0000","last_active":"2024-03-26T08:31:37.499295+0000","last_peered":"2024-03-26T08:31:37.499295+0000","last_clean":"2024-03-26T08:31:37.499295+0000","last_became_active":"2024-03-26T08:31:21.370843+0000","last_became_peered":"2024-03-26T08:31:21.370843+0000","last_unstale":"2024-03-26T08:31:37.499295+0000","last_undegraded":"2024-03-26T08:31:37.499295+0000","last_fullsized":"2024-03-26T08:31:37.499295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:32:00.518713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.492125+0000","last_change":"2024-03-26T08:31:22.379781+0000","last_active":"2024-03-26T08:31:37.492125+0000","last_peered":"2024-03-26T08:31:37.492125+0000","last_clean":"2024-03-26T08:31:37.492125+0000","last_became_active":"2024-03-26T08:31:22.379357+0000","last_became_peered":"2024-03-26T08:31:22.379357+0000","last_unstale":"2024-03-26T08:31:37.492125+0000","last_undegraded":"2024-03-26T08:31:37.492125+0000","last_fullsized":"2024-03-26T08:31:37.492125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:21:52.306343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.500702+0000","last_change":"2024-03-26T08:31:37.498490+0000","last_active":"2024-03-26T08:31:37.500702+0000","last_peered":"2024-03-26T08:31:37.500702+0000","last_clean":"2024-03-26T08:31:37.500702+0000","last_became_active":"2024-03-26T08:31:37.496608+0000","last_became_peered":"2024-03-26T08:31:37.496608+0000","last_unstale":"2024-03-26T08:31:37.500702+0000","last_undegraded":"2024-03-26T08:31:37.500702+0000","last_fullsized":"2024-03-26T08:31:37.500702+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:32:11.653087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.485003+0000","last_change":"2024-03-26T08:31:21.366945+0000","last_active":"2024-03-26T08:31:37.485003+0000","last_peered":"2024-03-26T08:31:37.485003+0000","last_clean":"2024-03-26T08:31:37.485003+0000","last_became_active":"2024-03-26T08:31:21.366548+0000","last_became_peered":"2024-03-26T08:31:21.366548+0000","last_unstale":"2024-03-26T08:31:37.485003+0000","last_undegraded":"2024-03-26T08:31:37.485003+0000","last_fullsized":"2024-03-26T08:31:37.485003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:13:52.037163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.490058+0000","last_change":"2024-03-26T08:31:22.387575+0000","last_active":"2024-03-26T08:31:37.490058+0000","last_peered":"2024-03-26T08:31:37.490058+0000","last_clean":"2024-03-26T08:31:37.490058+0000","last_became_active":"2024-03-26T08:31:22.383815+0000","last_became_peered":"2024-03-26T08:31:22.383815+0000","last_unstale":"2024-03-26T08:31:37.490058+0000","last_undegraded":"2024-03-26T08:31:37.490058+0000","last_fullsized":"2024-03-26T08:31:37.490058+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:10:20.318597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.490627+0000","last_change":"2024-03-26T08:31:26.440745+0000","last_active":"2024-03-26T08:31:37.490627+0000","last_peered":"2024-03-26T08:31:37.490627+0000","last_clean":"2024-03-26T08:31:37.490627+0000","last_became_active":"2024-03-26T08:31:26.440199+0000","last_became_peered":"2024-03-26T08:31:26.440199+0000","last_unstale":"2024-03-26T08:31:37.490627+0000","last_undegraded":"2024-03-26T08:31:37.490627+0000","last_fullsized":"2024-03-26T08:31:37.490627+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:25:11.407274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.481224+0000","last_change":"2024-03-26T08:31:21.369227+0000","last_active":"2024-03-26T08:31:37.481224+0000","last_peered":"2024-03-26T08:31:37.481224+0000","last_clean":"2024-03-26T08:31:37.481224+0000","last_became_active":"2024-03-26T08:31:21.368709+0000","last_became_peered":"2024-03-26T08:31:21.368709+0000","last_unstale":"2024-03-26T08:31:37.481224+0000","last_undegraded":"2024-03-26T08:31:37.481224+0000","last_fullsized":"2024-03-26T08:31:37.481224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:30:04.714715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.498997+0000","last_change":"2024-03-26T08:31:22.384199+0000","last_active":"2024-03-26T08:31:37.498997+0000","last_peered":"2024-03-26T08:31:37.498997+0000","last_clean":"2024-03-26T08:31:37.498997+0000","last_became_active":"2024-03-26T08:31:22.383854+0000","last_became_peered":"2024-03-26T08:31:22.383854+0000","last_unstale":"2024-03-26T08:31:37.498997+0000","last_undegraded":"2024-03-26T08:31:37.498997+0000","last_fullsized":"2024-03-26T08:31:37.498997+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:33:24.240505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.498650+0000","last_change":"2024-03-26T08:31:26.447657+0000","last_active":"2024-03-26T08:31:37.498650+0000","last_peered":"2024-03-26T08:31:37.498650+0000","last_clean":"2024-03-26T08:31:37.498650+0000","last_became_active":"2024-03-26T08:31:26.444812+0000","last_became_peered":"2024-03-26T08:31:26.444812+0000","last_unstale":"2024-03-26T08:31:37.498650+0000","last_undegraded":"2024-03-26T08:31:37.498650+0000","last_fullsized":"2024-03-26T08:31:37.498650+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:58:21.422318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:40.553106+0000","last_change":"2024-03-26T08:31:37.510665+0000","last_active":"2024-03-26T08:31:40.553106+0000","last_peered":"2024-03-26T08:31:40.553106+0000","last_clean":"2024-03-26T08:31:40.553106+0000","last_became_active":"2024-03-26T08:31:37.504109+0000","last_became_peered":"2024-03-26T08:31:37.504109+0000","last_unstale":"2024-03-26T08:31:40.553106+0000","last_undegraded":"2024-03-26T08:31:40.553106+0000","last_fullsized":"2024-03-26T08:31:40.553106+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:38:21.446881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.502088+0000","last_change":"2024-03-26T08:31:37.502088+0000","last_active":"2024-03-26T08:31:37.502088+0000","last_peered":"2024-03-26T08:31:37.502088+0000","last_clean":"2024-03-26T08:31:37.502088+0000","last_became_active":"2024-03-26T08:31:37.499767+0000","last_became_peered":"2024-03-26T08:31:37.499767+0000","last_unstale":"2024-03-26T08:31:37.502088+0000","last_undegraded":"2024-03-26T08:31:37.502088+0000","last_fullsized":"2024-03-26T08:31:37.502088+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:55:32.652231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.482121+0000","last_change":"2024-03-26T08:31:26.479654+0000","last_active":"2024-03-26T08:31:37.482121+0000","last_peered":"2024-03-26T08:31:37.482121+0000","last_clean":"2024-03-26T08:31:37.482121+0000","last_became_active":"2024-03-26T08:31:26.478948+0000","last_became_peered":"2024-03-26T08:31:26.478948+0000","last_unstale":"2024-03-26T08:31:37.482121+0000","last_undegraded":"2024-03-26T08:31:37.482121+0000","last_fullsized":"2024-03-26T08:31:37.482121+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:33:58.445531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.476079+0000","last_change":"2024-03-26T08:31:21.365203+0000","last_active":"2024-03-26T08:31:37.476079+0000","last_peered":"2024-03-26T08:31:37.476079+0000","last_clean":"2024-03-26T08:31:37.476079+0000","last_became_active":"2024-03-26T08:31:21.364816+0000","last_became_peered":"2024-03-26T08:31:21.364816+0000","last_unstale":"2024-03-26T08:31:37.476079+0000","last_undegraded":"2024-03-26T08:31:37.476079+0000","last_fullsized":"2024-03-26T08:31:37.476079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:59:50.157011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.492710+0000","last_change":"2024-03-26T08:31:22.392014+0000","last_active":"2024-03-26T08:31:37.492710+0000","last_peered":"2024-03-26T08:31:37.492710+0000","last_clean":"2024-03-26T08:31:37.492710+0000","last_became_active":"2024-03-26T08:31:22.391627+0000","last_became_peered":"2024-03-26T08:31:22.391627+0000","last_unstale":"2024-03-26T08:31:37.492710+0000","last_undegraded":"2024-03-26T08:31:37.492710+0000","last_fullsized":"2024-03-26T08:31:37.492710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:59:12.544248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.490621+0000","last_change":"2024-03-26T08:31:26.444754+0000","last_active":"2024-03-26T08:31:37.490621+0000","last_peered":"2024-03-26T08:31:37.490621+0000","last_clean":"2024-03-26T08:31:37.490621+0000","last_became_active":"2024-03-26T08:31:26.444331+0000","last_became_peered":"2024-03-26T08:31:26.444331+0000","last_unstale":"2024-03-26T08:31:37.490621+0000","last_undegraded":"2024-03-26T08:31:37.490621+0000","last_fullsized":"2024-03-26T08:31:37.490621+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:27:58.430695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.490937+0000","last_change":"2024-03-26T08:31:21.371028+0000","last_active":"2024-03-26T08:31:37.490937+0000","last_peered":"2024-03-26T08:31:37.490937+0000","last_clean":"2024-03-26T08:31:37.490937+0000","last_became_active":"2024-03-26T08:31:21.370631+0000","last_became_peered":"2024-03-26T08:31:21.370631+0000","last_unstale":"2024-03-26T08:31:37.490937+0000","last_undegraded":"2024-03-26T08:31:37.490937+0000","last_fullsized":"2024-03-26T08:31:37.490937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:33:25.546255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.490360+0000","last_change":"2024-03-26T08:31:22.388791+0000","last_active":"2024-03-26T08:31:37.490360+0000","last_peered":"2024-03-26T08:31:37.490360+0000","last_clean":"2024-03-26T08:31:37.490360+0000","last_became_active":"2024-03-26T08:31:22.388342+0000","last_became_peered":"2024-03-26T08:31:22.388342+0000","last_unstale":"2024-03-26T08:31:37.490360+0000","last_undegraded":"2024-03-26T08:31:37.490360+0000","last_fullsized":"2024-03-26T08:31:37.490360+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:26:30.537880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.489879+0000","last_change":"2024-03-26T08:31:26.478919+0000","last_active":"2024-03-26T08:31:37.489879+0000","last_peered":"2024-03-26T08:31:37.489879+0000","last_clean":"2024-03-26T08:31:37.489879+0000","last_became_active":"2024-03-26T08:31:26.478330+0000","last_became_peered":"2024-03-26T08:31:26.478330+0000","last_unstale":"2024-03-26T08:31:37.489879+0000","last_undegraded":"2024-03-26T08:31:37.489879+0000","last_fullsized":"2024-03-26T08:31:37.489879+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:37:36.421982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.497216+0000","last_change":"2024-03-26T08:31:22.384299+0000","last_active":"2024-03-26T08:31:37.497216+0000","last_peered":"2024-03-26T08:31:37.497216+0000","last_clean":"2024-03-26T08:31:37.497216+0000","last_became_active":"2024-03-26T08:31:22.383955+0000","last_became_peered":"2024-03-26T08:31:22.383955+0000","last_unstale":"2024-03-26T08:31:37.497216+0000","last_undegraded":"2024-03-26T08:31:37.497216+0000","last_fullsized":"2024-03-26T08:31:37.497216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:08:53.065524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.488692+0000","last_change":"2024-03-26T08:31:21.358464+0000","last_active":"2024-03-26T08:31:37.488692+0000","last_peered":"2024-03-26T08:31:37.488692+0000","last_clean":"2024-03-26T08:31:37.488692+0000","last_became_active":"2024-03-26T08:31:21.356853+0000","last_became_peered":"2024-03-26T08:31:21.356853+0000","last_unstale":"2024-03-26T08:31:37.488692+0000","last_undegraded":"2024-03-26T08:31:37.488692+0000","last_fullsized":"2024-03-26T08:31:37.488692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:08:21.395306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.502144+0000","last_change":"2024-03-26T08:31:37.502144+0000","last_active":"2024-03-26T08:31:37.502144+0000","last_peered":"2024-03-26T08:31:37.502144+0000","last_clean":"2024-03-26T08:31:37.502144+0000","last_became_active":"2024-03-26T08:31:37.501465+0000","last_became_peered":"2024-03-26T08:31:37.501465+0000","last_unstale":"2024-03-26T08:31:37.502144+0000","last_undegraded":"2024-03-26T08:31:37.502144+0000","last_fullsized":"2024-03-26T08:31:37.502144+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:30:18.771898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.495955+0000","last_change":"2024-03-26T08:31:22.403380+0000","last_active":"2024-03-26T08:31:37.495955+0000","last_peered":"2024-03-26T08:31:37.495955+0000","last_clean":"2024-03-26T08:31:37.495955+0000","last_became_active":"2024-03-26T08:31:22.400991+0000","last_became_peered":"2024-03-26T08:31:22.400991+0000","last_unstale":"2024-03-26T08:31:37.495955+0000","last_undegraded":"2024-03-26T08:31:37.495955+0000","last_fullsized":"2024-03-26T08:31:37.495955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:04:38.842919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.482253+0000","last_change":"2024-03-26T08:31:21.358570+0000","last_active":"2024-03-26T08:31:37.482253+0000","last_peered":"2024-03-26T08:31:37.482253+0000","last_clean":"2024-03-26T08:31:37.482253+0000","last_became_active":"2024-03-26T08:31:21.356603+0000","last_became_peered":"2024-03-26T08:31:21.356603+0000","last_unstale":"2024-03-26T08:31:37.482253+0000","last_undegraded":"2024-03-26T08:31:37.482253+0000","last_fullsized":"2024-03-26T08:31:37.482253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:10:28.829223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":351,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.491987+0000","last_change":"2024-03-26T08:29:39.048593+0000","last_active":"2024-03-26T08:31:37.491987+0000","last_peered":"2024-03-26T08:31:37.491987+0000","last_clean":"2024-03-26T08:31:37.491987+0000","last_became_active":"2024-03-26T08:29:39.047909+0000","last_became_peered":"2024-03-26T08:29:39.047909+0000","last_unstale":"2024-03-26T08:31:37.491987+0000","last_undegraded":"2024-03-26T08:31:37.491987+0000","last_fullsized":"2024-03-26T08:31:37.491987+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:24:55.011726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:24:55.011726+0000","last_clean_scrub_stamp":"2024-03-26T08:24:55.011726+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:46:55.510857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.495687+0000","last_change":"2024-03-26T08:31:26.478740+0000","last_active":"2024-03-26T08:31:37.495687+0000","last_peered":"2024-03-26T08:31:37.495687+0000","last_clean":"2024-03-26T08:31:37.495687+0000","last_became_active":"2024-03-26T08:31:26.478218+0000","last_became_peered":"2024-03-26T08:31:26.478218+0000","last_unstale":"2024-03-26T08:31:37.495687+0000","last_undegraded":"2024-03-26T08:31:37.495687+0000","last_fullsized":"2024-03-26T08:31:37.495687+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:02:54.506805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.488379+0000","last_change":"2024-03-26T08:31:22.381931+0000","last_active":"2024-03-26T08:31:37.488379+0000","last_peered":"2024-03-26T08:31:37.488379+0000","last_clean":"2024-03-26T08:31:37.488379+0000","last_became_active":"2024-03-26T08:31:22.379021+0000","last_became_peered":"2024-03-26T08:31:22.379021+0000","last_unstale":"2024-03-26T08:31:37.488379+0000","last_undegraded":"2024-03-26T08:31:37.488379+0000","last_fullsized":"2024-03-26T08:31:37.488379+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:07:31.555154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.489823+0000","last_change":"2024-03-26T08:31:21.375753+0000","last_active":"2024-03-26T08:31:37.489823+0000","last_peered":"2024-03-26T08:31:37.489823+0000","last_clean":"2024-03-26T08:31:37.489823+0000","last_became_active":"2024-03-26T08:31:21.375340+0000","last_became_peered":"2024-03-26T08:31:21.375340+0000","last_unstale":"2024-03-26T08:31:37.489823+0000","last_undegraded":"2024-03-26T08:31:37.489823+0000","last_fullsized":"2024-03-26T08:31:37.489823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:21:14.241774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.488046+0000","last_change":"2024-03-26T08:31:26.454026+0000","last_active":"2024-03-26T08:31:37.488046+0000","last_peered":"2024-03-26T08:31:37.488046+0000","last_clean":"2024-03-26T08:31:37.488046+0000","last_became_active":"2024-03-26T08:31:26.453494+0000","last_became_peered":"2024-03-26T08:31:26.453494+0000","last_unstale":"2024-03-26T08:31:37.488046+0000","last_undegraded":"2024-03-26T08:31:37.488046+0000","last_fullsized":"2024-03-26T08:31:37.488046+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:30:10.388280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.483076+0000","last_change":"2024-03-26T08:31:21.358930+0000","last_active":"2024-03-26T08:31:37.483076+0000","last_peered":"2024-03-26T08:31:37.483076+0000","last_clean":"2024-03-26T08:31:37.483076+0000","last_became_active":"2024-03-26T08:31:21.357573+0000","last_became_peered":"2024-03-26T08:31:21.357573+0000","last_unstale":"2024-03-26T08:31:37.483076+0000","last_undegraded":"2024-03-26T08:31:37.483076+0000","last_fullsized":"2024-03-26T08:31:37.483076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:36:41.572993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.490376+0000","last_change":"2024-03-26T08:31:22.378520+0000","last_active":"2024-03-26T08:31:37.490376+0000","last_peered":"2024-03-26T08:31:37.490376+0000","last_clean":"2024-03-26T08:31:37.490376+0000","last_became_active":"2024-03-26T08:31:22.378081+0000","last_became_peered":"2024-03-26T08:31:22.378081+0000","last_unstale":"2024-03-26T08:31:37.490376+0000","last_undegraded":"2024-03-26T08:31:37.490376+0000","last_fullsized":"2024-03-26T08:31:37.490376+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:09:34.748182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.491619+0000","last_change":"2024-03-26T08:31:26.479545+0000","last_active":"2024-03-26T08:31:37.491619+0000","last_peered":"2024-03-26T08:31:37.491619+0000","last_clean":"2024-03-26T08:31:37.491619+0000","last_became_active":"2024-03-26T08:31:26.479029+0000","last_became_peered":"2024-03-26T08:31:26.479029+0000","last_unstale":"2024-03-26T08:31:37.491619+0000","last_undegraded":"2024-03-26T08:31:37.491619+0000","last_fullsized":"2024-03-26T08:31:37.491619+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:49:03.795752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.478175+0000","last_change":"2024-03-26T08:31:21.360510+0000","last_active":"2024-03-26T08:31:37.478175+0000","last_peered":"2024-03-26T08:31:37.478175+0000","last_clean":"2024-03-26T08:31:37.478175+0000","last_became_active":"2024-03-26T08:31:21.359593+0000","last_became_peered":"2024-03-26T08:31:21.359593+0000","last_unstale":"2024-03-26T08:31:37.478175+0000","last_undegraded":"2024-03-26T08:31:37.478175+0000","last_fullsized":"2024-03-26T08:31:37.478175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:37:33.752689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.496466+0000","last_change":"2024-03-26T08:31:22.390181+0000","last_active":"2024-03-26T08:31:37.496466+0000","last_peered":"2024-03-26T08:31:37.496466+0000","last_clean":"2024-03-26T08:31:37.496466+0000","last_became_active":"2024-03-26T08:31:22.389686+0000","last_became_peered":"2024-03-26T08:31:22.389686+0000","last_unstale":"2024-03-26T08:31:37.496466+0000","last_undegraded":"2024-03-26T08:31:37.496466+0000","last_fullsized":"2024-03-26T08:31:37.496466+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:48:09.913134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.617390+0000","last_change":"2024-03-26T08:31:26.449160+0000","last_active":"2024-03-26T08:31:37.617390+0000","last_peered":"2024-03-26T08:31:37.617390+0000","last_clean":"2024-03-26T08:31:37.617390+0000","last_became_active":"2024-03-26T08:31:26.448570+0000","last_became_peered":"2024-03-26T08:31:26.448570+0000","last_unstale":"2024-03-26T08:31:37.617390+0000","last_undegraded":"2024-03-26T08:31:37.617390+0000","last_fullsized":"2024-03-26T08:31:37.617390+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:05:21.258982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.482445+0000","last_change":"2024-03-26T08:31:21.365723+0000","last_active":"2024-03-26T08:31:37.482445+0000","last_peered":"2024-03-26T08:31:37.482445+0000","last_clean":"2024-03-26T08:31:37.482445+0000","last_became_active":"2024-03-26T08:31:21.364893+0000","last_became_peered":"2024-03-26T08:31:21.364893+0000","last_unstale":"2024-03-26T08:31:37.482445+0000","last_undegraded":"2024-03-26T08:31:37.482445+0000","last_fullsized":"2024-03-26T08:31:37.482445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:24:17.210093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.484283+0000","last_change":"2024-03-26T08:31:22.396341+0000","last_active":"2024-03-26T08:31:37.484283+0000","last_peered":"2024-03-26T08:31:37.484283+0000","last_clean":"2024-03-26T08:31:37.484283+0000","last_became_active":"2024-03-26T08:31:22.395956+0000","last_became_peered":"2024-03-26T08:31:22.395956+0000","last_unstale":"2024-03-26T08:31:37.484283+0000","last_undegraded":"2024-03-26T08:31:37.484283+0000","last_fullsized":"2024-03-26T08:31:37.484283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:08:51.194370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.497971+0000","last_change":"2024-03-26T08:31:37.497971+0000","last_active":"2024-03-26T08:31:37.497971+0000","last_peered":"2024-03-26T08:31:37.497971+0000","last_clean":"2024-03-26T08:31:37.497971+0000","last_became_active":"2024-03-26T08:31:37.497379+0000","last_became_peered":"2024-03-26T08:31:37.497379+0000","last_unstale":"2024-03-26T08:31:37.497971+0000","last_undegraded":"2024-03-26T08:31:37.497971+0000","last_fullsized":"2024-03-26T08:31:37.497971+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:35:55.266756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.493616+0000","last_change":"2024-03-26T08:31:37.493616+0000","last_active":"2024-03-26T08:31:37.493616+0000","last_peered":"2024-03-26T08:31:37.493616+0000","last_clean":"2024-03-26T08:31:37.493616+0000","last_became_active":"2024-03-26T08:31:37.492288+0000","last_became_peered":"2024-03-26T08:31:37.492288+0000","last_unstale":"2024-03-26T08:31:37.493616+0000","last_undegraded":"2024-03-26T08:31:37.493616+0000","last_fullsized":"2024-03-26T08:31:37.493616+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:49:10.413751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.482550+0000","last_change":"2024-03-26T08:31:22.384347+0000","last_active":"2024-03-26T08:31:37.482550+0000","last_peered":"2024-03-26T08:31:37.482550+0000","last_clean":"2024-03-26T08:31:37.482550+0000","last_became_active":"2024-03-26T08:31:22.380639+0000","last_became_peered":"2024-03-26T08:31:22.380639+0000","last_unstale":"2024-03-26T08:31:37.482550+0000","last_undegraded":"2024-03-26T08:31:37.482550+0000","last_fullsized":"2024-03-26T08:31:37.482550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:39:07.778005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.476638+0000","last_change":"2024-03-26T08:31:26.455706+0000","last_active":"2024-03-26T08:31:37.476638+0000","last_peered":"2024-03-26T08:31:37.476638+0000","last_clean":"2024-03-26T08:31:37.476638+0000","last_became_active":"2024-03-26T08:31:26.454591+0000","last_became_peered":"2024-03-26T08:31:26.454591+0000","last_unstale":"2024-03-26T08:31:37.476638+0000","last_undegraded":"2024-03-26T08:31:37.476638+0000","last_fullsized":"2024-03-26T08:31:37.476638+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:07:30.612220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.492584+0000","last_change":"2024-03-26T08:31:21.357734+0000","last_active":"2024-03-26T08:31:37.492584+0000","last_peered":"2024-03-26T08:31:37.492584+0000","last_clean":"2024-03-26T08:31:37.492584+0000","last_became_active":"2024-03-26T08:31:21.357361+0000","last_became_peered":"2024-03-26T08:31:21.357361+0000","last_unstale":"2024-03-26T08:31:37.492584+0000","last_undegraded":"2024-03-26T08:31:37.492584+0000","last_fullsized":"2024-03-26T08:31:37.492584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:26:57.454577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.488392+0000","last_change":"2024-03-26T08:31:22.383118+0000","last_active":"2024-03-26T08:31:37.488392+0000","last_peered":"2024-03-26T08:31:37.488392+0000","last_clean":"2024-03-26T08:31:37.488392+0000","last_became_active":"2024-03-26T08:31:22.382773+0000","last_became_peered":"2024-03-26T08:31:22.382773+0000","last_unstale":"2024-03-26T08:31:37.488392+0000","last_undegraded":"2024-03-26T08:31:37.488392+0000","last_fullsized":"2024-03-26T08:31:37.488392+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:50:34.889914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.478266+0000","last_change":"2024-03-26T08:31:26.451467+0000","last_active":"2024-03-26T08:31:37.478266+0000","last_peered":"2024-03-26T08:31:37.478266+0000","last_clean":"2024-03-26T08:31:37.478266+0000","last_became_active":"2024-03-26T08:31:26.451022+0000","last_became_peered":"2024-03-26T08:31:26.451022+0000","last_unstale":"2024-03-26T08:31:37.478266+0000","last_undegraded":"2024-03-26T08:31:37.478266+0000","last_fullsized":"2024-03-26T08:31:37.478266+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:19:50.442779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.617859+0000","last_change":"2024-03-26T08:31:21.361596+0000","last_active":"2024-03-26T08:31:37.617859+0000","last_peered":"2024-03-26T08:31:37.617859+0000","last_clean":"2024-03-26T08:31:37.617859+0000","last_became_active":"2024-03-26T08:31:21.357502+0000","last_became_peered":"2024-03-26T08:31:21.357502+0000","last_unstale":"2024-03-26T08:31:37.617859+0000","last_undegraded":"2024-03-26T08:31:37.617859+0000","last_fullsized":"2024-03-26T08:31:37.617859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:05:33.951346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.496907+0000","last_change":"2024-03-26T08:31:22.397344+0000","last_active":"2024-03-26T08:31:37.496907+0000","last_peered":"2024-03-26T08:31:37.496907+0000","last_clean":"2024-03-26T08:31:37.496907+0000","last_became_active":"2024-03-26T08:31:22.396936+0000","last_became_peered":"2024-03-26T08:31:22.396936+0000","last_unstale":"2024-03-26T08:31:37.496907+0000","last_undegraded":"2024-03-26T08:31:37.496907+0000","last_fullsized":"2024-03-26T08:31:37.496907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:07:49.412098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.479959+0000","last_change":"2024-03-26T08:31:26.479060+0000","last_active":"2024-03-26T08:31:37.479959+0000","last_peered":"2024-03-26T08:31:37.479959+0000","last_clean":"2024-03-26T08:31:37.479959+0000","last_became_active":"2024-03-26T08:31:26.478519+0000","last_became_peered":"2024-03-26T08:31:26.478519+0000","last_unstale":"2024-03-26T08:31:37.479959+0000","last_undegraded":"2024-03-26T08:31:37.479959+0000","last_fullsized":"2024-03-26T08:31:37.479959+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:37:20.528845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.482635+0000","last_change":"2024-03-26T08:31:21.374128+0000","last_active":"2024-03-26T08:31:37.482635+0000","last_peered":"2024-03-26T08:31:37.482635+0000","last_clean":"2024-03-26T08:31:37.482635+0000","last_became_active":"2024-03-26T08:31:21.372934+0000","last_became_peered":"2024-03-26T08:31:21.372934+0000","last_unstale":"2024-03-26T08:31:37.482635+0000","last_undegraded":"2024-03-26T08:31:37.482635+0000","last_fullsized":"2024-03-26T08:31:37.482635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:25:15.919855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.491242+0000","last_change":"2024-03-26T08:31:22.381847+0000","last_active":"2024-03-26T08:31:37.491242+0000","last_peered":"2024-03-26T08:31:37.491242+0000","last_clean":"2024-03-26T08:31:37.491242+0000","last_became_active":"2024-03-26T08:31:22.379956+0000","last_became_peered":"2024-03-26T08:31:22.379956+0000","last_unstale":"2024-03-26T08:31:37.491242+0000","last_undegraded":"2024-03-26T08:31:37.491242+0000","last_fullsized":"2024-03-26T08:31:37.491242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:16:38.048495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.477598+0000","last_change":"2024-03-26T08:31:26.455503+0000","last_active":"2024-03-26T08:31:37.477598+0000","last_peered":"2024-03-26T08:31:37.477598+0000","last_clean":"2024-03-26T08:31:37.477598+0000","last_became_active":"2024-03-26T08:31:26.455023+0000","last_became_peered":"2024-03-26T08:31:26.455023+0000","last_unstale":"2024-03-26T08:31:37.477598+0000","last_undegraded":"2024-03-26T08:31:37.477598+0000","last_fullsized":"2024-03-26T08:31:37.477598+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:06:47.565407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.618849+0000","last_change":"2024-03-26T08:31:21.362832+0000","last_active":"2024-03-26T08:31:37.618849+0000","last_peered":"2024-03-26T08:31:37.618849+0000","last_clean":"2024-03-26T08:31:37.618849+0000","last_became_active":"2024-03-26T08:31:21.362411+0000","last_became_peered":"2024-03-26T08:31:21.362411+0000","last_unstale":"2024-03-26T08:31:37.618849+0000","last_undegraded":"2024-03-26T08:31:37.618849+0000","last_fullsized":"2024-03-26T08:31:37.618849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:34:07.646011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.494839+0000","last_change":"2024-03-26T08:31:22.403106+0000","last_active":"2024-03-26T08:31:37.494839+0000","last_peered":"2024-03-26T08:31:37.494839+0000","last_clean":"2024-03-26T08:31:37.494839+0000","last_became_active":"2024-03-26T08:31:22.401172+0000","last_became_peered":"2024-03-26T08:31:22.401172+0000","last_unstale":"2024-03-26T08:31:37.494839+0000","last_undegraded":"2024-03-26T08:31:37.494839+0000","last_fullsized":"2024-03-26T08:31:37.494839+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:31:42.654817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.495759+0000","last_change":"2024-03-26T08:31:37.495759+0000","last_active":"2024-03-26T08:31:37.495759+0000","last_peered":"2024-03-26T08:31:37.495759+0000","last_clean":"2024-03-26T08:31:37.495759+0000","last_became_active":"2024-03-26T08:31:37.493610+0000","last_became_peered":"2024-03-26T08:31:37.493610+0000","last_unstale":"2024-03-26T08:31:37.495759+0000","last_undegraded":"2024-03-26T08:31:37.495759+0000","last_fullsized":"2024-03-26T08:31:37.495759+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:17:05.345033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.496272+0000","last_change":"2024-03-26T08:31:21.369396+0000","last_active":"2024-03-26T08:31:37.496272+0000","last_peered":"2024-03-26T08:31:37.496272+0000","last_clean":"2024-03-26T08:31:37.496272+0000","last_became_active":"2024-03-26T08:31:21.364011+0000","last_became_peered":"2024-03-26T08:31:21.364011+0000","last_unstale":"2024-03-26T08:31:37.496272+0000","last_undegraded":"2024-03-26T08:31:37.496272+0000","last_fullsized":"2024-03-26T08:31:37.496272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:23:47.347158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.494166+0000","last_change":"2024-03-26T08:31:22.389586+0000","last_active":"2024-03-26T08:31:37.494166+0000","last_peered":"2024-03-26T08:31:37.494166+0000","last_clean":"2024-03-26T08:31:37.494166+0000","last_became_active":"2024-03-26T08:31:22.389200+0000","last_became_peered":"2024-03-26T08:31:22.389200+0000","last_unstale":"2024-03-26T08:31:37.494166+0000","last_undegraded":"2024-03-26T08:31:37.494166+0000","last_fullsized":"2024-03-26T08:31:37.494166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:54:30.076744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.494970+0000","last_change":"2024-03-26T08:31:26.452274+0000","last_active":"2024-03-26T08:31:37.494970+0000","last_peered":"2024-03-26T08:31:37.494970+0000","last_clean":"2024-03-26T08:31:37.494970+0000","last_became_active":"2024-03-26T08:31:26.451670+0000","last_became_peered":"2024-03-26T08:31:26.451670+0000","last_unstale":"2024-03-26T08:31:37.494970+0000","last_undegraded":"2024-03-26T08:31:37.494970+0000","last_fullsized":"2024-03-26T08:31:37.494970+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:48:36.028810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.490244+0000","last_change":"2024-03-26T08:31:21.356259+0000","last_active":"2024-03-26T08:31:37.490244+0000","last_peered":"2024-03-26T08:31:37.490244+0000","last_clean":"2024-03-26T08:31:37.490244+0000","last_became_active":"2024-03-26T08:31:21.355724+0000","last_became_peered":"2024-03-26T08:31:21.355724+0000","last_unstale":"2024-03-26T08:31:37.490244+0000","last_undegraded":"2024-03-26T08:31:37.490244+0000","last_fullsized":"2024-03-26T08:31:37.490244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:30:05.621417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.495928+0000","last_change":"2024-03-26T08:31:22.370869+0000","last_active":"2024-03-26T08:31:37.495928+0000","last_peered":"2024-03-26T08:31:37.495928+0000","last_clean":"2024-03-26T08:31:37.495928+0000","last_became_active":"2024-03-26T08:31:22.369797+0000","last_became_peered":"2024-03-26T08:31:22.369797+0000","last_unstale":"2024-03-26T08:31:37.495928+0000","last_undegraded":"2024-03-26T08:31:37.495928+0000","last_fullsized":"2024-03-26T08:31:37.495928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:59:33.064619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.488869+0000","last_change":"2024-03-26T08:31:22.377927+0000","last_active":"2024-03-26T08:31:37.488869+0000","last_peered":"2024-03-26T08:31:37.488869+0000","last_clean":"2024-03-26T08:31:37.488869+0000","last_became_active":"2024-03-26T08:31:22.377559+0000","last_became_peered":"2024-03-26T08:31:22.377559+0000","last_unstale":"2024-03-26T08:31:37.488869+0000","last_undegraded":"2024-03-26T08:31:37.488869+0000","last_fullsized":"2024-03-26T08:31:37.488869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:06:05.544179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.617757+0000","last_change":"2024-03-26T08:31:22.380287+0000","last_active":"2024-03-26T08:31:37.617757+0000","last_peered":"2024-03-26T08:31:37.617757+0000","last_clean":"2024-03-26T08:31:37.617757+0000","last_became_active":"2024-03-26T08:31:22.379948+0000","last_became_peered":"2024-03-26T08:31:22.379948+0000","last_unstale":"2024-03-26T08:31:37.617757+0000","last_undegraded":"2024-03-26T08:31:37.617757+0000","last_fullsized":"2024-03-26T08:31:37.617757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:57:12.636151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.617508+0000","last_change":"2024-03-26T08:31:22.390109+0000","last_active":"2024-03-26T08:31:37.617508+0000","last_peered":"2024-03-26T08:31:37.617508+0000","last_clean":"2024-03-26T08:31:37.617508+0000","last_became_active":"2024-03-26T08:31:22.389766+0000","last_became_peered":"2024-03-26T08:31:22.389766+0000","last_unstale":"2024-03-26T08:31:37.617508+0000","last_undegraded":"2024-03-26T08:31:37.617508+0000","last_fullsized":"2024-03-26T08:31:37.617508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:24:27.454598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.485963+0000","last_change":"2024-03-26T08:31:22.391882+0000","last_active":"2024-03-26T08:31:37.485963+0000","last_peered":"2024-03-26T08:31:37.485963+0000","last_clean":"2024-03-26T08:31:37.485963+0000","last_became_active":"2024-03-26T08:31:22.391395+0000","last_became_peered":"2024-03-26T08:31:22.391395+0000","last_unstale":"2024-03-26T08:31:37.485963+0000","last_undegraded":"2024-03-26T08:31:37.485963+0000","last_fullsized":"2024-03-26T08:31:37.485963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:45:06.685287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.489387+0000","last_change":"2024-03-26T08:31:22.385993+0000","last_active":"2024-03-26T08:31:37.489387+0000","last_peered":"2024-03-26T08:31:37.489387+0000","last_clean":"2024-03-26T08:31:37.489387+0000","last_became_active":"2024-03-26T08:31:22.385625+0000","last_became_peered":"2024-03-26T08:31:22.385625+0000","last_unstale":"2024-03-26T08:31:37.489387+0000","last_undegraded":"2024-03-26T08:31:37.489387+0000","last_fullsized":"2024-03-26T08:31:37.489387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:16:20.731718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.490485+0000","last_change":"2024-03-26T08:31:22.379588+0000","last_active":"2024-03-26T08:31:37.490485+0000","last_peered":"2024-03-26T08:31:37.490485+0000","last_clean":"2024-03-26T08:31:37.490485+0000","last_became_active":"2024-03-26T08:31:22.379215+0000","last_became_peered":"2024-03-26T08:31:22.379215+0000","last_unstale":"2024-03-26T08:31:37.490485+0000","last_undegraded":"2024-03-26T08:31:37.490485+0000","last_fullsized":"2024-03-26T08:31:37.490485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:23:20.339737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.618737+0000","last_change":"2024-03-26T08:31:22.404354+0000","last_active":"2024-03-26T08:31:37.618737+0000","last_peered":"2024-03-26T08:31:37.618737+0000","last_clean":"2024-03-26T08:31:37.618737+0000","last_became_active":"2024-03-26T08:31:22.403973+0000","last_became_peered":"2024-03-26T08:31:22.403973+0000","last_unstale":"2024-03-26T08:31:37.618737+0000","last_undegraded":"2024-03-26T08:31:37.618737+0000","last_fullsized":"2024-03-26T08:31:37.618737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:36:54.360147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.496860+0000","last_change":"2024-03-26T08:31:22.379642+0000","last_active":"2024-03-26T08:31:37.496860+0000","last_peered":"2024-03-26T08:31:37.496860+0000","last_clean":"2024-03-26T08:31:37.496860+0000","last_became_active":"2024-03-26T08:31:22.379241+0000","last_became_peered":"2024-03-26T08:31:22.379241+0000","last_unstale":"2024-03-26T08:31:37.496860+0000","last_undegraded":"2024-03-26T08:31:37.496860+0000","last_fullsized":"2024-03-26T08:31:37.496860+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:21:51.117616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.496330+0000","last_change":"2024-03-26T08:31:22.378809+0000","last_active":"2024-03-26T08:31:37.496330+0000","last_peered":"2024-03-26T08:31:37.496330+0000","last_clean":"2024-03-26T08:31:37.496330+0000","last_became_active":"2024-03-26T08:31:22.378388+0000","last_became_peered":"2024-03-26T08:31:22.378388+0000","last_unstale":"2024-03-26T08:31:37.496330+0000","last_undegraded":"2024-03-26T08:31:37.496330+0000","last_fullsized":"2024-03-26T08:31:37.496330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:33:21.583423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.492875+0000","last_change":"2024-03-26T08:31:22.389230+0000","last_active":"2024-03-26T08:31:37.492875+0000","last_peered":"2024-03-26T08:31:37.492875+0000","last_clean":"2024-03-26T08:31:37.492875+0000","last_became_active":"2024-03-26T08:31:22.388870+0000","last_became_peered":"2024-03-26T08:31:22.388870+0000","last_unstale":"2024-03-26T08:31:37.492875+0000","last_undegraded":"2024-03-26T08:31:37.492875+0000","last_fullsized":"2024-03-26T08:31:37.492875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:40:20.377122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.478732+0000","last_change":"2024-03-26T08:31:22.396232+0000","last_active":"2024-03-26T08:31:37.478732+0000","last_peered":"2024-03-26T08:31:37.478732+0000","last_clean":"2024-03-26T08:31:37.478732+0000","last_became_active":"2024-03-26T08:31:22.392940+0000","last_became_peered":"2024-03-26T08:31:22.392940+0000","last_unstale":"2024-03-26T08:31:37.478732+0000","last_undegraded":"2024-03-26T08:31:37.478732+0000","last_fullsized":"2024-03-26T08:31:37.478732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:01:15.460526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.490207+0000","last_change":"2024-03-26T08:31:22.370880+0000","last_active":"2024-03-26T08:31:37.490207+0000","last_peered":"2024-03-26T08:31:37.490207+0000","last_clean":"2024-03-26T08:31:37.490207+0000","last_became_active":"2024-03-26T08:31:22.370546+0000","last_became_peered":"2024-03-26T08:31:22.370546+0000","last_unstale":"2024-03-26T08:31:37.490207+0000","last_undegraded":"2024-03-26T08:31:37.490207+0000","last_fullsized":"2024-03-26T08:31:37.490207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:46:11.657763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T08:31:37.488781+0000","last_change":"2024-03-26T08:31:22.387573+0000","last_active":"2024-03-26T08:31:37.488781+0000","last_peered":"2024-03-26T08:31:37.488781+0000","last_clean":"2024-03-26T08:31:37.488781+0000","last_became_active":"2024-03-26T08:31:22.387142+0000","last_became_peered":"2024-03-26T08:31:22.387142+0000","last_unstale":"2024-03-26T08:31:37.488781+0000","last_undegraded":"2024-03-26T08:31:37.488781+0000","last_fullsized":"2024-03-26T08:31:37.488781+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:13:32.307195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281284,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":23939,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":15,"num_read_kb":21,"num_write":58,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":39819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350998,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":503251,"data_compressed":20702,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612637,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1093632,"data_stored":1116932,"data_compressed":56479,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776165,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":499715,"data_compressed":20702,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037804,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":499715,"data_compressed":20702,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299442,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":1090083,"data_compressed":56479,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462969,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":499133,"data_compressed":20702,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":502975,"data_compressed":20702,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":499223,"data_compressed":20702,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":503009,"data_compressed":20702,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149775,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":499237,"data_compressed":20702,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888135,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":499155,"data_compressed":20702,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724608,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1056768,"data_stored":1089683,"data_compressed":56479,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34133,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34133,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34133,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35777,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35777,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35777,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35777,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35777,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":10442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":23335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T08:31:53.379 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-26T08:31:53.380 INFO:tasks.cephadm:Setting up client nodes... 2024-03-26T08:31:53.380 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-26T08:31:53.873 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:31:53.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:53 smithi043 ceph-mon[31929]: pgmap v390: 129 pgs: 129 active+clean; 600 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 2 op/s; 1 B/s, 0 objects/s recovering 2024-03-26T08:31:53.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:53 smithi043 ceph-mon[31929]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:31:53.950 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:53 smithi183 ceph-mon[35818]: pgmap v390: 129 pgs: 129 active+clean; 600 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 2 op/s; 1 B/s, 0 objects/s recovering 2024-03-26T08:31:53.950 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:53 smithi183 ceph-mon[35818]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:31:54.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:53 smithi052 ceph-mon[35071]: pgmap v390: 129 pgs: 129 active+clean; 600 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 2 op/s; 1 B/s, 0 objects/s recovering 2024-03-26T08:31:54.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:53 smithi052 ceph-mon[35071]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:31:54.885 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:54 smithi043 ceph-mon[31929]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:31:55.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:54 smithi183 ceph-mon[35818]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:31:55.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:54 smithi052 ceph-mon[35071]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:31:55.751 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:55 smithi043 ceph-mon[31929]: pgmap v391: 129 pgs: 129 active+clean; 600 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.1 KiB/s wr, 2 op/s 2024-03-26T08:31:56.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:55 smithi183 ceph-mon[35818]: pgmap v391: 129 pgs: 129 active+clean; 600 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.1 KiB/s wr, 2 op/s 2024-03-26T08:31:56.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:55 smithi052 ceph-mon[35071]: pgmap v391: 129 pgs: 129 active+clean; 600 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.1 KiB/s wr, 2 op/s 2024-03-26T08:31:56.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.370+0000 7f446a000700 1 -- 172.21.15.43:0/1252310565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4464076b40 msgr2=0x7f4464076f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:56.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.370+0000 7f446a000700 1 --2- 172.21.15.43:0/1252310565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4464076b40 0x7f4464076f60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4454009990 tx=0x7f4454009ca0 comp rx=0 tx=0).stop 2024-03-26T08:31:56.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.370+0000 7f446a000700 1 -- 172.21.15.43:0/1252310565 shutdown_connections 2024-03-26T08:31:56.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.370+0000 7f446a000700 1 --2- 172.21.15.43:0/1252310565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4464078750 0x7f4464078fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:56.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.370+0000 7f446a000700 1 --2- 172.21.15.43:0/1252310565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4464076b40 0x7f4464076f60 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:56.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.370+0000 7f446a000700 1 --2- 172.21.15.43:0/1252310565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4464077d90 0x7f4464078210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:56.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.370+0000 7f446a000700 1 -- 172.21.15.43:0/1252310565 >> 172.21.15.43:0/1252310565 conn(0x7f44640ff9b0 msgr2=0x7f4464101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:56.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.371+0000 7f446a000700 1 -- 172.21.15.43:0/1252310565 shutdown_connections 2024-03-26T08:31:56.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.371+0000 7f446a000700 1 -- 172.21.15.43:0/1252310565 wait complete. 2024-03-26T08:31:56.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.372+0000 7f446a000700 1 Processor -- start 2024-03-26T08:31:56.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.372+0000 7f446a000700 1 -- start start 2024-03-26T08:31:56.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.372+0000 7f446a000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4464076b40 0x7f446419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:56.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.372+0000 7f446a000700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4464077d90 0x7f446419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:56.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.372+0000 7f446a000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4464078750 0x7f44641a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:56.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.372+0000 7f446a000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4464114290 con 0x7f4464076b40 2024-03-26T08:31:56.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.372+0000 7f446a000700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4464114110 con 0x7f4464077d90 2024-03-26T08:31:56.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.372+0000 7f446a000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4464114410 con 0x7f4464078750 2024-03-26T08:31:56.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.373+0000 7f4463400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4464076b40 0x7f446419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:56.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.373+0000 7f4463400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4464076b40 0x7f446419e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55696/0 (socket says 172.21.15.43:55696) 2024-03-26T08:31:56.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.373+0000 7f4463400700 1 -- 172.21.15.43:0/283897156 learned_addr learned my addr 172.21.15.43:0/283897156 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:31:56.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.373+0000 7f4462a00700 1 --2- 172.21.15.43:0/283897156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4464077d90 0x7f446419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:56.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.373+0000 7f4463e00700 1 --2- 172.21.15.43:0/283897156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4464078750 0x7f44641a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:56.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.373+0000 7f4463400700 1 -- 172.21.15.43:0/283897156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4464078750 msgr2=0x7f44641a5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:56.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.373+0000 7f4463400700 1 --2- 172.21.15.43:0/283897156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4464078750 0x7f44641a5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:56.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.373+0000 7f4463400700 1 -- 172.21.15.43:0/283897156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4464077d90 msgr2=0x7f446419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:56.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.373+0000 7f4463400700 1 --2- 172.21.15.43:0/283897156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4464077d90 0x7f446419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:56.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.373+0000 7f4463400700 1 -- 172.21.15.43:0/283897156 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44641a6460 con 0x7f4464076b40 2024-03-26T08:31:56.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.374+0000 7f4463e00700 1 --2- 172.21.15.43:0/283897156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4464078750 0x7f44641a5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:31:56.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.374+0000 7f4463400700 1 --2- 172.21.15.43:0/283897156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4464076b40 0x7f446419e800 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4454000c00 tx=0x7f44540047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:56.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.374+0000 7f445be00700 1 -- 172.21.15.43:0/283897156 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4454027070 con 0x7f4464076b40 2024-03-26T08:31:56.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.374+0000 7f445be00700 1 -- 172.21.15.43:0/283897156 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4454022470 con 0x7f4464076b40 2024-03-26T08:31:56.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.374+0000 7f445be00700 1 -- 172.21.15.43:0/283897156 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445400f6c0 con 0x7f4464076b40 2024-03-26T08:31:56.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.374+0000 7f446a000700 1 -- 172.21.15.43:0/283897156 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44641a66f0 con 0x7f4464076b40 2024-03-26T08:31:56.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.374+0000 7f446a000700 1 -- 172.21.15.43:0/283897156 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44641a6ba0 con 0x7f4464076b40 2024-03-26T08:31:56.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.376+0000 7f446a000700 1 -- 172.21.15.43:0/283897156 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44640506a0 con 0x7f4464076b40 2024-03-26T08:31:56.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.378+0000 7f445be00700 1 -- 172.21.15.43:0/283897156 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f445402a070 con 0x7f4464076b40 2024-03-26T08:31:56.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.379+0000 7f445be00700 1 --2- 172.21.15.43:0/283897156 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f44480a2a50 0x7f44480a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:56.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.379+0000 7f445be00700 1 -- 172.21.15.43:0/283897156 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f44540cfd10 con 0x7f4464076b40 2024-03-26T08:31:56.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.379+0000 7f4462a00700 1 --2- 172.21.15.43:0/283897156 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f44480a2a50 0x7f44480a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:56.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.380+0000 7f4462a00700 1 --2- 172.21.15.43:0/283897156 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f44480a2a50 0x7f44480a4f10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4450005950 tx=0x7f44500058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:56.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.384+0000 7f445be00700 1 -- 172.21.15.43:0/283897156 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f445405e740 con 0x7f4464076b40 2024-03-26T08:31:56.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.703+0000 7f446a000700 1 -- 172.21.15.43:0/283897156 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f446402da80 con 0x7f4464076b40 2024-03-26T08:31:56.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.707+0000 7f445be00700 1 -- 172.21.15.43:0/283897156 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v33) v1 ==== 170+0+59 (secure 0 0 0) 0x7f44540bae20 con 0x7f4464076b40 2024-03-26T08:31:56.700 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-03-26T08:31:56.700 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQD8hwJmvqYEKhAARZu0eUjZgKccsULSxvCpyA== 2024-03-26T08:31:56.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.710+0000 7f446a000700 1 -- 172.21.15.43:0/283897156 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f44480a2a50 msgr2=0x7f44480a4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:56.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.710+0000 7f446a000700 1 --2- 172.21.15.43:0/283897156 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f44480a2a50 0x7f44480a4f10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4450005950 tx=0x7f44500058e0 comp rx=0 tx=0).stop 2024-03-26T08:31:56.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.710+0000 7f446a000700 1 -- 172.21.15.43:0/283897156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4464076b40 msgr2=0x7f446419e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:56.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.710+0000 7f446a000700 1 --2- 172.21.15.43:0/283897156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4464076b40 0x7f446419e800 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4454000c00 tx=0x7f44540047b0 comp rx=0 tx=0).stop 2024-03-26T08:31:56.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.710+0000 7f446a000700 1 -- 172.21.15.43:0/283897156 shutdown_connections 2024-03-26T08:31:56.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.710+0000 7f446a000700 1 --2- 172.21.15.43:0/283897156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4464078750 0x7f44641a5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:56.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.710+0000 7f446a000700 1 --2- 172.21.15.43:0/283897156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4464076b40 0x7f446419e800 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:56.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.710+0000 7f446a000700 1 --2- 172.21.15.43:0/283897156 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f44480a2a50 0x7f44480a4f10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:56.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.710+0000 7f446a000700 1 --2- 172.21.15.43:0/283897156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4464077d90 0x7f446419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:56.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.710+0000 7f446a000700 1 -- 172.21.15.43:0/283897156 >> 172.21.15.43:0/283897156 conn(0x7f44640ff9b0 msgr2=0x7f44641014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:56.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.711+0000 7f446a000700 1 -- 172.21.15.43:0/283897156 shutdown_connections 2024-03-26T08:31:56.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:31:56.711+0000 7f446a000700 1 -- 172.21.15.43:0/283897156 wait complete. 2024-03-26T08:31:57.562 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-26T08:31:57.562 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-26T08:31:57.562 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-26T08:31:57.733 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-26T08:31:57.733 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-26T08:31:57.734 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph mgr dump --format=json 2024-03-26T08:31:57.863 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:57 smithi043 ceph-mon[31929]: pgmap v392: 129 pgs: 129 active+clean; 624 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-03-26T08:31:57.863 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:57 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/283897156' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T08:31:57.863 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:57 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/283897156' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-26T08:31:58.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:57 smithi183 ceph-mon[35818]: pgmap v392: 129 pgs: 129 active+clean; 624 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-03-26T08:31:58.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:57 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/283897156' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T08:31:58.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:57 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/283897156' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-26T08:31:58.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:57 smithi052 ceph-mon[35071]: pgmap v392: 129 pgs: 129 active+clean; 624 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-03-26T08:31:58.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:57 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/283897156' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T08:31:58.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:57 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/283897156' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-26T08:31:58.254 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:31:59.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:31:59 smithi043 ceph-mon[31929]: pgmap v393: 129 pgs: 129 active+clean; 624 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-03-26T08:32:00.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:31:59 smithi183 ceph-mon[35818]: pgmap v393: 129 pgs: 129 active+clean; 624 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-03-26T08:32:00.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:31:59 smithi052 ceph-mon[35071]: pgmap v393: 129 pgs: 129 active+clean; 624 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-03-26T08:32:00.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.770+0000 7f3ace600700 1 -- 172.21.15.43:0/865172052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ac81045e0 msgr2=0x7f3ac81049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:00.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.770+0000 7f3ace600700 1 --2- 172.21.15.43:0/865172052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ac81045e0 0x7f3ac81049c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80099c0 tx=0x7f3ab8009cd0 comp rx=0 tx=0).stop 2024-03-26T08:32:00.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.770+0000 7f3ace600700 1 -- 172.21.15.43:0/865172052 shutdown_connections 2024-03-26T08:32:00.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.770+0000 7f3ace600700 1 --2- 172.21.15.43:0/865172052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ac81096c0 0x7f3ac8111290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:00.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.770+0000 7f3ace600700 1 --2- 172.21.15.43:0/865172052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ac81045e0 0x7f3ac81049c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:00.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.770+0000 7f3ace600700 1 --2- 172.21.15.43:0/865172052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ac8104f90 0x7f3ac8108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:00.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.770+0000 7f3ace600700 1 -- 172.21.15.43:0/865172052 >> 172.21.15.43:0/865172052 conn(0x7f3ac80ffe70 msgr2=0x7f3ac8102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:00.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.770+0000 7f3ace600700 1 -- 172.21.15.43:0/865172052 shutdown_connections 2024-03-26T08:32:00.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.770+0000 7f3ace600700 1 -- 172.21.15.43:0/865172052 wait complete. 2024-03-26T08:32:00.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.771+0000 7f3ace600700 1 Processor -- start 2024-03-26T08:32:00.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.771+0000 7f3ace600700 1 -- start start 2024-03-26T08:32:00.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.772+0000 7f3ace600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ac81045e0 0x7f3ac81a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:00.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.772+0000 7f3ace600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ac8104f90 0x7f3ac81a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:00.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.772+0000 7f3ace600700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ac81096c0 0x7f3ac819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:00.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.772+0000 7f3ace600700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac81145a0 con 0x7f3ac81045e0 2024-03-26T08:32:00.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.772+0000 7f3ace600700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac8114420 con 0x7f3ac81096c0 2024-03-26T08:32:00.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.772+0000 7f3ace600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac8114720 con 0x7f3ac8104f90 2024-03-26T08:32:00.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.772+0000 7f3ac7400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ac81045e0 0x7f3ac81a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:00.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.772+0000 7f3ac7400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ac81045e0 0x7f3ac81a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55720/0 (socket says 172.21.15.43:55720) 2024-03-26T08:32:00.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.772+0000 7f3ac7400700 1 -- 172.21.15.43:0/1228287871 learned_addr learned my addr 172.21.15.43:0/1228287871 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:32:00.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.772+0000 7f3ac6a00700 1 --2- 172.21.15.43:0/1228287871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ac8104f90 0x7f3ac81a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:00.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.772+0000 7f3ac7e00700 1 --2- 172.21.15.43:0/1228287871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ac81096c0 0x7f3ac819a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:00.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.773+0000 7f3ac7400700 1 -- 172.21.15.43:0/1228287871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ac8104f90 msgr2=0x7f3ac81a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:00.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.773+0000 7f3ac7400700 1 --2- 172.21.15.43:0/1228287871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ac8104f90 0x7f3ac81a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:00.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.773+0000 7f3ac7400700 1 -- 172.21.15.43:0/1228287871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ac81096c0 msgr2=0x7f3ac819a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:00.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.773+0000 7f3ac7400700 1 --2- 172.21.15.43:0/1228287871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ac81096c0 0x7f3ac819a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:00.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.773+0000 7f3ac7400700 1 -- 172.21.15.43:0/1228287871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac819b020 con 0x7f3ac81045e0 2024-03-26T08:32:00.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.773+0000 7f3ac7400700 1 --2- 172.21.15.43:0/1228287871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ac81045e0 0x7f3ac81a06c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8005b40 tx=0x7f3ab8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:00.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.773+0000 7f3abfe00700 1 -- 172.21.15.43:0/1228287871 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8027070 con 0x7f3ac81045e0 2024-03-26T08:32:00.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.773+0000 7f3abfe00700 1 -- 172.21.15.43:0/1228287871 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3ab8003fd0 con 0x7f3ac81045e0 2024-03-26T08:32:00.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.773+0000 7f3ace600700 1 -- 172.21.15.43:0/1228287871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac819b2b0 con 0x7f3ac81045e0 2024-03-26T08:32:00.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.774+0000 7f3abfe00700 1 -- 172.21.15.43:0/1228287871 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8022c60 con 0x7f3ac81045e0 2024-03-26T08:32:00.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.774+0000 7f3ace600700 1 -- 172.21.15.43:0/1228287871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac81a7520 con 0x7f3ac81045e0 2024-03-26T08:32:00.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.775+0000 7f3ace600700 1 -- 172.21.15.43:0/1228287871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac8105ac0 con 0x7f3ac81045e0 2024-03-26T08:32:00.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.777+0000 7f3abfe00700 1 -- 172.21.15.43:0/1228287871 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f3ab802a070 con 0x7f3ac81045e0 2024-03-26T08:32:00.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.778+0000 7f3abfe00700 1 --2- 172.21.15.43:0/1228287871 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3aac0a29a0 0x7f3aac0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:00.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.778+0000 7f3abfe00700 1 -- 172.21.15.43:0/1228287871 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f3ab8022e00 con 0x7f3ac81045e0 2024-03-26T08:32:00.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.779+0000 7f3ac6a00700 1 --2- 172.21.15.43:0/1228287871 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3aac0a29a0 0x7f3aac0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:00.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.779+0000 7f3ac6a00700 1 --2- 172.21.15.43:0/1228287871 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3aac0a29a0 0x7f3aac0a4e60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4005950 tx=0x7f3ab40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:00.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:00.783+0000 7f3abfe00700 1 -- 172.21.15.43:0/1228287871 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3ab805e740 con 0x7f3ac81045e0 2024-03-26T08:32:01.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:01.077+0000 7f3ace600700 1 -- 172.21.15.43:0/1228287871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f3ac819bd70 con 0x7f3ac81045e0 2024-03-26T08:32:01.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:01.086+0000 7f3abfe00700 1 -- 172.21.15.43:0/1228287871 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+262661 (secure 0 0 0) 0x7f3ab8013070 con 0x7f3ac81045e0 2024-03-26T08:32:01.083 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:32:01.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:01.091+0000 7f3ace600700 1 -- 172.21.15.43:0/1228287871 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3aac0a29a0 msgr2=0x7f3aac0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:01.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:01.091+0000 7f3ace600700 1 --2- 172.21.15.43:0/1228287871 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3aac0a29a0 0x7f3aac0a4e60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4005950 tx=0x7f3ab40058e0 comp rx=0 tx=0).stop 2024-03-26T08:32:01.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:01.091+0000 7f3ace600700 1 -- 172.21.15.43:0/1228287871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ac81045e0 msgr2=0x7f3ac81a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:01.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:01.091+0000 7f3ace600700 1 --2- 172.21.15.43:0/1228287871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ac81045e0 0x7f3ac81a06c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8005b40 tx=0x7f3ab8005e80 comp rx=0 tx=0).stop 2024-03-26T08:32:01.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:01.091+0000 7f3ace600700 1 -- 172.21.15.43:0/1228287871 shutdown_connections 2024-03-26T08:32:01.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:01.091+0000 7f3ace600700 1 --2- 172.21.15.43:0/1228287871 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3aac0a29a0 0x7f3aac0a4e60 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:01.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:01.091+0000 7f3ace600700 1 --2- 172.21.15.43:0/1228287871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ac81096c0 0x7f3ac819a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:01.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:01.091+0000 7f3ace600700 1 --2- 172.21.15.43:0/1228287871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ac81045e0 0x7f3ac81a06c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:01.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:01.091+0000 7f3ace600700 1 --2- 172.21.15.43:0/1228287871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ac8104f90 0x7f3ac81a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:01.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:01.091+0000 7f3ace600700 1 -- 172.21.15.43:0/1228287871 >> 172.21.15.43:0/1228287871 conn(0x7f3ac80ffe70 msgr2=0x7f3ac8102140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:01.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:01.091+0000 7f3ace600700 1 -- 172.21.15.43:0/1228287871 shutdown_connections 2024-03-26T08:32:01.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:01.091+0000 7f3ace600700 1 -- 172.21.15.43:0/1228287871 wait complete. 2024-03-26T08:32:01.950 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:01 smithi043 ceph-mon[31929]: pgmap v394: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.0 KiB/s wr, 5 op/s 2024-03-26T08:32:01.951 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:01 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1228287871' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-26T08:32:02.011 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3390816575},{"type":"v1","addr":"172.21.15.43:6801","nonce":3390816575}]},"active_addr":"172.21.15.43:6801/3390816575","active_change":"2024-03-26T08:21:36.149425+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":34112,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.43:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":1151707060}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":1743507119}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":3641175829}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":2978127728}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":3264044308}]}]} 2024-03-26T08:32:02.016 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-26T08:32:02.016 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-26T08:32:02.017 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-26T08:32:02.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:01 smithi183 ceph-mon[35818]: pgmap v394: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.0 KiB/s wr, 5 op/s 2024-03-26T08:32:02.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:01 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1228287871' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-26T08:32:02.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:01 smithi052 ceph-mon[35071]: pgmap v394: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.0 KiB/s wr, 5 op/s 2024-03-26T08:32:02.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:01 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1228287871' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-26T08:32:02.474 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:32:04.018 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:03 smithi043 ceph-mon[31929]: pgmap v395: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.1 KiB/s wr, 4 op/s 2024-03-26T08:32:04.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:03 smithi183 ceph-mon[35818]: pgmap v395: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.1 KiB/s wr, 4 op/s 2024-03-26T08:32:04.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:03 smithi052 ceph-mon[35071]: pgmap v395: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.1 KiB/s wr, 4 op/s 2024-03-26T08:32:04.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.941+0000 7f565ae00700 1 -- 172.21.15.43:0/3011207371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5654104b90 msgr2=0x7f5654105020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:04.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.941+0000 7f565ae00700 1 --2- 172.21.15.43:0/3011207371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5654104b90 0x7f5654105020 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f564c009990 tx=0x7f564c009ca0 comp rx=0 tx=0).stop 2024-03-26T08:32:04.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.941+0000 7f565ae00700 1 -- 172.21.15.43:0/3011207371 shutdown_connections 2024-03-26T08:32:04.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.941+0000 7f565ae00700 1 --2- 172.21.15.43:0/3011207371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5654104b90 0x7f5654105020 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:04.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.941+0000 7f565ae00700 1 --2- 172.21.15.43:0/3011207371 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5654107080 0x7f5654069e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:04.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.941+0000 7f565ae00700 1 --2- 172.21.15.43:0/3011207371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f565406a380 0x7f565406a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:04.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.941+0000 7f565ae00700 1 -- 172.21.15.43:0/3011207371 >> 172.21.15.43:0/3011207371 conn(0x7f5654077fc0 msgr2=0x7f56540783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:04.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.941+0000 7f565ae00700 1 -- 172.21.15.43:0/3011207371 shutdown_connections 2024-03-26T08:32:04.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.941+0000 7f565ae00700 1 -- 172.21.15.43:0/3011207371 wait complete. 2024-03-26T08:32:04.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.942+0000 7f565ae00700 1 Processor -- start 2024-03-26T08:32:04.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.942+0000 7f565ae00700 1 -- start start 2024-03-26T08:32:04.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.943+0000 7f565ae00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f565406a380 0x7f565419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:04.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.943+0000 7f565ae00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5654104b90 0x7f565419f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:04.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.943+0000 7f565ae00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5654107080 0x7f56541a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:04.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.943+0000 7f565ae00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5654118930 con 0x7f5654107080 2024-03-26T08:32:04.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.943+0000 7f565ae00700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56541187b0 con 0x7f565406a380 2024-03-26T08:32:04.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.943+0000 7f565ae00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5654118ab0 con 0x7f5654104b90 2024-03-26T08:32:04.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.943+0000 7f5658e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5654107080 0x7f56541a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:04.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.943+0000 7f5658e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5654107080 0x7f56541a36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60960/0 (socket says 172.21.15.43:60960) 2024-03-26T08:32:04.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.943+0000 7f5658e00700 1 -- 172.21.15.43:0/915760307 learned_addr learned my addr 172.21.15.43:0/915760307 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:32:04.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.943+0000 7f5653e00700 1 --2- 172.21.15.43:0/915760307 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f565406a380 0x7f565419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:04.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.944+0000 7f5653400700 1 --2- 172.21.15.43:0/915760307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5654104b90 0x7f565419f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:04.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.944+0000 7f5658e00700 1 -- 172.21.15.43:0/915760307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5654104b90 msgr2=0x7f565419f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:04.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.944+0000 7f5658e00700 1 --2- 172.21.15.43:0/915760307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5654104b90 0x7f565419f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:04.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.944+0000 7f5658e00700 1 -- 172.21.15.43:0/915760307 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f565406a380 msgr2=0x7f565419ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:04.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.944+0000 7f5658e00700 1 --2- 172.21.15.43:0/915760307 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f565406a380 0x7f565419ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:04.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.944+0000 7f5658e00700 1 -- 172.21.15.43:0/915760307 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56541a3e00 con 0x7f5654107080 2024-03-26T08:32:04.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.944+0000 7f5653400700 1 --2- 172.21.15.43:0/915760307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5654104b90 0x7f565419f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:32:04.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.944+0000 7f5658e00700 1 --2- 172.21.15.43:0/915760307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5654107080 0x7f56541a36c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f564c005b40 tx=0x7f564c009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:04.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.945+0000 7f5650c00700 1 -- 172.21.15.43:0/915760307 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c027070 con 0x7f5654107080 2024-03-26T08:32:04.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.945+0000 7f5650c00700 1 -- 172.21.15.43:0/915760307 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f564c003f90 con 0x7f5654107080 2024-03-26T08:32:04.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.945+0000 7f5650c00700 1 -- 172.21.15.43:0/915760307 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c022d00 con 0x7f5654107080 2024-03-26T08:32:04.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.945+0000 7f565ae00700 1 -- 172.21.15.43:0/915760307 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56541a4090 con 0x7f5654107080 2024-03-26T08:32:04.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.945+0000 7f565ae00700 1 -- 172.21.15.43:0/915760307 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56541ab970 con 0x7f5654107080 2024-03-26T08:32:04.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.946+0000 7f565ae00700 1 -- 172.21.15.43:0/915760307 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5654068e50 con 0x7f5654107080 2024-03-26T08:32:04.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.948+0000 7f5650c00700 1 -- 172.21.15.43:0/915760307 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f564c02a070 con 0x7f5654107080 2024-03-26T08:32:04.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.949+0000 7f5650c00700 1 --2- 172.21.15.43:0/915760307 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f56380a29a0 0x7f56380a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:04.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.949+0000 7f5650c00700 1 -- 172.21.15.43:0/915760307 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f564c0cfe70 con 0x7f5654107080 2024-03-26T08:32:04.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.950+0000 7f5653e00700 1 --2- 172.21.15.43:0/915760307 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f56380a29a0 0x7f56380a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:04.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.950+0000 7f5653e00700 1 --2- 172.21.15.43:0/915760307 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f56380a29a0 0x7f56380a4e60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5644009c90 tx=0x7f5644009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:04.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:04.954+0000 7f5650c00700 1 -- 172.21.15.43:0/915760307 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f564c05e740 con 0x7f5654107080 2024-03-26T08:32:05.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:05.188+0000 7f565ae00700 1 -- 172.21.15.43:0/915760307 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56541abb40 con 0x7f5654107080 2024-03-26T08:32:05.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:05.191+0000 7f5650c00700 1 -- 172.21.15.43:0/915760307 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24576 (secure 0 0 0) 0x7f564c013070 con 0x7f5654107080 2024-03-26T08:32:05.183 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:32:05.183 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":92,"fsid":"88759420-eb49-11ee-95d0-87774f69a715","created":"2024-03-26T08:20:27.817107+0000","modified":"2024-03-26T08:31:37.455519+0000","last_up_change":"2024-03-26T08:30:10.520976+0000","last_in_change":"2024-03-26T08:29:46.394360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:24:54.852427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:31:19.924902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:31:21.323881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T08:31:24.951015+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"28670673-9193-4fa3-9cc0-c1a5a6b27284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6803","nonce":936601146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6805","nonce":936601146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6809","nonce":936601146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6807","nonce":936601146}]},"public_addr":"172.21.15.43:6803/936601146","cluster_addr":"172.21.15.43:6805/936601146","heartbeat_back_addr":"172.21.15.43:6809/936601146","heartbeat_front_addr":"172.21.15.43:6807/936601146","state":["exists","up"]},{"osd":1,"uuid":"fa8ad6b6-277a-42b8-b0f8-b36fc219de2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6801","nonce":3268412328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6803","nonce":3268412328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6807","nonce":3268412328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6805","nonce":3268412328}]},"public_addr":"172.21.15.52:6801/3268412328","cluster_addr":"172.21.15.52:6803/3268412328","heartbeat_back_addr":"172.21.15.52:6807/3268412328","heartbeat_front_addr":"172.21.15.52:6805/3268412328","state":["exists","up"]},{"osd":2,"uuid":"ec0039c3-f9bd-419a-81de-fa4418359808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6801","nonce":2103180908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6803","nonce":2103180908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6807","nonce":2103180908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6805","nonce":2103180908}]},"public_addr":"172.21.15.183:6801/2103180908","cluster_addr":"172.21.15.183:6803/2103180908","heartbeat_back_addr":"172.21.15.183:6807/2103180908","heartbeat_front_addr":"172.21.15.183:6805/2103180908","state":["exists","up"]},{"osd":3,"uuid":"e7dcbf00-6586-4664-a69e-3cb0df8dfd30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6811","nonce":3722846033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6813","nonce":3722846033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6817","nonce":3722846033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6815","nonce":3722846033}]},"public_addr":"172.21.15.43:6811/3722846033","cluster_addr":"172.21.15.43:6813/3722846033","heartbeat_back_addr":"172.21.15.43:6817/3722846033","heartbeat_front_addr":"172.21.15.43:6815/3722846033","state":["exists","up"]},{"osd":4,"uuid":"db089a49-5e98-432d-9687-cdc044114c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6809","nonce":4028436523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6811","nonce":4028436523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6815","nonce":4028436523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6813","nonce":4028436523}]},"public_addr":"172.21.15.52:6809/4028436523","cluster_addr":"172.21.15.52:6811/4028436523","heartbeat_back_addr":"172.21.15.52:6815/4028436523","heartbeat_front_addr":"172.21.15.52:6813/4028436523","state":["exists","up"]},{"osd":5,"uuid":"556ac804-02ea-423b-a599-55baf5be4ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6809","nonce":787132252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6811","nonce":787132252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6815","nonce":787132252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6813","nonce":787132252}]},"public_addr":"172.21.15.183:6809/787132252","cluster_addr":"172.21.15.183:6811/787132252","heartbeat_back_addr":"172.21.15.183:6815/787132252","heartbeat_front_addr":"172.21.15.183:6813/787132252","state":["exists","up"]},{"osd":6,"uuid":"3de2a49d-6cef-4580-9d8a-f90cdfcecd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6819","nonce":4206861158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6821","nonce":4206861158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6825","nonce":4206861158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6823","nonce":4206861158}]},"public_addr":"172.21.15.43:6819/4206861158","cluster_addr":"172.21.15.43:6821/4206861158","heartbeat_back_addr":"172.21.15.43:6825/4206861158","heartbeat_front_addr":"172.21.15.43:6823/4206861158","state":["exists","up"]},{"osd":7,"uuid":"a3b832f7-7d7d-4a62-b679-db6da98275d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6817","nonce":3452064146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6819","nonce":3452064146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6823","nonce":3452064146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6821","nonce":3452064146}]},"public_addr":"172.21.15.52:6817/3452064146","cluster_addr":"172.21.15.52:6819/3452064146","heartbeat_back_addr":"172.21.15.52:6823/3452064146","heartbeat_front_addr":"172.21.15.52:6821/3452064146","state":["exists","up"]},{"osd":8,"uuid":"ddc6b07f-d703-414d-a59a-02316376b13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6817","nonce":1023131725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6819","nonce":1023131725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6823","nonce":1023131725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6821","nonce":1023131725}]},"public_addr":"172.21.15.183:6817/1023131725","cluster_addr":"172.21.15.183:6819/1023131725","heartbeat_back_addr":"172.21.15.183:6823/1023131725","heartbeat_front_addr":"172.21.15.183:6821/1023131725","state":["exists","up"]},{"osd":9,"uuid":"ac16bb0e-6232-45c8-b16a-4d878874c588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6827","nonce":3935195657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6829","nonce":3935195657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6833","nonce":3935195657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6831","nonce":3935195657}]},"public_addr":"172.21.15.43:6827/3935195657","cluster_addr":"172.21.15.43:6829/3935195657","heartbeat_back_addr":"172.21.15.43:6833/3935195657","heartbeat_front_addr":"172.21.15.43:6831/3935195657","state":["exists","up"]},{"osd":10,"uuid":"fd839188-f398-4cff-aa7b-36522ff1a8a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6825","nonce":377628320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6827","nonce":377628320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6831","nonce":377628320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6829","nonce":377628320}]},"public_addr":"172.21.15.52:6825/377628320","cluster_addr":"172.21.15.52:6827/377628320","heartbeat_back_addr":"172.21.15.52:6831/377628320","heartbeat_front_addr":"172.21.15.52:6829/377628320","state":["exists","up"]},{"osd":11,"uuid":"a40282a0-0c64-49ee-b64e-6f3f8119e827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6825","nonce":1373086433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6827","nonce":1373086433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6831","nonce":1373086433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6829","nonce":1373086433}]},"public_addr":"172.21.15.183:6825/1373086433","cluster_addr":"172.21.15.183:6827/1373086433","heartbeat_back_addr":"172.21.15.183:6831/1373086433","heartbeat_front_addr":"172.21.15.183:6829/1373086433","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:23:43.478905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:17.573532+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:50.286610+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:25:27.407663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:03.081294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:36.057416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:12.062013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:47.032271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:21.023201+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:57.364739+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:29:34.039995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:30:08.827577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6801/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/4240775295":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/510203140":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/4251957687":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:6801/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1885244665":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/298320867":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1301092625":"2024-03-27T08:21:36.149092+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:32:05.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:05.194+0000 7f565ae00700 1 -- 172.21.15.43:0/915760307 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f56380a29a0 msgr2=0x7f56380a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:05.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:05.194+0000 7f565ae00700 1 --2- 172.21.15.43:0/915760307 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f56380a29a0 0x7f56380a4e60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5644009c90 tx=0x7f5644009040 comp rx=0 tx=0).stop 2024-03-26T08:32:05.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:05.194+0000 7f565ae00700 1 -- 172.21.15.43:0/915760307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5654107080 msgr2=0x7f56541a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:05.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:05.194+0000 7f565ae00700 1 --2- 172.21.15.43:0/915760307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5654107080 0x7f56541a36c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f564c005b40 tx=0x7f564c009f70 comp rx=0 tx=0).stop 2024-03-26T08:32:05.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:05.194+0000 7f565ae00700 1 -- 172.21.15.43:0/915760307 shutdown_connections 2024-03-26T08:32:05.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:05.194+0000 7f565ae00700 1 --2- 172.21.15.43:0/915760307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5654107080 0x7f56541a36c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:05.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:05.194+0000 7f565ae00700 1 --2- 172.21.15.43:0/915760307 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f56380a29a0 0x7f56380a4e60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:05.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:05.194+0000 7f565ae00700 1 --2- 172.21.15.43:0/915760307 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f565406a380 0x7f565419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:05.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:05.194+0000 7f565ae00700 1 --2- 172.21.15.43:0/915760307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5654104b90 0x7f565419f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:05.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:05.194+0000 7f565ae00700 1 -- 172.21.15.43:0/915760307 >> 172.21.15.43:0/915760307 conn(0x7f5654077fc0 msgr2=0x7f5654103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:05.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:05.194+0000 7f565ae00700 1 -- 172.21.15.43:0/915760307 shutdown_connections 2024-03-26T08:32:05.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:05.194+0000 7f565ae00700 1 -- 172.21.15.43:0/915760307 wait complete. 2024-03-26T08:32:05.843 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:05 smithi043 ceph-mon[31929]: pgmap v396: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.1 KiB/s wr, 4 op/s 2024-03-26T08:32:05.843 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:05 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/915760307' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:32:06.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:05 smithi183 ceph-mon[35818]: pgmap v396: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.1 KiB/s wr, 4 op/s 2024-03-26T08:32:06.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:05 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/915760307' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:32:06.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:05 smithi052 ceph-mon[35071]: pgmap v396: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.1 KiB/s wr, 4 op/s 2024-03-26T08:32:06.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:05 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/915760307' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:32:06.159 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-26T08:32:06.160 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-26T08:32:06.591 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:32:08.020 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:07 smithi043 ceph-mon[31929]: pgmap v397: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.0 KiB/s wr, 4 op/s 2024-03-26T08:32:08.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:07 smithi183 ceph-mon[35818]: pgmap v397: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.0 KiB/s wr, 4 op/s 2024-03-26T08:32:08.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:07 smithi052 ceph-mon[35071]: pgmap v397: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.0 KiB/s wr, 4 op/s 2024-03-26T08:32:08.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.948+0000 7f40db400700 1 -- 172.21.15.43:0/3453196847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40d4104da0 msgr2=0x7f40d41112b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:08.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.948+0000 7f40db400700 1 --2- 172.21.15.43:0/3453196847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40d4104da0 0x7f40d41112b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f40cc00b0a0 tx=0x7f40cc00b3b0 comp rx=0 tx=0).stop 2024-03-26T08:32:08.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.948+0000 7f40db400700 1 -- 172.21.15.43:0/3453196847 shutdown_connections 2024-03-26T08:32:08.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.948+0000 7f40db400700 1 --2- 172.21.15.43:0/3453196847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40d4104da0 0x7f40d41112b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:08.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.948+0000 7f40db400700 1 --2- 172.21.15.43:0/3453196847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40d410a470 0x7f40d410a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:08.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.948+0000 7f40db400700 1 --2- 172.21.15.43:0/3453196847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40d41043e0 0x7f40d4104860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:08.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.948+0000 7f40db400700 1 -- 172.21.15.43:0/3453196847 >> 172.21.15.43:0/3453196847 conn(0x7f40d40fff30 msgr2=0x7f40d4102350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:08.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.948+0000 7f40db400700 1 -- 172.21.15.43:0/3453196847 shutdown_connections 2024-03-26T08:32:08.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.948+0000 7f40db400700 1 -- 172.21.15.43:0/3453196847 wait complete. 2024-03-26T08:32:08.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.950+0000 7f40db400700 1 Processor -- start 2024-03-26T08:32:08.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.950+0000 7f40db400700 1 -- start start 2024-03-26T08:32:08.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.950+0000 7f40db400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40d41043e0 0x7f40d41a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:08.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.950+0000 7f40db400700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40d4104da0 0x7f40d41a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:08.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.950+0000 7f40db400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40d410a470 0x7f40d419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:08.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.950+0000 7f40db400700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d41145f0 con 0x7f40d41043e0 2024-03-26T08:32:08.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.950+0000 7f40db400700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d4114470 con 0x7f40d4104da0 2024-03-26T08:32:08.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.950+0000 7f40db400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d4114770 con 0x7f40d410a470 2024-03-26T08:32:08.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.951+0000 7f40d8a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40d41043e0 0x7f40d41a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:08.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.951+0000 7f40d8a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40d41043e0 0x7f40d41a07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60976/0 (socket says 172.21.15.43:60976) 2024-03-26T08:32:08.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.951+0000 7f40d8a00700 1 -- 172.21.15.43:0/1292427289 learned_addr learned my addr 172.21.15.43:0/1292427289 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:32:08.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.951+0000 7f40d3e00700 1 --2- 172.21.15.43:0/1292427289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40d4104da0 0x7f40d41a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:08.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.951+0000 7f40d9400700 1 --2- 172.21.15.43:0/1292427289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40d410a470 0x7f40d419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:08.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.951+0000 7f40d8a00700 1 -- 172.21.15.43:0/1292427289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40d410a470 msgr2=0x7f40d419a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:08.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.951+0000 7f40d8a00700 1 --2- 172.21.15.43:0/1292427289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40d410a470 0x7f40d419a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:08.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.951+0000 7f40d8a00700 1 -- 172.21.15.43:0/1292427289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40d4104da0 msgr2=0x7f40d41a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:08.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.951+0000 7f40d8a00700 1 --2- 172.21.15.43:0/1292427289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40d4104da0 0x7f40d41a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:08.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.951+0000 7f40d8a00700 1 -- 172.21.15.43:0/1292427289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d419b0d0 con 0x7f40d41043e0 2024-03-26T08:32:08.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.952+0000 7f40d8a00700 1 --2- 172.21.15.43:0/1292427289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40d41043e0 0x7f40d41a07b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f40c800e9c0 tx=0x7f40c800ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:08.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.952+0000 7f40d1600700 1 -- 172.21.15.43:0/1292427289 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c800cbd0 con 0x7f40d41043e0 2024-03-26T08:32:08.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.952+0000 7f40db400700 1 -- 172.21.15.43:0/1292427289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d419b3c0 con 0x7f40d41043e0 2024-03-26T08:32:08.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.952+0000 7f40d1600700 1 -- 172.21.15.43:0/1292427289 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f40c800cd70 con 0x7f40d41043e0 2024-03-26T08:32:08.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.952+0000 7f40d1600700 1 -- 172.21.15.43:0/1292427289 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8010430 con 0x7f40d41043e0 2024-03-26T08:32:08.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.952+0000 7f40db400700 1 -- 172.21.15.43:0/1292427289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d41a7680 con 0x7f40d41043e0 2024-03-26T08:32:08.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.954+0000 7f40db400700 1 -- 172.21.15.43:0/1292427289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d4068e50 con 0x7f40d41043e0 2024-03-26T08:32:08.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.956+0000 7f40d1600700 1 -- 172.21.15.43:0/1292427289 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f40c8004750 con 0x7f40d41043e0 2024-03-26T08:32:08.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.957+0000 7f40d1600700 1 --2- 172.21.15.43:0/1292427289 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f40bc0a2920 0x7f40bc0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:08.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.957+0000 7f40d1600700 1 -- 172.21.15.43:0/1292427289 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f40c80cf310 con 0x7f40d41043e0 2024-03-26T08:32:08.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.957+0000 7f40d3e00700 1 --2- 172.21.15.43:0/1292427289 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f40bc0a2920 0x7f40bc0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:08.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.958+0000 7f40d3e00700 1 --2- 172.21.15.43:0/1292427289 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f40bc0a2920 0x7f40bc0a4de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f40d4109ae0 tx=0x7f40c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:08.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:08.962+0000 7f40d1600700 1 -- 172.21.15.43:0/1292427289 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f40c809beb0 con 0x7f40d41043e0 2024-03-26T08:32:09.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:09.191+0000 7f40db400700 1 -- 172.21.15.43:0/1292427289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40d419bea0 con 0x7f40d41043e0 2024-03-26T08:32:09.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:09.194+0000 7f40d1600700 1 -- 172.21.15.43:0/1292427289 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24576 (secure 0 0 0) 0x7f40c809b7b0 con 0x7f40d41043e0 2024-03-26T08:32:09.188 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:32:09.188 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":92,"fsid":"88759420-eb49-11ee-95d0-87774f69a715","created":"2024-03-26T08:20:27.817107+0000","modified":"2024-03-26T08:31:37.455519+0000","last_up_change":"2024-03-26T08:30:10.520976+0000","last_in_change":"2024-03-26T08:29:46.394360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:24:54.852427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:31:19.924902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:31:21.323881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T08:31:24.951015+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"28670673-9193-4fa3-9cc0-c1a5a6b27284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6803","nonce":936601146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6805","nonce":936601146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6809","nonce":936601146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6807","nonce":936601146}]},"public_addr":"172.21.15.43:6803/936601146","cluster_addr":"172.21.15.43:6805/936601146","heartbeat_back_addr":"172.21.15.43:6809/936601146","heartbeat_front_addr":"172.21.15.43:6807/936601146","state":["exists","up"]},{"osd":1,"uuid":"fa8ad6b6-277a-42b8-b0f8-b36fc219de2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6801","nonce":3268412328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6803","nonce":3268412328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6807","nonce":3268412328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6805","nonce":3268412328}]},"public_addr":"172.21.15.52:6801/3268412328","cluster_addr":"172.21.15.52:6803/3268412328","heartbeat_back_addr":"172.21.15.52:6807/3268412328","heartbeat_front_addr":"172.21.15.52:6805/3268412328","state":["exists","up"]},{"osd":2,"uuid":"ec0039c3-f9bd-419a-81de-fa4418359808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6801","nonce":2103180908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6803","nonce":2103180908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6807","nonce":2103180908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6805","nonce":2103180908}]},"public_addr":"172.21.15.183:6801/2103180908","cluster_addr":"172.21.15.183:6803/2103180908","heartbeat_back_addr":"172.21.15.183:6807/2103180908","heartbeat_front_addr":"172.21.15.183:6805/2103180908","state":["exists","up"]},{"osd":3,"uuid":"e7dcbf00-6586-4664-a69e-3cb0df8dfd30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6811","nonce":3722846033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6813","nonce":3722846033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6817","nonce":3722846033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6815","nonce":3722846033}]},"public_addr":"172.21.15.43:6811/3722846033","cluster_addr":"172.21.15.43:6813/3722846033","heartbeat_back_addr":"172.21.15.43:6817/3722846033","heartbeat_front_addr":"172.21.15.43:6815/3722846033","state":["exists","up"]},{"osd":4,"uuid":"db089a49-5e98-432d-9687-cdc044114c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6809","nonce":4028436523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6811","nonce":4028436523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6815","nonce":4028436523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6813","nonce":4028436523}]},"public_addr":"172.21.15.52:6809/4028436523","cluster_addr":"172.21.15.52:6811/4028436523","heartbeat_back_addr":"172.21.15.52:6815/4028436523","heartbeat_front_addr":"172.21.15.52:6813/4028436523","state":["exists","up"]},{"osd":5,"uuid":"556ac804-02ea-423b-a599-55baf5be4ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6809","nonce":787132252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6811","nonce":787132252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6815","nonce":787132252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6813","nonce":787132252}]},"public_addr":"172.21.15.183:6809/787132252","cluster_addr":"172.21.15.183:6811/787132252","heartbeat_back_addr":"172.21.15.183:6815/787132252","heartbeat_front_addr":"172.21.15.183:6813/787132252","state":["exists","up"]},{"osd":6,"uuid":"3de2a49d-6cef-4580-9d8a-f90cdfcecd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6819","nonce":4206861158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6821","nonce":4206861158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6825","nonce":4206861158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6823","nonce":4206861158}]},"public_addr":"172.21.15.43:6819/4206861158","cluster_addr":"172.21.15.43:6821/4206861158","heartbeat_back_addr":"172.21.15.43:6825/4206861158","heartbeat_front_addr":"172.21.15.43:6823/4206861158","state":["exists","up"]},{"osd":7,"uuid":"a3b832f7-7d7d-4a62-b679-db6da98275d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6817","nonce":3452064146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6819","nonce":3452064146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6823","nonce":3452064146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6821","nonce":3452064146}]},"public_addr":"172.21.15.52:6817/3452064146","cluster_addr":"172.21.15.52:6819/3452064146","heartbeat_back_addr":"172.21.15.52:6823/3452064146","heartbeat_front_addr":"172.21.15.52:6821/3452064146","state":["exists","up"]},{"osd":8,"uuid":"ddc6b07f-d703-414d-a59a-02316376b13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6817","nonce":1023131725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6819","nonce":1023131725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6823","nonce":1023131725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6821","nonce":1023131725}]},"public_addr":"172.21.15.183:6817/1023131725","cluster_addr":"172.21.15.183:6819/1023131725","heartbeat_back_addr":"172.21.15.183:6823/1023131725","heartbeat_front_addr":"172.21.15.183:6821/1023131725","state":["exists","up"]},{"osd":9,"uuid":"ac16bb0e-6232-45c8-b16a-4d878874c588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6827","nonce":3935195657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6829","nonce":3935195657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6833","nonce":3935195657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6831","nonce":3935195657}]},"public_addr":"172.21.15.43:6827/3935195657","cluster_addr":"172.21.15.43:6829/3935195657","heartbeat_back_addr":"172.21.15.43:6833/3935195657","heartbeat_front_addr":"172.21.15.43:6831/3935195657","state":["exists","up"]},{"osd":10,"uuid":"fd839188-f398-4cff-aa7b-36522ff1a8a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6825","nonce":377628320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6827","nonce":377628320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6831","nonce":377628320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6829","nonce":377628320}]},"public_addr":"172.21.15.52:6825/377628320","cluster_addr":"172.21.15.52:6827/377628320","heartbeat_back_addr":"172.21.15.52:6831/377628320","heartbeat_front_addr":"172.21.15.52:6829/377628320","state":["exists","up"]},{"osd":11,"uuid":"a40282a0-0c64-49ee-b64e-6f3f8119e827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6825","nonce":1373086433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6827","nonce":1373086433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6831","nonce":1373086433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6829","nonce":1373086433}]},"public_addr":"172.21.15.183:6825/1373086433","cluster_addr":"172.21.15.183:6827/1373086433","heartbeat_back_addr":"172.21.15.183:6831/1373086433","heartbeat_front_addr":"172.21.15.183:6829/1373086433","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:23:43.478905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:17.573532+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:50.286610+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:25:27.407663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:03.081294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:36.057416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:12.062013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:47.032271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:21.023201+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:57.364739+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:29:34.039995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:30:08.827577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6801/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/4240775295":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/510203140":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/4251957687":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:6801/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1885244665":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/298320867":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1301092625":"2024-03-27T08:21:36.149092+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:32:09.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:09.197+0000 7f40db400700 1 -- 172.21.15.43:0/1292427289 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f40bc0a2920 msgr2=0x7f40bc0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:09.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:09.197+0000 7f40db400700 1 --2- 172.21.15.43:0/1292427289 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f40bc0a2920 0x7f40bc0a4de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f40d4109ae0 tx=0x7f40c0009260 comp rx=0 tx=0).stop 2024-03-26T08:32:09.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:09.197+0000 7f40db400700 1 -- 172.21.15.43:0/1292427289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40d41043e0 msgr2=0x7f40d41a07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:09.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:09.197+0000 7f40db400700 1 --2- 172.21.15.43:0/1292427289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40d41043e0 0x7f40d41a07b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f40c800e9c0 tx=0x7f40c800ed80 comp rx=0 tx=0).stop 2024-03-26T08:32:09.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:09.197+0000 7f40db400700 1 -- 172.21.15.43:0/1292427289 shutdown_connections 2024-03-26T08:32:09.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:09.197+0000 7f40db400700 1 --2- 172.21.15.43:0/1292427289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40d410a470 0x7f40d419a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:09.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:09.197+0000 7f40db400700 1 --2- 172.21.15.43:0/1292427289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40d41043e0 0x7f40d41a07b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:09.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:09.197+0000 7f40db400700 1 --2- 172.21.15.43:0/1292427289 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f40bc0a2920 0x7f40bc0a4de0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:09.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:09.197+0000 7f40db400700 1 --2- 172.21.15.43:0/1292427289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40d4104da0 0x7f40d41a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:09.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:09.197+0000 7f40db400700 1 -- 172.21.15.43:0/1292427289 >> 172.21.15.43:0/1292427289 conn(0x7f40d40fff30 msgr2=0x7f40d41005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:09.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:09.198+0000 7f40db400700 1 -- 172.21.15.43:0/1292427289 shutdown_connections 2024-03-26T08:32:09.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:09.198+0000 7f40db400700 1 -- 172.21.15.43:0/1292427289 wait complete. 2024-03-26T08:32:09.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:09 smithi043 ceph-mon[31929]: pgmap v398: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-03-26T08:32:09.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:09 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1292427289' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:32:10.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:09 smithi183 ceph-mon[35818]: pgmap v398: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-03-26T08:32:10.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:09 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1292427289' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:32:10.060 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-26T08:32:10.060 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-26T08:32:10.060 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-26T08:32:10.061 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-26T08:32:10.061 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-26T08:32:10.062 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-26T08:32:10.062 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-26T08:32:10.062 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-26T08:32:10.063 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-26T08:32:10.063 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-26T08:32:10.064 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-26T08:32:10.064 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-26T08:32:10.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:09 smithi052 ceph-mon[35071]: pgmap v398: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-03-26T08:32:10.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:09 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1292427289' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:32:11.138 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:32:11.138 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:32:11.139 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:32:11.141 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:32:11.618 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:32:11.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:11 smithi043 ceph-mon[31929]: pgmap v399: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-03-26T08:32:12.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:11 smithi183 ceph-mon[35818]: pgmap v399: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-03-26T08:32:12.075 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:11 smithi052 ceph-mon[35071]: pgmap v399: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-03-26T08:32:12.651 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:32:13.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:13 smithi043 ceph-mon[31929]: pgmap v400: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:14.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:13 smithi183 ceph-mon[35818]: pgmap v400: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:14.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:13 smithi052 ceph-mon[35071]: pgmap v400: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:14.094 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:32:14.895 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:32:14.902 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:32:14.903 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:32:14.905 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:32:15.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:15 smithi043 ceph-mon[31929]: pgmap v401: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:16.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:15 smithi183 ceph-mon[35818]: pgmap v401: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:16.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:15 smithi052 ceph-mon[35071]: pgmap v401: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:17.887 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:17 smithi043 ceph-mon[31929]: pgmap v402: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:32:18.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:17 smithi183 ceph-mon[35818]: pgmap v402: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:32:18.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:17 smithi052 ceph-mon[35071]: pgmap v402: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:32:18.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.124+0000 7f29ca000700 1 -- 172.21.15.43:0/192921742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29c41081e0 msgr2=0x7f29c41085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:18.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.124+0000 7f29ca000700 1 --2- 172.21.15.43:0/192921742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29c41081e0 0x7f29c41085c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f29b400b0d0 tx=0x7f29b400b3e0 comp rx=0 tx=0).stop 2024-03-26T08:32:18.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.125+0000 7f29ca000700 1 -- 172.21.15.43:0/192921742 shutdown_connections 2024-03-26T08:32:18.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.125+0000 7f29ca000700 1 --2- 172.21.15.43:0/192921742 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29c406a7e0 0x7f29c410f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:18.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.125+0000 7f29ca000700 1 --2- 172.21.15.43:0/192921742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29c41081e0 0x7f29c41085c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:18.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.125+0000 7f29ca000700 1 --2- 172.21.15.43:0/192921742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29c4069e40 0x7f29c406a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:18.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.125+0000 7f29ca000700 1 -- 172.21.15.43:0/192921742 >> 172.21.15.43:0/192921742 conn(0x7f29c4077fc0 msgr2=0x7f29c40783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:18.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.125+0000 7f29ca000700 1 -- 172.21.15.43:0/192921742 shutdown_connections 2024-03-26T08:32:18.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.125+0000 7f29ca000700 1 -- 172.21.15.43:0/192921742 wait complete. 2024-03-26T08:32:18.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.126+0000 7f29ca000700 1 Processor -- start 2024-03-26T08:32:18.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.126+0000 7f29ca000700 1 -- start start 2024-03-26T08:32:18.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.126+0000 7f29ca000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29c4069e40 0x7f29c41a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:18.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.126+0000 7f29ca000700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29c406a7e0 0x7f29c41a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:18.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.126+0000 7f29ca000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29c41081e0 0x7f29c419a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:18.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.126+0000 7f29ca000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c4112340 con 0x7f29c41081e0 2024-03-26T08:32:18.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.126+0000 7f29ca000700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c41121c0 con 0x7f29c406a7e0 2024-03-26T08:32:18.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.126+0000 7f29ca000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c41124c0 con 0x7f29c4069e40 2024-03-26T08:32:18.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.127+0000 7f29c3e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29c41081e0 0x7f29c419a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:18.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.127+0000 7f29c3e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29c41081e0 0x7f29c419a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59556/0 (socket says 172.21.15.43:59556) 2024-03-26T08:32:18.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.127+0000 7f29c3e00700 1 -- 172.21.15.43:0/3127914699 learned_addr learned my addr 172.21.15.43:0/3127914699 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:32:18.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.127+0000 7f29c2a00700 1 --2- 172.21.15.43:0/3127914699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29c406a7e0 0x7f29c41a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:18.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.127+0000 7f29c3400700 1 --2- 172.21.15.43:0/3127914699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29c4069e40 0x7f29c41a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:18.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.127+0000 7f29c3e00700 1 -- 172.21.15.43:0/3127914699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29c4069e40 msgr2=0x7f29c41a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:18.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.127+0000 7f29c3e00700 1 --2- 172.21.15.43:0/3127914699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29c4069e40 0x7f29c41a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:18.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.127+0000 7f29c3e00700 1 -- 172.21.15.43:0/3127914699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29c406a7e0 msgr2=0x7f29c41a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:18.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.127+0000 7f29c3e00700 1 --2- 172.21.15.43:0/3127914699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29c406a7e0 0x7f29c41a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:18.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.128+0000 7f29c3e00700 1 -- 172.21.15.43:0/3127914699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c419b020 con 0x7f29c41081e0 2024-03-26T08:32:18.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.128+0000 7f29c3e00700 1 --2- 172.21.15.43:0/3127914699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29c41081e0 0x7f29c419a8c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00d980 tx=0x7f29bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:18.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.128+0000 7f29bbe00700 1 -- 172.21.15.43:0/3127914699 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc017070 con 0x7f29c41081e0 2024-03-26T08:32:18.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.128+0000 7f29ca000700 1 -- 172.21.15.43:0/3127914699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c419b310 con 0x7f29c41081e0 2024-03-26T08:32:18.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.128+0000 7f29bbe00700 1 -- 172.21.15.43:0/3127914699 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f29bc005020 con 0x7f29c41081e0 2024-03-26T08:32:18.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.128+0000 7f29bbe00700 1 -- 172.21.15.43:0/3127914699 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc00fa90 con 0x7f29c41081e0 2024-03-26T08:32:18.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.128+0000 7f29ca000700 1 -- 172.21.15.43:0/3127914699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c41a75d0 con 0x7f29c41081e0 2024-03-26T08:32:18.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.129+0000 7f29ca000700 1 -- 172.21.15.43:0/3127914699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f29c410dd90 con 0x7f29c41081e0 2024-03-26T08:32:18.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.132+0000 7f29bbe00700 1 -- 172.21.15.43:0/3127914699 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f29bc0051d0 con 0x7f29c41081e0 2024-03-26T08:32:18.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.133+0000 7f29bbe00700 1 --2- 172.21.15.43:0/3127914699 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f29a80a2920 0x7f29a80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:18.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.133+0000 7f29bbe00700 1 -- 172.21.15.43:0/3127914699 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f29bc0cf250 con 0x7f29c41081e0 2024-03-26T08:32:18.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.133+0000 7f29c3400700 1 --2- 172.21.15.43:0/3127914699 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f29a80a2920 0x7f29a80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:18.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.133+0000 7f29bbe00700 1 --2- 172.21.15.43:0/3127914699 >> [v2:172.21.15.52:6800/3268412328,v1:172.21.15.52:6801/3268412328] conn(0x7f29a80ada00 0x7f29a80afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:18.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.133+0000 7f29bbe00700 1 -- 172.21.15.43:0/3127914699 --> [v2:172.21.15.52:6800/3268412328,v1:172.21.15.52:6801/3268412328] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f29bc015e80 con 0x7f29a80ada00 2024-03-26T08:32:18.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.133+0000 7f29bbe00700 1 -- 172.21.15.43:0/3127914699 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29bc0cf600 con 0x7f29c41081e0 2024-03-26T08:32:18.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.134+0000 7f29c3400700 1 --2- 172.21.15.43:0/3127914699 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f29a80a2920 0x7f29a80a4de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f29c419ba00 tx=0x7f29b400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:18.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.134+0000 7f29c2a00700 1 --2- 172.21.15.43:0/3127914699 >> [v2:172.21.15.52:6800/3268412328,v1:172.21.15.52:6801/3268412328] conn(0x7f29a80ada00 0x7f29a80afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:18.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.135+0000 7f29c2a00700 1 --2- 172.21.15.43:0/3127914699 >> [v2:172.21.15.52:6800/3268412328,v1:172.21.15.52:6801/3268412328] conn(0x7f29a80ada00 0x7f29a80afea0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:18.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.138+0000 7f29bbe00700 1 -- 172.21.15.43:0/3127914699 <== osd.1 v2:172.21.15.52:6800/3268412328 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f29bc015e80 con 0x7f29a80ada00 2024-03-26T08:32:18.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.156+0000 7f29ca000700 1 -- 172.21.15.43:0/3127914699 --> [v2:172.21.15.52:6800/3268412328,v1:172.21.15.52:6801/3268412328] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f29c4103890 con 0x7f29a80ada00 2024-03-26T08:32:18.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.157+0000 7f29bbe00700 1 -- 172.21.15.43:0/3127914699 <== osd.1 v2:172.21.15.52:6800/3268412328 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f29c4103890 con 0x7f29a80ada00 2024-03-26T08:32:18.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.158+0000 7f29ca000700 1 -- 172.21.15.43:0/3127914699 >> [v2:172.21.15.52:6800/3268412328,v1:172.21.15.52:6801/3268412328] conn(0x7f29a80ada00 msgr2=0x7f29a80afea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:18.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.158+0000 7f29ca000700 1 --2- 172.21.15.43:0/3127914699 >> [v2:172.21.15.52:6800/3268412328,v1:172.21.15.52:6801/3268412328] conn(0x7f29a80ada00 0x7f29a80afea0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:18.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.158+0000 7f29ca000700 1 -- 172.21.15.43:0/3127914699 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f29a80a2920 msgr2=0x7f29a80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:18.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.158+0000 7f29ca000700 1 --2- 172.21.15.43:0/3127914699 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f29a80a2920 0x7f29a80a4de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f29c419ba00 tx=0x7f29b400b040 comp rx=0 tx=0).stop 2024-03-26T08:32:18.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.158+0000 7f29ca000700 1 -- 172.21.15.43:0/3127914699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29c41081e0 msgr2=0x7f29c419a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:18.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.158+0000 7f29ca000700 1 --2- 172.21.15.43:0/3127914699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29c41081e0 0x7f29c419a8c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00d980 tx=0x7f29bc00dc90 comp rx=0 tx=0).stop 2024-03-26T08:32:18.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.158+0000 7f29c3e00700 1 -- 172.21.15.43:0/3127914699 reap_dead start 2024-03-26T08:32:18.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.158+0000 7f29ca000700 1 -- 172.21.15.43:0/3127914699 shutdown_connections 2024-03-26T08:32:18.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.158+0000 7f29ca000700 1 -- 172.21.15.43:0/3127914699 >> 172.21.15.43:0/3127914699 conn(0x7f29c4077fc0 msgr2=0x7f29c410d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:18.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.158+0000 7f29ca000700 1 -- 172.21.15.43:0/3127914699 shutdown_connections 2024-03-26T08:32:18.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:18.158+0000 7f29ca000700 1 -- 172.21.15.43:0/3127914699 wait complete. 2024-03-26T08:32:19.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:19 smithi043 ceph-mon[31929]: pgmap v403: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:20.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:19 smithi183 ceph-mon[35818]: pgmap v403: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:20.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:19 smithi052 ceph-mon[35071]: pgmap v403: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:21.245 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:32:21.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:21 smithi043 ceph-mon[31929]: pgmap v404: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:32:21.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.623+0000 7f2363000700 1 -- 172.21.15.43:0/4167818687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f235c104fb0 msgr2=0x7f235c109000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:21.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.623+0000 7f2363000700 1 --2- 172.21.15.43:0/4167818687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f235c104fb0 0x7f235c109000 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f23480099c0 tx=0x7f2348009cd0 comp rx=0 tx=0).stop 2024-03-26T08:32:21.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.623+0000 7f2363000700 1 -- 172.21.15.43:0/4167818687 shutdown_connections 2024-03-26T08:32:21.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.623+0000 7f2363000700 1 --2- 172.21.15.43:0/4167818687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f235c1096e0 0x7f235c1112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:21.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.623+0000 7f2363000700 1 --2- 172.21.15.43:0/4167818687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f235c104600 0x7f235c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:21.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.623+0000 7f2363000700 1 --2- 172.21.15.43:0/4167818687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f235c104fb0 0x7f235c109000 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:21.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.623+0000 7f2363000700 1 -- 172.21.15.43:0/4167818687 >> 172.21.15.43:0/4167818687 conn(0x7f235c0ffe90 msgr2=0x7f235c1022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:21.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.623+0000 7f2363000700 1 -- 172.21.15.43:0/4167818687 shutdown_connections 2024-03-26T08:32:21.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.623+0000 7f2363000700 1 -- 172.21.15.43:0/4167818687 wait complete. 2024-03-26T08:32:21.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.624+0000 7f2363000700 1 Processor -- start 2024-03-26T08:32:21.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.624+0000 7f2363000700 1 -- start start 2024-03-26T08:32:21.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.624+0000 7f2363000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f235c104600 0x7f235c1a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:21.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.624+0000 7f2363000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f235c104fb0 0x7f235c1a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.624+0000 7f2363000700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f235c1096e0 0x7f235c19a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.624+0000 7f2363000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c1145a0 con 0x7f235c104fb0 2024-03-26T08:32:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.624+0000 7f2363000700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c114420 con 0x7f235c1096e0 2024-03-26T08:32:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.624+0000 7f2363000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c114720 con 0x7f235c104600 2024-03-26T08:32:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.625+0000 7f235b400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f235c104fb0 0x7f235c1a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.625+0000 7f235b400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f235c104fb0 0x7f235c1a0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59594/0 (socket says 172.21.15.43:59594) 2024-03-26T08:32:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.625+0000 7f235b400700 1 -- 172.21.15.43:0/3905615755 learned_addr learned my addr 172.21.15.43:0/3905615755 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:32:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.625+0000 7f235be00700 1 --2- 172.21.15.43:0/3905615755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f235c104600 0x7f235c1a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.625+0000 7f2361000700 1 --2- 172.21.15.43:0/3905615755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f235c1096e0 0x7f235c19a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.625+0000 7f235b400700 1 -- 172.21.15.43:0/3905615755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f235c104600 msgr2=0x7f235c1a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.625+0000 7f235b400700 1 --2- 172.21.15.43:0/3905615755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f235c104600 0x7f235c1a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.625+0000 7f235b400700 1 -- 172.21.15.43:0/3905615755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f235c1096e0 msgr2=0x7f235c19a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.625+0000 7f235b400700 1 --2- 172.21.15.43:0/3905615755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f235c1096e0 0x7f235c19a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.625+0000 7f235b400700 1 -- 172.21.15.43:0/3905615755 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c19b060 con 0x7f235c104fb0 2024-03-26T08:32:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.625+0000 7f235b400700 1 --2- 172.21.15.43:0/3905615755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f235c104fb0 0x7f235c1a0c30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f234800c010 tx=0x7f2348009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.626+0000 7f2358c00700 1 -- 172.21.15.43:0/3905615755 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2348027070 con 0x7f235c104fb0 2024-03-26T08:32:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.626+0000 7f2358c00700 1 -- 172.21.15.43:0/3905615755 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2348003f50 con 0x7f235c104fb0 2024-03-26T08:32:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.626+0000 7f2363000700 1 -- 172.21.15.43:0/3905615755 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f235c19b2f0 con 0x7f235c104fb0 2024-03-26T08:32:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.626+0000 7f2358c00700 1 -- 172.21.15.43:0/3905615755 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2348022ce0 con 0x7f235c104fb0 2024-03-26T08:32:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.626+0000 7f2363000700 1 -- 172.21.15.43:0/3905615755 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235c1a75c0 con 0x7f235c104fb0 2024-03-26T08:32:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.627+0000 7f2363000700 1 -- 172.21.15.43:0/3905615755 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f235c062920 con 0x7f235c104fb0 2024-03-26T08:32:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.629+0000 7f2358c00700 1 -- 172.21.15.43:0/3905615755 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f234802a070 con 0x7f235c104fb0 2024-03-26T08:32:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.630+0000 7f2358c00700 1 --2- 172.21.15.43:0/3905615755 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f23400a29a0 0x7f23400a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.630+0000 7f2358c00700 1 -- 172.21.15.43:0/3905615755 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f23480d0ae0 con 0x7f235c104fb0 2024-03-26T08:32:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.631+0000 7f235be00700 1 --2- 172.21.15.43:0/3905615755 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f23400a29a0 0x7f23400a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.631+0000 7f2358c00700 1 --2- 172.21.15.43:0/3905615755 >> [v2:172.21.15.43:6810/3722846033,v1:172.21.15.43:6811/3722846033] conn(0x7f23400ada80 0x7f23400aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.631+0000 7f2358c00700 1 -- 172.21.15.43:0/3905615755 --> [v2:172.21.15.43:6810/3722846033,v1:172.21.15.43:6811/3722846033] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2348009030 con 0x7f23400ada80 2024-03-26T08:32:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.631+0000 7f2358c00700 1 -- 172.21.15.43:0/3905615755 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23480d0e90 con 0x7f235c104fb0 2024-03-26T08:32:21.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.631+0000 7f2361000700 1 --2- 172.21.15.43:0/3905615755 >> [v2:172.21.15.43:6810/3722846033,v1:172.21.15.43:6811/3722846033] conn(0x7f23400ada80 0x7f23400aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:21.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.631+0000 7f235be00700 1 --2- 172.21.15.43:0/3905615755 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f23400a29a0 0x7f23400a4e60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f234c005950 tx=0x7f234c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:21.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.632+0000 7f2361000700 1 --2- 172.21.15.43:0/3905615755 >> [v2:172.21.15.43:6810/3722846033,v1:172.21.15.43:6811/3722846033] conn(0x7f23400ada80 0x7f23400aff20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:21.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.633+0000 7f2358c00700 1 -- 172.21.15.43:0/3905615755 <== osd.3 v2:172.21.15.43:6810/3722846033 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2348009030 con 0x7f23400ada80 2024-03-26T08:32:21.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.651+0000 7f2363000700 1 -- 172.21.15.43:0/3905615755 --> [v2:172.21.15.43:6810/3722846033,v1:172.21.15.43:6811/3722846033] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f235c062d20 con 0x7f23400ada80 2024-03-26T08:32:21.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.652+0000 7f2358c00700 1 -- 172.21.15.43:0/3905615755 <== osd.3 v2:172.21.15.43:6810/3722846033 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f235c062d20 con 0x7f23400ada80 2024-03-26T08:32:21.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.652+0000 7f2363000700 1 -- 172.21.15.43:0/3905615755 >> [v2:172.21.15.43:6810/3722846033,v1:172.21.15.43:6811/3722846033] conn(0x7f23400ada80 msgr2=0x7f23400aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:21.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.652+0000 7f2363000700 1 --2- 172.21.15.43:0/3905615755 >> [v2:172.21.15.43:6810/3722846033,v1:172.21.15.43:6811/3722846033] conn(0x7f23400ada80 0x7f23400aff20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:21.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.652+0000 7f2363000700 1 -- 172.21.15.43:0/3905615755 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f23400a29a0 msgr2=0x7f23400a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:21.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.652+0000 7f2363000700 1 --2- 172.21.15.43:0/3905615755 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f23400a29a0 0x7f23400a4e60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f234c005950 tx=0x7f234c0058e0 comp rx=0 tx=0).stop 2024-03-26T08:32:21.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.652+0000 7f2363000700 1 -- 172.21.15.43:0/3905615755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f235c104fb0 msgr2=0x7f235c1a0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:21.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.652+0000 7f2363000700 1 --2- 172.21.15.43:0/3905615755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f235c104fb0 0x7f235c1a0c30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f234800c010 tx=0x7f2348009f10 comp rx=0 tx=0).stop 2024-03-26T08:32:21.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.652+0000 7f2361000700 1 -- 172.21.15.43:0/3905615755 reap_dead start 2024-03-26T08:32:21.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.652+0000 7f2363000700 1 -- 172.21.15.43:0/3905615755 shutdown_connections 2024-03-26T08:32:21.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.652+0000 7f2363000700 1 -- 172.21.15.43:0/3905615755 >> 172.21.15.43:0/3905615755 conn(0x7f235c0ffe90 msgr2=0x7f235c1021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:21.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.653+0000 7f2363000700 1 -- 172.21.15.43:0/3905615755 shutdown_connections 2024-03-26T08:32:21.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.653+0000 7f2363000700 1 -- 172.21.15.43:0/3905615755 wait complete. 2024-03-26T08:32:22.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:21 smithi183 ceph-mon[35818]: pgmap v404: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:32:22.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:21 smithi052 ceph-mon[35071]: pgmap v404: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:32:24.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:23 smithi183 ceph-mon[35818]: pgmap v405: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:24.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:23 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:32:24.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:23 smithi052 ceph-mon[35071]: pgmap v405: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:24.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:23 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:32:24.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:23 smithi043 ceph-mon[31929]: pgmap v405: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:24.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:23 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:32:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.532+0000 7f2595800700 1 -- 172.21.15.43:0/2537196275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2590102440 msgr2=0x7f2590102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.532+0000 7f2595800700 1 --2- 172.21.15.43:0/2537196275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2590102440 0x7f2590102820 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f25800099c0 tx=0x7f2580009cd0 comp rx=0 tx=0).stop 2024-03-26T08:32:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.532+0000 7f2595800700 1 -- 172.21.15.43:0/2537196275 shutdown_connections 2024-03-26T08:32:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.532+0000 7f2595800700 1 --2- 172.21.15.43:0/2537196275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2590107510 0x7f259010f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.532+0000 7f2595800700 1 --2- 172.21.15.43:0/2537196275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2590102440 0x7f2590102820 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.532+0000 7f2595800700 1 --2- 172.21.15.43:0/2537196275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2590102df0 0x7f2590106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.532+0000 7f2595800700 1 -- 172.21.15.43:0/2537196275 >> 172.21.15.43:0/2537196275 conn(0x7f25900792f0 msgr2=0x7f2590079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.532+0000 7f2595800700 1 -- 172.21.15.43:0/2537196275 shutdown_connections 2024-03-26T08:32:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.532+0000 7f2595800700 1 -- 172.21.15.43:0/2537196275 wait complete. 2024-03-26T08:32:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.533+0000 7f2595800700 1 Processor -- start 2024-03-26T08:32:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.534+0000 7f2595800700 1 -- start start 2024-03-26T08:32:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.534+0000 7f2595800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2590102440 0x7f259019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.534+0000 7f2595800700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2590102df0 0x7f259019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.534+0000 7f2595800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2590107510 0x7f2590198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.534+0000 7f2595800700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25901123d0 con 0x7f2590107510 2024-03-26T08:32:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.534+0000 7f2595800700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2590112250 con 0x7f2590102df0 2024-03-26T08:32:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.534+0000 7f2595800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2590112550 con 0x7f2590102440 2024-03-26T08:32:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.534+0000 7f258f400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2590107510 0x7f2590198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.535+0000 7f258f400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2590107510 0x7f2590198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59576/0 (socket says 172.21.15.43:59576) 2024-03-26T08:32:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.535+0000 7f258f400700 1 -- 172.21.15.43:0/568951361 learned_addr learned my addr 172.21.15.43:0/568951361 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:32:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.535+0000 7f258ea00700 1 --2- 172.21.15.43:0/568951361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2590102440 0x7f259019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.535+0000 7f258e000700 1 --2- 172.21.15.43:0/568951361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2590102df0 0x7f259019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.535+0000 7f258f400700 1 -- 172.21.15.43:0/568951361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2590102440 msgr2=0x7f259019e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.535+0000 7f258f400700 1 --2- 172.21.15.43:0/568951361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2590102440 0x7f259019e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.535+0000 7f258f400700 1 -- 172.21.15.43:0/568951361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2590102df0 msgr2=0x7f259019eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.535+0000 7f258f400700 1 --2- 172.21.15.43:0/568951361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2590102df0 0x7f259019eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.535+0000 7f258f400700 1 -- 172.21.15.43:0/568951361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2590198ff0 con 0x7f2590107510 2024-03-26T08:32:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.535+0000 7f258e000700 1 --2- 172.21.15.43:0/568951361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2590102df0 0x7f259019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:32:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.535+0000 7f258f400700 1 --2- 172.21.15.43:0/568951361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2590107510 0x7f2590198710 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2588007f10 tx=0x7f258800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.536+0000 7f2587400700 1 -- 172.21.15.43:0/568951361 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2588009e20 con 0x7f2590107510 2024-03-26T08:32:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.536+0000 7f2587400700 1 -- 172.21.15.43:0/568951361 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f258800dbd0 con 0x7f2590107510 2024-03-26T08:32:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.536+0000 7f2595800700 1 -- 172.21.15.43:0/568951361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25901992e0 con 0x7f2590107510 2024-03-26T08:32:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.536+0000 7f2587400700 1 -- 172.21.15.43:0/568951361 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2588005990 con 0x7f2590107510 2024-03-26T08:32:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.536+0000 7f2595800700 1 -- 172.21.15.43:0/568951361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25901a55c0 con 0x7f2590107510 2024-03-26T08:32:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.537+0000 7f2595800700 1 -- 172.21.15.43:0/568951361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2590062820 con 0x7f2590107510 2024-03-26T08:32:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.539+0000 7f2587400700 1 -- 172.21.15.43:0/568951361 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f258801a020 con 0x7f2590107510 2024-03-26T08:32:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.541+0000 7f2587400700 1 --2- 172.21.15.43:0/568951361 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f25740a2920 0x7f25740a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.541+0000 7f2587400700 1 -- 172.21.15.43:0/568951361 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f25880ced50 con 0x7f2590107510 2024-03-26T08:32:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.541+0000 7f258ea00700 1 --2- 172.21.15.43:0/568951361 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f25740a2920 0x7f25740a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.541+0000 7f2587400700 1 --2- 172.21.15.43:0/568951361 >> [v2:172.21.15.43:6802/936601146,v1:172.21.15.43:6803/936601146] conn(0x7f25740ada00 0x7f25740afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.541+0000 7f2587400700 1 -- 172.21.15.43:0/568951361 --> [v2:172.21.15.43:6802/936601146,v1:172.21.15.43:6803/936601146] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f258801ee70 con 0x7f25740ada00 2024-03-26T08:32:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.541+0000 7f2587400700 1 -- 172.21.15.43:0/568951361 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25880cf100 con 0x7f2590107510 2024-03-26T08:32:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.541+0000 7f258e000700 1 --2- 172.21.15.43:0/568951361 >> [v2:172.21.15.43:6802/936601146,v1:172.21.15.43:6803/936601146] conn(0x7f25740ada00 0x7f25740afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.542+0000 7f258ea00700 1 --2- 172.21.15.43:0/568951361 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f25740a2920 0x7f25740a4de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2580000c00 tx=0x7f2580005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.542+0000 7f258e000700 1 --2- 172.21.15.43:0/568951361 >> [v2:172.21.15.43:6802/936601146,v1:172.21.15.43:6803/936601146] conn(0x7f25740ada00 0x7f25740afea0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.543+0000 7f2587400700 1 -- 172.21.15.43:0/568951361 <== osd.0 v2:172.21.15.43:6802/936601146 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f258801ee70 con 0x7f25740ada00 2024-03-26T08:32:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.562+0000 7f2595800700 1 -- 172.21.15.43:0/568951361 --> [v2:172.21.15.43:6802/936601146,v1:172.21.15.43:6803/936601146] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2590062ca0 con 0x7f25740ada00 2024-03-26T08:32:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.563+0000 7f2587400700 1 -- 172.21.15.43:0/568951361 <== osd.0 v2:172.21.15.43:6802/936601146 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2590062ca0 con 0x7f25740ada00 2024-03-26T08:32:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.563+0000 7f2595800700 1 -- 172.21.15.43:0/568951361 >> [v2:172.21.15.43:6802/936601146,v1:172.21.15.43:6803/936601146] conn(0x7f25740ada00 msgr2=0x7f25740afea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.563+0000 7f2595800700 1 --2- 172.21.15.43:0/568951361 >> [v2:172.21.15.43:6802/936601146,v1:172.21.15.43:6803/936601146] conn(0x7f25740ada00 0x7f25740afea0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.563+0000 7f2595800700 1 -- 172.21.15.43:0/568951361 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f25740a2920 msgr2=0x7f25740a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.563+0000 7f2595800700 1 --2- 172.21.15.43:0/568951361 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f25740a2920 0x7f25740a4de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2580000c00 tx=0x7f2580005fb0 comp rx=0 tx=0).stop 2024-03-26T08:32:25.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.563+0000 7f2595800700 1 -- 172.21.15.43:0/568951361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2590107510 msgr2=0x7f2590198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:25.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.563+0000 7f2595800700 1 --2- 172.21.15.43:0/568951361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2590107510 0x7f2590198710 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2588007f10 tx=0x7f258800d3b0 comp rx=0 tx=0).stop 2024-03-26T08:32:25.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.563+0000 7f258f400700 1 -- 172.21.15.43:0/568951361 reap_dead start 2024-03-26T08:32:25.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.564+0000 7f2595800700 1 -- 172.21.15.43:0/568951361 shutdown_connections 2024-03-26T08:32:25.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.564+0000 7f2595800700 1 -- 172.21.15.43:0/568951361 >> 172.21.15.43:0/568951361 conn(0x7f25900792f0 msgr2=0x7f2590100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:25.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.564+0000 7f2595800700 1 -- 172.21.15.43:0/568951361 shutdown_connections 2024-03-26T08:32:25.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:21.564+0000 7f2595800700 1 -- 172.21.15.43:0/568951361 wait complete. 2024-03-26T08:32:25.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:25 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:32:25.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:25 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:32:25.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:25 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:32:25.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:25 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:32:25.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:25 smithi183 ceph-mon[35818]: pgmap v406: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:25.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:25 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:32:25.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:25 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:32:25.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:25 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:32:25.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:25 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:32:25.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:25 smithi052 ceph-mon[35071]: pgmap v406: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:25.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:25 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:32:25.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:25 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:32:25.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:25 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:32:25.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:25 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:32:25.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:25 smithi043 ceph-mon[31929]: pgmap v406: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:25.772 INFO:teuthology.orchestra.run.smithi043.stdout:55834574946 2024-03-26T08:32:25.773 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-26T08:32:26.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.710+0000 7f2d2cc00700 1 -- 172.21.15.43:0/832605898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d28074ee0 msgr2=0x7f2d2810e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:26.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.710+0000 7f2d2cc00700 1 --2- 172.21.15.43:0/832605898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d28074ee0 0x7f2d2810e370 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009990 tx=0x7f2d14009ca0 comp rx=0 tx=0).stop 2024-03-26T08:32:26.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.710+0000 7f2d2cc00700 1 -- 172.21.15.43:0/832605898 shutdown_connections 2024-03-26T08:32:26.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.710+0000 7f2d2cc00700 1 --2- 172.21.15.43:0/832605898 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d2810e8b0 0x7f2d28110d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:26.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.710+0000 7f2d2cc00700 1 --2- 172.21.15.43:0/832605898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d280745c0 0x7f2d280749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:26.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.710+0000 7f2d2cc00700 1 --2- 172.21.15.43:0/832605898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d28074ee0 0x7f2d2810e370 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:26.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.710+0000 7f2d2cc00700 1 -- 172.21.15.43:0/832605898 >> 172.21.15.43:0/832605898 conn(0x7f2d280fdb60 msgr2=0x7f2d280fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:26.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.710+0000 7f2d2cc00700 1 -- 172.21.15.43:0/832605898 shutdown_connections 2024-03-26T08:32:26.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.710+0000 7f2d2cc00700 1 -- 172.21.15.43:0/832605898 wait complete. 2024-03-26T08:32:26.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.711+0000 7f2d2cc00700 1 Processor -- start 2024-03-26T08:32:26.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.711+0000 7f2d2cc00700 1 -- start start 2024-03-26T08:32:26.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.712+0000 7f2d2cc00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d280745c0 0x7f2d281a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:26.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.712+0000 7f2d2cc00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d28074ee0 0x7f2d281a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:26.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.712+0000 7f2d2cc00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d2810e8b0 0x7f2d2819a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:26.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.712+0000 7f2d2cc00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d281144a0 con 0x7f2d280745c0 2024-03-26T08:32:26.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.712+0000 7f2d2cc00700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d28114320 con 0x7f2d28074ee0 2024-03-26T08:32:26.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.712+0000 7f2d2cc00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d28114620 con 0x7f2d2810e8b0 2024-03-26T08:32:26.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.712+0000 7f2d27400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d280745c0 0x7f2d281a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:26.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.712+0000 7f2d26a00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d28074ee0 0x7f2d281a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:26.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.712+0000 7f2d26a00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d28074ee0 0x7f2d281a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.43:36362/0 (socket says 172.21.15.43:36362) 2024-03-26T08:32:26.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.712+0000 7f2d26a00700 1 -- 172.21.15.43:0/566338155 learned_addr learned my addr 172.21.15.43:0/566338155 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:32:26.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.712+0000 7f2d27e00700 1 --2- 172.21.15.43:0/566338155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d2810e8b0 0x7f2d2819a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:26.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.713+0000 7f2d27400700 1 -- 172.21.15.43:0/566338155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d2810e8b0 msgr2=0x7f2d2819a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:26.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.713+0000 7f2d27400700 1 --2- 172.21.15.43:0/566338155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d2810e8b0 0x7f2d2819a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:26.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.713+0000 7f2d27400700 1 -- 172.21.15.43:0/566338155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d28074ee0 msgr2=0x7f2d281a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:26.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.713+0000 7f2d27400700 1 --2- 172.21.15.43:0/566338155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d28074ee0 0x7f2d281a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:26.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.713+0000 7f2d27400700 1 -- 172.21.15.43:0/566338155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2819b000 con 0x7f2d280745c0 2024-03-26T08:32:26.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.713+0000 7f2d26a00700 1 --2- 172.21.15.43:0/566338155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d28074ee0 0x7f2d281a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:32:26.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.713+0000 7f2d27400700 1 --2- 172.21.15.43:0/566338155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d280745c0 0x7f2d281a0650 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800d750 tx=0x7f2d1800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:26.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.713+0000 7f2d1fe00700 1 -- 172.21.15.43:0/566338155 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18017070 con 0x7f2d280745c0 2024-03-26T08:32:26.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.713+0000 7f2d1fe00700 1 -- 172.21.15.43:0/566338155 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2d18005020 con 0x7f2d280745c0 2024-03-26T08:32:26.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.713+0000 7f2d2cc00700 1 -- 172.21.15.43:0/566338155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d2819b2f0 con 0x7f2d280745c0 2024-03-26T08:32:26.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.713+0000 7f2d1fe00700 1 -- 172.21.15.43:0/566338155 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1800fa30 con 0x7f2d280745c0 2024-03-26T08:32:26.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.714+0000 7f2d2cc00700 1 -- 172.21.15.43:0/566338155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d281a74b0 con 0x7f2d280745c0 2024-03-26T08:32:26.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.715+0000 7f2d2cc00700 1 -- 172.21.15.43:0/566338155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2d28062920 con 0x7f2d280745c0 2024-03-26T08:32:26.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.717+0000 7f2d1fe00700 1 -- 172.21.15.43:0/566338155 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f2d180051d0 con 0x7f2d280745c0 2024-03-26T08:32:26.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.718+0000 7f2d1fe00700 1 --2- 172.21.15.43:0/566338155 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2d0c0a29a0 0x7f2d0c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:26.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.718+0000 7f2d1fe00700 1 -- 172.21.15.43:0/566338155 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f2d180ce710 con 0x7f2d280745c0 2024-03-26T08:32:26.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.718+0000 7f2d26a00700 1 --2- 172.21.15.43:0/566338155 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2d0c0a29a0 0x7f2d0c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:26.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.719+0000 7f2d1fe00700 1 --2- 172.21.15.43:0/566338155 >> [v2:172.21.15.183:6800/2103180908,v1:172.21.15.183:6801/2103180908] conn(0x7f2d0c0ada80 0x7f2d0c0aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:26.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.719+0000 7f2d1fe00700 1 -- 172.21.15.43:0/566338155 --> [v2:172.21.15.183:6800/2103180908,v1:172.21.15.183:6801/2103180908] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2d0c0b05f0 con 0x7f2d0c0ada80 2024-03-26T08:32:26.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.719+0000 7f2d1fe00700 1 -- 172.21.15.43:0/566338155 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d180d5050 con 0x7f2d280745c0 2024-03-26T08:32:26.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.719+0000 7f2d26a00700 1 --2- 172.21.15.43:0/566338155 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2d0c0a29a0 0x7f2d0c0a4e60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2d2810fd90 tx=0x7f2d1400b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:26.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.719+0000 7f2d27e00700 1 --2- 172.21.15.43:0/566338155 >> [v2:172.21.15.183:6800/2103180908,v1:172.21.15.183:6801/2103180908] conn(0x7f2d0c0ada80 0x7f2d0c0aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:26.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.720+0000 7f2d27e00700 1 --2- 172.21.15.43:0/566338155 >> [v2:172.21.15.183:6800/2103180908,v1:172.21.15.183:6801/2103180908] conn(0x7f2d0c0ada80 0x7f2d0c0aff20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:26.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.723+0000 7f2d1fe00700 1 -- 172.21.15.43:0/566338155 <== osd.2 v2:172.21.15.183:6800/2103180908 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2d0c0b05f0 con 0x7f2d0c0ada80 2024-03-26T08:32:26.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.741+0000 7f2d2cc00700 1 -- 172.21.15.43:0/566338155 --> [v2:172.21.15.183:6800/2103180908,v1:172.21.15.183:6801/2103180908] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2d28062da0 con 0x7f2d0c0ada80 2024-03-26T08:32:26.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.742+0000 7f2d1fe00700 1 -- 172.21.15.43:0/566338155 <== osd.2 v2:172.21.15.183:6800/2103180908 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2d28062da0 con 0x7f2d0c0ada80 2024-03-26T08:32:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.743+0000 7f2d2cc00700 1 -- 172.21.15.43:0/566338155 >> [v2:172.21.15.183:6800/2103180908,v1:172.21.15.183:6801/2103180908] conn(0x7f2d0c0ada80 msgr2=0x7f2d0c0aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.743+0000 7f2d2cc00700 1 --2- 172.21.15.43:0/566338155 >> [v2:172.21.15.183:6800/2103180908,v1:172.21.15.183:6801/2103180908] conn(0x7f2d0c0ada80 0x7f2d0c0aff20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.743+0000 7f2d2cc00700 1 -- 172.21.15.43:0/566338155 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2d0c0a29a0 msgr2=0x7f2d0c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.743+0000 7f2d2cc00700 1 --2- 172.21.15.43:0/566338155 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2d0c0a29a0 0x7f2d0c0a4e60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2d2810fd90 tx=0x7f2d1400b280 comp rx=0 tx=0).stop 2024-03-26T08:32:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.743+0000 7f2d2cc00700 1 -- 172.21.15.43:0/566338155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d280745c0 msgr2=0x7f2d281a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.743+0000 7f2d2cc00700 1 --2- 172.21.15.43:0/566338155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d280745c0 0x7f2d281a0650 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800d750 tx=0x7f2d1800da60 comp rx=0 tx=0).stop 2024-03-26T08:32:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.743+0000 7f2d27e00700 1 -- 172.21.15.43:0/566338155 reap_dead start 2024-03-26T08:32:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.743+0000 7f2d2cc00700 1 -- 172.21.15.43:0/566338155 shutdown_connections 2024-03-26T08:32:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.743+0000 7f2d2cc00700 1 -- 172.21.15.43:0/566338155 >> 172.21.15.43:0/566338155 conn(0x7f2d280fdb60 msgr2=0x7f2d280fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.743+0000 7f2d2cc00700 1 -- 172.21.15.43:0/566338155 shutdown_connections 2024-03-26T08:32:26.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.743+0000 7f2d2cc00700 1 -- 172.21.15.43:0/566338155 wait complete. 2024-03-26T08:32:27.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.468+0000 7f8987200700 1 -- 172.21.15.43:0/764648218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f898006a7e0 msgr2=0x7f898010f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:27.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.468+0000 7f8987200700 1 --2- 172.21.15.43:0/764648218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f898006a7e0 0x7f898010f020 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f897800b0d0 tx=0x7f897800b3e0 comp rx=0 tx=0).stop 2024-03-26T08:32:27.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.468+0000 7f8987200700 1 -- 172.21.15.43:0/764648218 shutdown_connections 2024-03-26T08:32:27.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.468+0000 7f8987200700 1 --2- 172.21.15.43:0/764648218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f898006a7e0 0x7f898010f020 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:27.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.468+0000 7f8987200700 1 --2- 172.21.15.43:0/764648218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89801081e0 0x7f89801085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:27.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.468+0000 7f8987200700 1 --2- 172.21.15.43:0/764648218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8980069e40 0x7f898006a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:27.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.468+0000 7f8987200700 1 -- 172.21.15.43:0/764648218 >> 172.21.15.43:0/764648218 conn(0x7f8980077fc0 msgr2=0x7f89800783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:27.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.468+0000 7f8987200700 1 -- 172.21.15.43:0/764648218 shutdown_connections 2024-03-26T08:32:27.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.468+0000 7f8987200700 1 -- 172.21.15.43:0/764648218 wait complete. 2024-03-26T08:32:27.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.469+0000 7f8987200700 1 Processor -- start 2024-03-26T08:32:27.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.469+0000 7f8987200700 1 -- start start 2024-03-26T08:32:27.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.470+0000 7f8987200700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8980069e40 0x7f8980109030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:27.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.470+0000 7f8987200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f898006a7e0 0x7f8980109570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:27.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.470+0000 7f8987200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89801081e0 0x7f8980109ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:27.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.470+0000 7f8987200700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8980112330 con 0x7f898006a7e0 2024-03-26T08:32:27.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.470+0000 7f8987200700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89801121b0 con 0x7f8980069e40 2024-03-26T08:32:27.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.470+0000 7f8987200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89801124b0 con 0x7f89801081e0 2024-03-26T08:32:27.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.470+0000 7f897f400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f898006a7e0 0x7f8980109570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:27.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.470+0000 7f897f400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f898006a7e0 0x7f8980109570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44442/0 (socket says 172.21.15.43:44442) 2024-03-26T08:32:27.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.470+0000 7f897f400700 1 -- 172.21.15.43:0/3219925605 learned_addr learned my addr 172.21.15.43:0/3219925605 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:32:27.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.470+0000 7f897fe00700 1 --2- 172.21.15.43:0/3219925605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8980069e40 0x7f8980109030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:27.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.470+0000 7f8985200700 1 --2- 172.21.15.43:0/3219925605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89801081e0 0x7f8980109ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:27.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.471+0000 7f897f400700 1 -- 172.21.15.43:0/3219925605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89801081e0 msgr2=0x7f8980109ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:27.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.471+0000 7f897f400700 1 --2- 172.21.15.43:0/3219925605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89801081e0 0x7f8980109ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:27.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.471+0000 7f897f400700 1 -- 172.21.15.43:0/3219925605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8980069e40 msgr2=0x7f8980109030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:27.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.471+0000 7f897f400700 1 --2- 172.21.15.43:0/3219925605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8980069e40 0x7f8980109030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:27.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.471+0000 7f897f400700 1 -- 172.21.15.43:0/3219925605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89801ab040 con 0x7f898006a7e0 2024-03-26T08:32:27.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.471+0000 7f897fe00700 1 --2- 172.21.15.43:0/3219925605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8980069e40 0x7f8980109030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:32:27.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.471+0000 7f897f400700 1 --2- 172.21.15.43:0/3219925605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f898006a7e0 0x7f8980109570 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f896c00e9a0 tx=0x7f896c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:27.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.471+0000 7f897cc00700 1 -- 172.21.15.43:0/3219925605 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c00cbd0 con 0x7f898006a7e0 2024-03-26T08:32:27.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.471+0000 7f8987200700 1 -- 172.21.15.43:0/3219925605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89801ab330 con 0x7f898006a7e0 2024-03-26T08:32:27.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.472+0000 7f8987200700 1 -- 172.21.15.43:0/3219925605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89801ab8c0 con 0x7f898006a7e0 2024-03-26T08:32:27.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.472+0000 7f897cc00700 1 -- 172.21.15.43:0/3219925605 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f896c00cd70 con 0x7f898006a7e0 2024-03-26T08:32:27.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.472+0000 7f897cc00700 1 -- 172.21.15.43:0/3219925605 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c018680 con 0x7f898006a7e0 2024-03-26T08:32:27.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.472+0000 7f8987200700 1 -- 172.21.15.43:0/3219925605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8960001030 con 0x7f898006a7e0 2024-03-26T08:32:27.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.475+0000 7f897cc00700 1 -- 172.21.15.43:0/3219925605 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f896c018820 con 0x7f898006a7e0 2024-03-26T08:32:27.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.476+0000 7f897cc00700 1 --2- 172.21.15.43:0/3219925605 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f89640a2920 0x7f89640a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:27.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.476+0000 7f897cc00700 1 -- 172.21.15.43:0/3219925605 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f896c0cf310 con 0x7f898006a7e0 2024-03-26T08:32:27.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.476+0000 7f897fe00700 1 --2- 172.21.15.43:0/3219925605 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f89640a2920 0x7f89640a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:27.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.477+0000 7f897cc00700 1 --2- 172.21.15.43:0/3219925605 >> [v2:172.21.15.43:6818/4206861158,v1:172.21.15.43:6819/4206861158] conn(0x7f89640ada00 0x7f89640afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:27.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.477+0000 7f897cc00700 1 -- 172.21.15.43:0/3219925605 --> [v2:172.21.15.43:6818/4206861158,v1:172.21.15.43:6819/4206861158] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f896c018e50 con 0x7f89640ada00 2024-03-26T08:32:27.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.477+0000 7f897cc00700 1 -- 172.21.15.43:0/3219925605 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f896c0cf6c0 con 0x7f898006a7e0 2024-03-26T08:32:27.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.477+0000 7f8985200700 1 --2- 172.21.15.43:0/3219925605 >> [v2:172.21.15.43:6818/4206861158,v1:172.21.15.43:6819/4206861158] conn(0x7f89640ada00 0x7f89640afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:27.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.477+0000 7f897fe00700 1 --2- 172.21.15.43:0/3219925605 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f89640a2920 0x7f89640a4de0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f897000afa0 tx=0x7f897000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:27.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.478+0000 7f8985200700 1 --2- 172.21.15.43:0/3219925605 >> [v2:172.21.15.43:6818/4206861158,v1:172.21.15.43:6819/4206861158] conn(0x7f89640ada00 0x7f89640afea0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:27.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.479+0000 7f897cc00700 1 -- 172.21.15.43:0/3219925605 <== osd.6 v2:172.21.15.43:6818/4206861158 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f896c018e50 con 0x7f89640ada00 2024-03-26T08:32:27.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.497+0000 7f8987200700 1 -- 172.21.15.43:0/3219925605 --> [v2:172.21.15.43:6818/4206861158,v1:172.21.15.43:6819/4206861158] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8960002e60 con 0x7f89640ada00 2024-03-26T08:32:27.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.498+0000 7f897cc00700 1 -- 172.21.15.43:0/3219925605 <== osd.6 v2:172.21.15.43:6818/4206861158 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8960002e60 con 0x7f89640ada00 2024-03-26T08:32:27.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.499+0000 7f8987200700 1 -- 172.21.15.43:0/3219925605 >> [v2:172.21.15.43:6818/4206861158,v1:172.21.15.43:6819/4206861158] conn(0x7f89640ada00 msgr2=0x7f89640afea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:27.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.499+0000 7f8987200700 1 --2- 172.21.15.43:0/3219925605 >> [v2:172.21.15.43:6818/4206861158,v1:172.21.15.43:6819/4206861158] conn(0x7f89640ada00 0x7f89640afea0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:27.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.499+0000 7f8987200700 1 -- 172.21.15.43:0/3219925605 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f89640a2920 msgr2=0x7f89640a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:27.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.499+0000 7f8987200700 1 --2- 172.21.15.43:0/3219925605 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f89640a2920 0x7f89640a4de0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f897000afa0 tx=0x7f897000a380 comp rx=0 tx=0).stop 2024-03-26T08:32:27.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.499+0000 7f8987200700 1 -- 172.21.15.43:0/3219925605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f898006a7e0 msgr2=0x7f8980109570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:27.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.499+0000 7f8987200700 1 --2- 172.21.15.43:0/3219925605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f898006a7e0 0x7f8980109570 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f896c00e9a0 tx=0x7f896c00ecb0 comp rx=0 tx=0).stop 2024-03-26T08:32:27.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.499+0000 7f8985200700 1 -- 172.21.15.43:0/3219925605 reap_dead start 2024-03-26T08:32:27.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.499+0000 7f8987200700 1 -- 172.21.15.43:0/3219925605 shutdown_connections 2024-03-26T08:32:27.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.499+0000 7f8987200700 1 -- 172.21.15.43:0/3219925605 >> 172.21.15.43:0/3219925605 conn(0x7f8980077fc0 msgr2=0x7f89800ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:27.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.499+0000 7f8987200700 1 -- 172.21.15.43:0/3219925605 shutdown_connections 2024-03-26T08:32:27.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:26.499+0000 7f8987200700 1 -- 172.21.15.43:0/3219925605 wait complete. 2024-03-26T08:32:27.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:27 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:32:27.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:27 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:32:27.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:27 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:32:27.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:27 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:32:27.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:27 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:32:27.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:27 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:32:28.384 INFO:teuthology.orchestra.run.smithi043.stdout:34359738473 2024-03-26T08:32:28.384 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-26T08:32:28.478 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:28 smithi043 ceph-mon[31929]: pgmap v407: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:32:28.478 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:28 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:32:28.478 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:28 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:32:28.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:28 smithi183 ceph-mon[35818]: pgmap v407: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:32:28.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:28 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:32:28.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:28 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:32:28.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:28 smithi052 ceph-mon[35071]: pgmap v407: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:32:28.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:28 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:32:28.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:28 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:32:29.313 INFO:teuthology.orchestra.run.smithi043.stdout:111669149782 2024-03-26T08:32:29.314 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-26T08:32:30.469 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:30 smithi043 ceph-mon[31929]: pgmap v408: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:30.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:30 smithi183 ceph-mon[35818]: pgmap v408: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:30.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:30 smithi052 ceph-mon[35071]: pgmap v408: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:31.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.955+0000 7f7577a00700 1 -- 172.21.15.43:0/2282069980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7570104f90 msgr2=0x7f7570108fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:31.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.955+0000 7f7577a00700 1 --2- 172.21.15.43:0/2282069980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7570104f90 0x7f7570108fe0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f755c0099c0 tx=0x7f755c009cd0 comp rx=0 tx=0).stop 2024-03-26T08:32:31.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.956+0000 7f7577a00700 1 -- 172.21.15.43:0/2282069980 shutdown_connections 2024-03-26T08:32:31.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.956+0000 7f7577a00700 1 --2- 172.21.15.43:0/2282069980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75701096c0 0x7f7570111290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:31.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.956+0000 7f7577a00700 1 --2- 172.21.15.43:0/2282069980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75701045e0 0x7f75701049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:31.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.956+0000 7f7577a00700 1 --2- 172.21.15.43:0/2282069980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7570104f90 0x7f7570108fe0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:31.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.956+0000 7f7577a00700 1 -- 172.21.15.43:0/2282069980 >> 172.21.15.43:0/2282069980 conn(0x7f75700ffe70 msgr2=0x7f7570102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:31.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.956+0000 7f7577a00700 1 -- 172.21.15.43:0/2282069980 shutdown_connections 2024-03-26T08:32:31.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.956+0000 7f7577a00700 1 -- 172.21.15.43:0/2282069980 wait complete. 2024-03-26T08:32:31.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.957+0000 7f7577a00700 1 Processor -- start 2024-03-26T08:32:31.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.957+0000 7f7577a00700 1 -- start start 2024-03-26T08:32:31.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.957+0000 7f7577a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75701045e0 0x7f75701a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:31.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.957+0000 7f7577a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7570104f90 0x7f75701a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:31.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.957+0000 7f7577a00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75701096c0 0x7f757019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:31.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.957+0000 7f7577a00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75701145a0 con 0x7f7570104f90 2024-03-26T08:32:31.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.957+0000 7f7577a00700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7570114420 con 0x7f75701096c0 2024-03-26T08:32:31.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.957+0000 7f7577a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7570114720 con 0x7f75701045e0 2024-03-26T08:32:31.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.958+0000 7f756fe00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7570104f90 0x7f75701a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:31.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.958+0000 7f756fe00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7570104f90 0x7f75701a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44464/0 (socket says 172.21.15.43:44464) 2024-03-26T08:32:31.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.958+0000 7f756fe00700 1 -- 172.21.15.43:0/2143838057 learned_addr learned my addr 172.21.15.43:0/2143838057 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:32:31.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.958+0000 7f7575a00700 1 --2- 172.21.15.43:0/2143838057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75701096c0 0x7f757019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:31.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.958+0000 7f7575000700 1 --2- 172.21.15.43:0/2143838057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75701045e0 0x7f75701a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:31.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.958+0000 7f756fe00700 1 -- 172.21.15.43:0/2143838057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75701045e0 msgr2=0x7f75701a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:31.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.958+0000 7f756fe00700 1 --2- 172.21.15.43:0/2143838057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75701045e0 0x7f75701a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:31.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.958+0000 7f756fe00700 1 -- 172.21.15.43:0/2143838057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75701096c0 msgr2=0x7f757019a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:31.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.958+0000 7f756fe00700 1 --2- 172.21.15.43:0/2143838057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75701096c0 0x7f757019a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:31.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.958+0000 7f756fe00700 1 -- 172.21.15.43:0/2143838057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757019b040 con 0x7f7570104f90 2024-03-26T08:32:31.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.959+0000 7f756fe00700 1 --2- 172.21.15.43:0/2143838057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7570104f90 0x7f75701a0c10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f755c00c010 tx=0x7f755c009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:31.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.959+0000 7f756d600700 1 -- 172.21.15.43:0/2143838057 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c027070 con 0x7f7570104f90 2024-03-26T08:32:31.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.959+0000 7f756d600700 1 -- 172.21.15.43:0/2143838057 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f755c003f50 con 0x7f7570104f90 2024-03-26T08:32:31.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.959+0000 7f7577a00700 1 -- 172.21.15.43:0/2143838057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757019b2d0 con 0x7f7570104f90 2024-03-26T08:32:31.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.959+0000 7f756d600700 1 -- 172.21.15.43:0/2143838057 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c022ce0 con 0x7f7570104f90 2024-03-26T08:32:31.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.959+0000 7f7577a00700 1 -- 172.21.15.43:0/2143838057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75701a75a0 con 0x7f7570104f90 2024-03-26T08:32:31.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.960+0000 7f7577a00700 1 -- 172.21.15.43:0/2143838057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7570062920 con 0x7f7570104f90 2024-03-26T08:32:31.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.962+0000 7f756d600700 1 -- 172.21.15.43:0/2143838057 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f755c02a070 con 0x7f7570104f90 2024-03-26T08:32:31.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.964+0000 7f756d600700 1 --2- 172.21.15.43:0/2143838057 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f75580a2920 0x7f75580a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:31.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.964+0000 7f756d600700 1 -- 172.21.15.43:0/2143838057 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f755c0d0ae0 con 0x7f7570104f90 2024-03-26T08:32:31.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.964+0000 7f7575000700 1 --2- 172.21.15.43:0/2143838057 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f75580a2920 0x7f75580a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:31.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.964+0000 7f756d600700 1 --2- 172.21.15.43:0/2143838057 >> [v2:172.21.15.43:6826/3935195657,v1:172.21.15.43:6827/3935195657] conn(0x7f75580ada00 0x7f75580afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:31.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.964+0000 7f756d600700 1 -- 172.21.15.43:0/2143838057 --> [v2:172.21.15.43:6826/3935195657,v1:172.21.15.43:6827/3935195657] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f755c009030 con 0x7f75580ada00 2024-03-26T08:32:31.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.964+0000 7f756d600700 1 -- 172.21.15.43:0/2143838057 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f755c0d0e90 con 0x7f7570104f90 2024-03-26T08:32:31.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.965+0000 7f7575a00700 1 --2- 172.21.15.43:0/2143838057 >> [v2:172.21.15.43:6826/3935195657,v1:172.21.15.43:6827/3935195657] conn(0x7f75580ada00 0x7f75580afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:31.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.965+0000 7f7575000700 1 --2- 172.21.15.43:0/2143838057 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f75580a2920 0x7f75580a4de0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7564005950 tx=0x7f75640058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:31.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.965+0000 7f7575a00700 1 --2- 172.21.15.43:0/2143838057 >> [v2:172.21.15.43:6826/3935195657,v1:172.21.15.43:6827/3935195657] conn(0x7f75580ada00 0x7f75580afea0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:31.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.966+0000 7f756d600700 1 -- 172.21.15.43:0/2143838057 <== osd.9 v2:172.21.15.43:6826/3935195657 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f755c009030 con 0x7f75580ada00 2024-03-26T08:32:31.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.985+0000 7f7577a00700 1 -- 172.21.15.43:0/2143838057 --> [v2:172.21.15.43:6826/3935195657,v1:172.21.15.43:6827/3935195657] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7570062da0 con 0x7f75580ada00 2024-03-26T08:32:31.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.985+0000 7f756d600700 1 -- 172.21.15.43:0/2143838057 <== osd.9 v2:172.21.15.43:6826/3935195657 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7570062da0 con 0x7f75580ada00 2024-03-26T08:32:31.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.985+0000 7f7577a00700 1 -- 172.21.15.43:0/2143838057 >> [v2:172.21.15.43:6826/3935195657,v1:172.21.15.43:6827/3935195657] conn(0x7f75580ada00 msgr2=0x7f75580afea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:31.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.985+0000 7f7577a00700 1 --2- 172.21.15.43:0/2143838057 >> [v2:172.21.15.43:6826/3935195657,v1:172.21.15.43:6827/3935195657] conn(0x7f75580ada00 0x7f75580afea0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:31.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.986+0000 7f7577a00700 1 -- 172.21.15.43:0/2143838057 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f75580a2920 msgr2=0x7f75580a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:31.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.986+0000 7f7577a00700 1 --2- 172.21.15.43:0/2143838057 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f75580a2920 0x7f75580a4de0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7564005950 tx=0x7f75640058e0 comp rx=0 tx=0).stop 2024-03-26T08:32:31.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.986+0000 7f7577a00700 1 -- 172.21.15.43:0/2143838057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7570104f90 msgr2=0x7f75701a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:31.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.986+0000 7f7577a00700 1 --2- 172.21.15.43:0/2143838057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7570104f90 0x7f75701a0c10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f755c00c010 tx=0x7f755c009f10 comp rx=0 tx=0).stop 2024-03-26T08:32:31.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.986+0000 7f7575a00700 1 -- 172.21.15.43:0/2143838057 reap_dead start 2024-03-26T08:32:31.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.986+0000 7f7577a00700 1 -- 172.21.15.43:0/2143838057 shutdown_connections 2024-03-26T08:32:31.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.986+0000 7f7577a00700 1 -- 172.21.15.43:0/2143838057 >> 172.21.15.43:0/2143838057 conn(0x7f75700ffe70 msgr2=0x7f75701009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:31.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.986+0000 7f7577a00700 1 -- 172.21.15.43:0/2143838057 shutdown_connections 2024-03-26T08:32:31.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:28.986+0000 7f7577a00700 1 -- 172.21.15.43:0/2143838057 wait complete. 2024-03-26T08:32:32.076 INFO:teuthology.orchestra.run.smithi043.stdout:201863462977 2024-03-26T08:32:32.077 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-26T08:32:32.278 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:32 smithi043 ceph-mon[31929]: pgmap v409: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:32:32.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:32 smithi183 ceph-mon[35818]: pgmap v409: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:32:32.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:32 smithi052 ceph-mon[35071]: pgmap v409: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:32:33.103 INFO:teuthology.orchestra.run.smithi043.stdout:77309411422 2024-03-26T08:32:33.103 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-26T08:32:33.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.322+0000 7f73e7200700 1 -- 172.21.15.43:0/466634488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73e006a7e0 msgr2=0x7f73e010f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:33.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.322+0000 7f73e7200700 1 --2- 172.21.15.43:0/466634488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73e006a7e0 0x7f73e010f020 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f73d800b0d0 tx=0x7f73d800b3e0 comp rx=0 tx=0).stop 2024-03-26T08:32:33.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.322+0000 7f73e7200700 1 -- 172.21.15.43:0/466634488 shutdown_connections 2024-03-26T08:32:33.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.322+0000 7f73e7200700 1 --2- 172.21.15.43:0/466634488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73e006a7e0 0x7f73e010f020 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:33.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.322+0000 7f73e7200700 1 --2- 172.21.15.43:0/466634488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73e01081e0 0x7f73e01085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:33.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.322+0000 7f73e7200700 1 --2- 172.21.15.43:0/466634488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73e0069e40 0x7f73e006a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:33.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.322+0000 7f73e7200700 1 -- 172.21.15.43:0/466634488 >> 172.21.15.43:0/466634488 conn(0x7f73e0077fc0 msgr2=0x7f73e00783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:33.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.322+0000 7f73e7200700 1 -- 172.21.15.43:0/466634488 shutdown_connections 2024-03-26T08:32:33.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.322+0000 7f73e7200700 1 -- 172.21.15.43:0/466634488 wait complete. 2024-03-26T08:32:33.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.323+0000 7f73e7200700 1 Processor -- start 2024-03-26T08:32:33.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.323+0000 7f73e7200700 1 -- start start 2024-03-26T08:32:33.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.324+0000 7f73e7200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73e0069e40 0x7f73e01a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:33.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.324+0000 7f73e7200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73e006a7e0 0x7f73e01a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:33.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.324+0000 7f73e7200700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73e01081e0 0x7f73e019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:33.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.324+0000 7f73e7200700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e0112360 con 0x7f73e006a7e0 2024-03-26T08:32:33.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.324+0000 7f73e7200700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e01121e0 con 0x7f73e01081e0 2024-03-26T08:32:33.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.324+0000 7f73e7200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e01124e0 con 0x7f73e0069e40 2024-03-26T08:32:33.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.324+0000 7f73df400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73e006a7e0 0x7f73e01a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:33.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.324+0000 7f73df400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73e006a7e0 0x7f73e01a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44496/0 (socket says 172.21.15.43:44496) 2024-03-26T08:32:33.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.324+0000 7f73df400700 1 -- 172.21.15.43:0/2306568936 learned_addr learned my addr 172.21.15.43:0/2306568936 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:32:33.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.324+0000 7f73dfe00700 1 --2- 172.21.15.43:0/2306568936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73e0069e40 0x7f73e01a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:33.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.324+0000 7f73e5200700 1 --2- 172.21.15.43:0/2306568936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73e01081e0 0x7f73e019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:33.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.325+0000 7f73df400700 1 -- 172.21.15.43:0/2306568936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73e0069e40 msgr2=0x7f73e01a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:33.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.325+0000 7f73df400700 1 --2- 172.21.15.43:0/2306568936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73e0069e40 0x7f73e01a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:33.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.325+0000 7f73df400700 1 -- 172.21.15.43:0/2306568936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73e01081e0 msgr2=0x7f73e019a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:33.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.325+0000 7f73df400700 1 --2- 172.21.15.43:0/2306568936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f73e01081e0 0x7f73e019a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:33.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.325+0000 7f73df400700 1 -- 172.21.15.43:0/2306568936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e019afc0 con 0x7f73e006a7e0 2024-03-26T08:32:33.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.325+0000 7f73df400700 1 --2- 172.21.15.43:0/2306568936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73e006a7e0 0x7f73e01a0be0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00d980 tx=0x7f73cc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:33.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.325+0000 7f73dcc00700 1 -- 172.21.15.43:0/2306568936 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc017070 con 0x7f73e006a7e0 2024-03-26T08:32:33.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.325+0000 7f73dcc00700 1 -- 172.21.15.43:0/2306568936 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f73cc005020 con 0x7f73e006a7e0 2024-03-26T08:32:33.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.326+0000 7f73dcc00700 1 -- 172.21.15.43:0/2306568936 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc00fa90 con 0x7f73e006a7e0 2024-03-26T08:32:33.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.326+0000 7f73e7200700 1 -- 172.21.15.43:0/2306568936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e019b2b0 con 0x7f73e006a7e0 2024-03-26T08:32:33.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.326+0000 7f73e7200700 1 -- 172.21.15.43:0/2306568936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e01a7570 con 0x7f73e006a7e0 2024-03-26T08:32:33.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.327+0000 7f73e7200700 1 -- 172.21.15.43:0/2306568936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f73e010dd90 con 0x7f73e006a7e0 2024-03-26T08:32:33.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.329+0000 7f73dcc00700 1 -- 172.21.15.43:0/2306568936 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f73cc0051d0 con 0x7f73e006a7e0 2024-03-26T08:32:33.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.330+0000 7f73dcc00700 1 --2- 172.21.15.43:0/2306568936 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f73c40a2920 0x7f73c40a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:33.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.330+0000 7f73dcc00700 1 -- 172.21.15.43:0/2306568936 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f73cc0cf320 con 0x7f73e006a7e0 2024-03-26T08:32:33.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.331+0000 7f73dfe00700 1 --2- 172.21.15.43:0/2306568936 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f73c40a2920 0x7f73c40a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:33.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.331+0000 7f73dcc00700 1 --2- 172.21.15.43:0/2306568936 >> [v2:172.21.15.52:6816/3452064146,v1:172.21.15.52:6817/3452064146] conn(0x7f73c40ada00 0x7f73c40afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:33.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.331+0000 7f73dcc00700 1 -- 172.21.15.43:0/2306568936 --> [v2:172.21.15.52:6816/3452064146,v1:172.21.15.52:6817/3452064146] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f73cc0044e0 con 0x7f73c40ada00 2024-03-26T08:32:33.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.331+0000 7f73dcc00700 1 -- 172.21.15.43:0/2306568936 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73cc0cf800 con 0x7f73e006a7e0 2024-03-26T08:32:33.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.331+0000 7f73dfe00700 1 --2- 172.21.15.43:0/2306568936 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f73c40a2920 0x7f73c40a4de0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f73e019b9a0 tx=0x7f73d00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:33.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.332+0000 7f73e5200700 1 --2- 172.21.15.43:0/2306568936 >> [v2:172.21.15.52:6816/3452064146,v1:172.21.15.52:6817/3452064146] conn(0x7f73c40ada00 0x7f73c40afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.333+0000 7f73e5200700 1 --2- 172.21.15.43:0/2306568936 >> [v2:172.21.15.52:6816/3452064146,v1:172.21.15.52:6817/3452064146] conn(0x7f73c40ada00 0x7f73c40afea0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:33.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.337+0000 7f73dcc00700 1 -- 172.21.15.43:0/2306568936 <== osd.7 v2:172.21.15.52:6816/3452064146 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f73cc0044e0 con 0x7f73c40ada00 2024-03-26T08:32:33.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.356+0000 7f73e7200700 1 -- 172.21.15.43:0/2306568936 --> [v2:172.21.15.52:6816/3452064146,v1:172.21.15.52:6817/3452064146] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f73e0103890 con 0x7f73c40ada00 2024-03-26T08:32:33.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.357+0000 7f73dcc00700 1 -- 172.21.15.43:0/2306568936 <== osd.7 v2:172.21.15.52:6816/3452064146 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f73e0103890 con 0x7f73c40ada00 2024-03-26T08:32:33.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.357+0000 7f73e7200700 1 -- 172.21.15.43:0/2306568936 >> [v2:172.21.15.52:6816/3452064146,v1:172.21.15.52:6817/3452064146] conn(0x7f73c40ada00 msgr2=0x7f73c40afea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:33.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.357+0000 7f73e7200700 1 --2- 172.21.15.43:0/2306568936 >> [v2:172.21.15.52:6816/3452064146,v1:172.21.15.52:6817/3452064146] conn(0x7f73c40ada00 0x7f73c40afea0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:33.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.358+0000 7f73e7200700 1 -- 172.21.15.43:0/2306568936 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f73c40a2920 msgr2=0x7f73c40a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:33.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.358+0000 7f73e7200700 1 --2- 172.21.15.43:0/2306568936 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f73c40a2920 0x7f73c40a4de0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f73e019b9a0 tx=0x7f73d00073d0 comp rx=0 tx=0).stop 2024-03-26T08:32:33.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.358+0000 7f73e7200700 1 -- 172.21.15.43:0/2306568936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73e006a7e0 msgr2=0x7f73e01a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:33.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.358+0000 7f73e7200700 1 --2- 172.21.15.43:0/2306568936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73e006a7e0 0x7f73e01a0be0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00d980 tx=0x7f73cc00dc90 comp rx=0 tx=0).stop 2024-03-26T08:32:33.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.358+0000 7f73e5200700 1 -- 172.21.15.43:0/2306568936 reap_dead start 2024-03-26T08:32:33.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.358+0000 7f73e7200700 1 -- 172.21.15.43:0/2306568936 shutdown_connections 2024-03-26T08:32:33.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.358+0000 7f73e7200700 1 -- 172.21.15.43:0/2306568936 >> 172.21.15.43:0/2306568936 conn(0x7f73e0077fc0 msgr2=0x7f73e010d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:33.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.358+0000 7f73e7200700 1 -- 172.21.15.43:0/2306568936 shutdown_connections 2024-03-26T08:32:33.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:33.359+0000 7f73e7200700 1 -- 172.21.15.43:0/2306568936 wait complete. 2024-03-26T08:32:34.490 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:34 smithi043 ceph-mon[31929]: pgmap v410: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:34.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:34 smithi183 ceph-mon[35818]: pgmap v410: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:34.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:34 smithi052 ceph-mon[35071]: pgmap v410: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:34.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.556+0000 7f2255a00700 1 -- 172.21.15.43:0/3864504899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22501070a0 msgr2=0x7f225006a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:34.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.556+0000 7f2255a00700 1 --2- 172.21.15.43:0/3864504899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22501070a0 0x7f225006a830 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2240009990 tx=0x7f2240009ca0 comp rx=0 tx=0).stop 2024-03-26T08:32:34.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.556+0000 7f2255a00700 1 -- 172.21.15.43:0/3864504899 shutdown_connections 2024-03-26T08:32:34.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.556+0000 7f2255a00700 1 --2- 172.21.15.43:0/3864504899 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2250104bc0 0x7f2250105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:34.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.556+0000 7f2255a00700 1 --2- 172.21.15.43:0/3864504899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22501070a0 0x7f225006a830 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:34.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.556+0000 7f2255a00700 1 --2- 172.21.15.43:0/3864504899 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f225006ad70 0x7f225006b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:34.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.556+0000 7f2255a00700 1 -- 172.21.15.43:0/3864504899 >> 172.21.15.43:0/3864504899 conn(0x7f2250078000 msgr2=0x7f2250078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:34.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.556+0000 7f2255a00700 1 -- 172.21.15.43:0/3864504899 shutdown_connections 2024-03-26T08:32:34.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.556+0000 7f2255a00700 1 -- 172.21.15.43:0/3864504899 wait complete. 2024-03-26T08:32:34.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.557+0000 7f2255a00700 1 Processor -- start 2024-03-26T08:32:34.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.558+0000 7f2255a00700 1 -- start start 2024-03-26T08:32:34.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.558+0000 7f2255a00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f225006ad70 0x7f225019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:34.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.558+0000 7f2255a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2250104bc0 0x7f225019f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:34.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.558+0000 7f2255a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22501070a0 0x7f22501a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:34.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.558+0000 7f2255a00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2250118880 con 0x7f2250104bc0 2024-03-26T08:32:34.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.558+0000 7f2255a00700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2250118700 con 0x7f225006ad70 2024-03-26T08:32:34.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.558+0000 7f2255a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2250118a00 con 0x7f22501070a0 2024-03-26T08:32:34.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.559+0000 7f224e000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2250104bc0 0x7f225019f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:34.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.559+0000 7f224e000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2250104bc0 0x7f225019f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44476/0 (socket says 172.21.15.43:44476) 2024-03-26T08:32:34.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.559+0000 7f224e000700 1 -- 172.21.15.43:0/3463498541 learned_addr learned my addr 172.21.15.43:0/3463498541 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:32:34.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.559+0000 7f224ea00700 1 --2- 172.21.15.43:0/3463498541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f225006ad70 0x7f225019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:34.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.559+0000 7f224f400700 1 --2- 172.21.15.43:0/3463498541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22501070a0 0x7f22501a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:34.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.559+0000 7f224e000700 1 -- 172.21.15.43:0/3463498541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22501070a0 msgr2=0x7f22501a3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:34.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.559+0000 7f224e000700 1 --2- 172.21.15.43:0/3463498541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22501070a0 0x7f22501a3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:34.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.559+0000 7f224e000700 1 -- 172.21.15.43:0/3463498541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f225006ad70 msgr2=0x7f225019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:34.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.559+0000 7f224e000700 1 --2- 172.21.15.43:0/3463498541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f225006ad70 0x7f225019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:34.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.559+0000 7f224e000700 1 -- 172.21.15.43:0/3463498541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22501a3ec0 con 0x7f2250104bc0 2024-03-26T08:32:34.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.560+0000 7f224e000700 1 --2- 172.21.15.43:0/3463498541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2250104bc0 0x7f225019f3f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f223c00e9a0 tx=0x7f223c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:34.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.560+0000 7f2247400700 1 -- 172.21.15.43:0/3463498541 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c00cbd0 con 0x7f2250104bc0 2024-03-26T08:32:34.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.560+0000 7f2255a00700 1 -- 172.21.15.43:0/3463498541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22501a41b0 con 0x7f2250104bc0 2024-03-26T08:32:34.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.560+0000 7f2247400700 1 -- 172.21.15.43:0/3463498541 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f223c00cd70 con 0x7f2250104bc0 2024-03-26T08:32:34.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.560+0000 7f2247400700 1 -- 172.21.15.43:0/3463498541 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c018680 con 0x7f2250104bc0 2024-03-26T08:32:34.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.560+0000 7f2255a00700 1 -- 172.21.15.43:0/3463498541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22501aba30 con 0x7f2250104bc0 2024-03-26T08:32:34.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.561+0000 7f2255a00700 1 -- 172.21.15.43:0/3463498541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2250104580 con 0x7f2250104bc0 2024-03-26T08:32:34.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.564+0000 7f2247400700 1 -- 172.21.15.43:0/3463498541 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f223c010430 con 0x7f2250104bc0 2024-03-26T08:32:34.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.565+0000 7f2247400700 1 --2- 172.21.15.43:0/3463498541 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f22340a2920 0x7f22340a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:34.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.565+0000 7f2247400700 1 -- 172.21.15.43:0/3463498541 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f223c0cf9e0 con 0x7f2250104bc0 2024-03-26T08:32:34.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.565+0000 7f224ea00700 1 --2- 172.21.15.43:0/3463498541 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f22340a2920 0x7f22340a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:34.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.565+0000 7f2247400700 1 --2- 172.21.15.43:0/3463498541 >> [v2:172.21.15.52:6808/4028436523,v1:172.21.15.52:6809/4028436523] conn(0x7f22340ada00 0x7f22340afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:34.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.565+0000 7f2247400700 1 -- 172.21.15.43:0/3463498541 --> [v2:172.21.15.52:6808/4028436523,v1:172.21.15.52:6809/4028436523] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f223c023e50 con 0x7f22340ada00 2024-03-26T08:32:34.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.565+0000 7f2247400700 1 -- 172.21.15.43:0/3463498541 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f223c0cfd90 con 0x7f2250104bc0 2024-03-26T08:32:34.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.566+0000 7f224ea00700 1 --2- 172.21.15.43:0/3463498541 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f22340a2920 0x7f22340a4de0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f224000b2e0 tx=0x7f2240005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:34.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.566+0000 7f224f400700 1 --2- 172.21.15.43:0/3463498541 >> [v2:172.21.15.52:6808/4028436523,v1:172.21.15.52:6809/4028436523] conn(0x7f22340ada00 0x7f22340afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:34.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.567+0000 7f224f400700 1 --2- 172.21.15.43:0/3463498541 >> [v2:172.21.15.52:6808/4028436523,v1:172.21.15.52:6809/4028436523] conn(0x7f22340ada00 0x7f22340afea0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:34.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.569+0000 7f2247400700 1 -- 172.21.15.43:0/3463498541 <== osd.4 v2:172.21.15.52:6808/4028436523 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f223c023e50 con 0x7f22340ada00 2024-03-26T08:32:34.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.588+0000 7f2255a00700 1 -- 172.21.15.43:0/3463498541 --> [v2:172.21.15.52:6808/4028436523,v1:172.21.15.52:6809/4028436523] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f225006a0e0 con 0x7f22340ada00 2024-03-26T08:32:34.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.589+0000 7f2247400700 1 -- 172.21.15.43:0/3463498541 <== osd.4 v2:172.21.15.52:6808/4028436523 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f225006a0e0 con 0x7f22340ada00 2024-03-26T08:32:34.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.589+0000 7f2255a00700 1 -- 172.21.15.43:0/3463498541 >> [v2:172.21.15.52:6808/4028436523,v1:172.21.15.52:6809/4028436523] conn(0x7f22340ada00 msgr2=0x7f22340afea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:34.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.589+0000 7f2255a00700 1 --2- 172.21.15.43:0/3463498541 >> [v2:172.21.15.52:6808/4028436523,v1:172.21.15.52:6809/4028436523] conn(0x7f22340ada00 0x7f22340afea0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:34.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.590+0000 7f2255a00700 1 -- 172.21.15.43:0/3463498541 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f22340a2920 msgr2=0x7f22340a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:34.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.590+0000 7f2255a00700 1 --2- 172.21.15.43:0/3463498541 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f22340a2920 0x7f22340a4de0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f224000b2e0 tx=0x7f2240005960 comp rx=0 tx=0).stop 2024-03-26T08:32:34.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.590+0000 7f2255a00700 1 -- 172.21.15.43:0/3463498541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2250104bc0 msgr2=0x7f225019f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:34.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.590+0000 7f2255a00700 1 --2- 172.21.15.43:0/3463498541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2250104bc0 0x7f225019f3f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f223c00e9a0 tx=0x7f223c00ecb0 comp rx=0 tx=0).stop 2024-03-26T08:32:34.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.590+0000 7f224f400700 1 -- 172.21.15.43:0/3463498541 reap_dead start 2024-03-26T08:32:34.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.590+0000 7f2255a00700 1 -- 172.21.15.43:0/3463498541 shutdown_connections 2024-03-26T08:32:34.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.590+0000 7f2255a00700 1 -- 172.21.15.43:0/3463498541 >> 172.21.15.43:0/3463498541 conn(0x7f2250078000 msgr2=0x7f2250103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:34.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.590+0000 7f2255a00700 1 -- 172.21.15.43:0/3463498541 shutdown_connections 2024-03-26T08:32:34.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:32.590+0000 7f2255a00700 1 -- 172.21.15.43:0/3463498541 wait complete. 2024-03-26T08:32:35.267 INFO:teuthology.orchestra.run.smithi043.stdout:223338299451 2024-03-26T08:32:35.267 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-26T08:32:35.691 INFO:teuthology.orchestra.run.smithi043.stdout:292057776173 2024-03-26T08:32:35.691 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-26T08:32:36.020 INFO:teuthology.orchestra.run.smithi043.stdout:146028888144 2024-03-26T08:32:36.020 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-26T08:32:36.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:36 smithi043 ceph-mon[31929]: pgmap v411: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:36.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:36 smithi183 ceph-mon[35818]: pgmap v411: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:36.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:36 smithi052 ceph-mon[35071]: pgmap v411: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:37.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:37 smithi043 ceph-mon[31929]: pgmap v412: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:32:37.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:37 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-03-26T08:32:37.495 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:37 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2024-03-26T08:32:37.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:37 smithi183 ceph-mon[35818]: pgmap v412: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:32:37.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:37 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-03-26T08:32:37.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:37 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2024-03-26T08:32:37.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:37 smithi052 ceph-mon[35071]: pgmap v412: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:32:37.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:37 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-03-26T08:32:37.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:37 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2024-03-26T08:32:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.746+0000 7f09dd600700 1 -- 172.21.15.43:0/2352005838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09d8104f90 msgr2=0x7f09d8108fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.746+0000 7f09dd600700 1 --2- 172.21.15.43:0/2352005838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09d8104f90 0x7f09d8108fe0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f09c40099c0 tx=0x7f09c4009cd0 comp rx=0 tx=0).stop 2024-03-26T08:32:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.747+0000 7f09dd600700 1 -- 172.21.15.43:0/2352005838 shutdown_connections 2024-03-26T08:32:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.747+0000 7f09dd600700 1 --2- 172.21.15.43:0/2352005838 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09d81096c0 0x7f09d8111290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.747+0000 7f09dd600700 1 --2- 172.21.15.43:0/2352005838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09d81045e0 0x7f09d81049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.747+0000 7f09dd600700 1 --2- 172.21.15.43:0/2352005838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09d8104f90 0x7f09d8108fe0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.747+0000 7f09dd600700 1 -- 172.21.15.43:0/2352005838 >> 172.21.15.43:0/2352005838 conn(0x7f09d80ffe90 msgr2=0x7f09d81022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.747+0000 7f09dd600700 1 -- 172.21.15.43:0/2352005838 shutdown_connections 2024-03-26T08:32:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.747+0000 7f09dd600700 1 -- 172.21.15.43:0/2352005838 wait complete. 2024-03-26T08:32:37.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.748+0000 7f09dd600700 1 Processor -- start 2024-03-26T08:32:37.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.748+0000 7f09dd600700 1 -- start start 2024-03-26T08:32:37.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.748+0000 7f09dd600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09d81045e0 0x7f09d81a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:37.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.748+0000 7f09dd600700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09d8104f90 0x7f09d81a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:37.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.748+0000 7f09dd600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09d81096c0 0x7f09d819a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:37.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.748+0000 7f09dd600700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d8114580 con 0x7f09d81045e0 2024-03-26T08:32:37.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.748+0000 7f09dd600700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d8114400 con 0x7f09d8104f90 2024-03-26T08:32:37.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.748+0000 7f09dd600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d8114700 con 0x7f09d81096c0 2024-03-26T08:32:37.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.749+0000 7f09d6a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09d81045e0 0x7f09d81a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:37.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.749+0000 7f09d6a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09d81045e0 0x7f09d81a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42876/0 (socket says 172.21.15.43:42876) 2024-03-26T08:32:37.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.749+0000 7f09d6a00700 1 -- 172.21.15.43:0/1535301745 learned_addr learned my addr 172.21.15.43:0/1535301745 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:32:37.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.749+0000 7f09d6000700 1 --2- 172.21.15.43:0/1535301745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09d8104f90 0x7f09d81a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:37.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.749+0000 7f09d7400700 1 --2- 172.21.15.43:0/1535301745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09d81096c0 0x7f09d819a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:37.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.749+0000 7f09d6a00700 1 -- 172.21.15.43:0/1535301745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09d81096c0 msgr2=0x7f09d819a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:37.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.749+0000 7f09d6a00700 1 --2- 172.21.15.43:0/1535301745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09d81096c0 0x7f09d819a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:37.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.749+0000 7f09d6a00700 1 -- 172.21.15.43:0/1535301745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09d8104f90 msgr2=0x7f09d81a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:37.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.749+0000 7f09d6a00700 1 --2- 172.21.15.43:0/1535301745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09d8104f90 0x7f09d81a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:37.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.749+0000 7f09d6a00700 1 -- 172.21.15.43:0/1535301745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d819b040 con 0x7f09d81045e0 2024-03-26T08:32:37.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.750+0000 7f09d6a00700 1 --2- 172.21.15.43:0/1535301745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09d81045e0 0x7f09d81a0690 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f09c800c9b0 tx=0x7f09c800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:37.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.750+0000 7f09cf400700 1 -- 172.21.15.43:0/1535301745 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c8007b00 con 0x7f09d81045e0 2024-03-26T08:32:37.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.750+0000 7f09cf400700 1 -- 172.21.15.43:0/1535301745 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f09c8007ca0 con 0x7f09d81045e0 2024-03-26T08:32:37.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.750+0000 7f09dd600700 1 -- 172.21.15.43:0/1535301745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d819b330 con 0x7f09d81045e0 2024-03-26T08:32:37.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.750+0000 7f09cf400700 1 -- 172.21.15.43:0/1535301745 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c8018660 con 0x7f09d81045e0 2024-03-26T08:32:37.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.750+0000 7f09dd600700 1 -- 172.21.15.43:0/1535301745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d81a74f0 con 0x7f09d81045e0 2024-03-26T08:32:37.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.751+0000 7f09dd600700 1 -- 172.21.15.43:0/1535301745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f09d8062920 con 0x7f09d81045e0 2024-03-26T08:32:37.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.753+0000 7f09cf400700 1 -- 172.21.15.43:0/1535301745 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f09c800f450 con 0x7f09d81045e0 2024-03-26T08:32:37.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.755+0000 7f09cf400700 1 --2- 172.21.15.43:0/1535301745 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f09bc0a2920 0x7f09bc0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:37.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.755+0000 7f09cf400700 1 -- 172.21.15.43:0/1535301745 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f09c80cf3e0 con 0x7f09d81045e0 2024-03-26T08:32:37.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.756+0000 7f09d6000700 1 --2- 172.21.15.43:0/1535301745 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f09bc0a2920 0x7f09bc0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:37.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.756+0000 7f09cf400700 1 --2- 172.21.15.43:0/1535301745 >> [v2:172.21.15.183:6824/1373086433,v1:172.21.15.183:6825/1373086433] conn(0x7f09bc0ada80 0x7f09bc0aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:37.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.756+0000 7f09cf400700 1 -- 172.21.15.43:0/1535301745 --> [v2:172.21.15.183:6824/1373086433,v1:172.21.15.183:6825/1373086433] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09bc0b05f0 con 0x7f09bc0ada80 2024-03-26T08:32:37.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.756+0000 7f09cf400700 1 -- 172.21.15.43:0/1535301745 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09c80cf790 con 0x7f09d81045e0 2024-03-26T08:32:37.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.757+0000 7f09d7400700 1 --2- 172.21.15.43:0/1535301745 >> [v2:172.21.15.183:6824/1373086433,v1:172.21.15.183:6825/1373086433] conn(0x7f09bc0ada80 0x7f09bc0aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:37.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.757+0000 7f09d6000700 1 --2- 172.21.15.43:0/1535301745 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f09bc0a2920 0x7f09bc0a4de0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f09c4000c00 tx=0x7f09c4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:37.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.758+0000 7f09d7400700 1 --2- 172.21.15.43:0/1535301745 >> [v2:172.21.15.183:6824/1373086433,v1:172.21.15.183:6825/1373086433] conn(0x7f09bc0ada80 0x7f09bc0aff20 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:37.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.760+0000 7f09cf400700 1 -- 172.21.15.43:0/1535301745 <== osd.11 v2:172.21.15.183:6824/1373086433 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f09bc0b05f0 con 0x7f09bc0ada80 2024-03-26T08:32:37.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.779+0000 7f09dd600700 1 -- 172.21.15.43:0/1535301745 --> [v2:172.21.15.183:6824/1373086433,v1:172.21.15.183:6825/1373086433] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f09d8062da0 con 0x7f09bc0ada80 2024-03-26T08:32:37.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.780+0000 7f09cf400700 1 -- 172.21.15.43:0/1535301745 <== osd.11 v2:172.21.15.183:6824/1373086433 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f09d8062da0 con 0x7f09bc0ada80 2024-03-26T08:32:37.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.780+0000 7f09dd600700 1 -- 172.21.15.43:0/1535301745 >> [v2:172.21.15.183:6824/1373086433,v1:172.21.15.183:6825/1373086433] conn(0x7f09bc0ada80 msgr2=0x7f09bc0aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:37.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.780+0000 7f09dd600700 1 --2- 172.21.15.43:0/1535301745 >> [v2:172.21.15.183:6824/1373086433,v1:172.21.15.183:6825/1373086433] conn(0x7f09bc0ada80 0x7f09bc0aff20 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:37.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.780+0000 7f09dd600700 1 -- 172.21.15.43:0/1535301745 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f09bc0a2920 msgr2=0x7f09bc0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:37.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.780+0000 7f09dd600700 1 --2- 172.21.15.43:0/1535301745 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f09bc0a2920 0x7f09bc0a4de0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f09c4000c00 tx=0x7f09c4005fb0 comp rx=0 tx=0).stop 2024-03-26T08:32:37.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.780+0000 7f09dd600700 1 -- 172.21.15.43:0/1535301745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09d81045e0 msgr2=0x7f09d81a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:37.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.780+0000 7f09dd600700 1 --2- 172.21.15.43:0/1535301745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09d81045e0 0x7f09d81a0690 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f09c800c9b0 tx=0x7f09c800ccc0 comp rx=0 tx=0).stop 2024-03-26T08:32:37.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.780+0000 7f09d7400700 1 -- 172.21.15.43:0/1535301745 reap_dead start 2024-03-26T08:32:37.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.781+0000 7f09dd600700 1 -- 172.21.15.43:0/1535301745 shutdown_connections 2024-03-26T08:32:37.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.781+0000 7f09dd600700 1 -- 172.21.15.43:0/1535301745 >> 172.21.15.43:0/1535301745 conn(0x7f09d80ffe90 msgr2=0x7f09d8102170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:37.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.781+0000 7f09dd600700 1 -- 172.21.15.43:0/1535301745 shutdown_connections 2024-03-26T08:32:37.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:37.781+0000 7f09dd600700 1 -- 172.21.15.43:0/1535301745 wait complete. 2024-03-26T08:32:38.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.798+0000 7f8538800700 1 -- 172.21.15.43:0/833157623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85301096e0 msgr2=0x7f85301112b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:38.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.798+0000 7f8538800700 1 --2- 172.21.15.43:0/833157623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85301096e0 0x7f85301112b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f852c00b0d0 tx=0x7f852c00b3e0 comp rx=0 tx=0).stop 2024-03-26T08:32:38.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.798+0000 7f8538800700 1 -- 172.21.15.43:0/833157623 shutdown_connections 2024-03-26T08:32:38.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.798+0000 7f8538800700 1 --2- 172.21.15.43:0/833157623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85301096e0 0x7f85301112b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:38.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.798+0000 7f8538800700 1 --2- 172.21.15.43:0/833157623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8530104600 0x7f85301049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:38.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.798+0000 7f8538800700 1 --2- 172.21.15.43:0/833157623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8530104fb0 0x7f8530109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:38.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.798+0000 7f8538800700 1 -- 172.21.15.43:0/833157623 >> 172.21.15.43:0/833157623 conn(0x7f85300ffe90 msgr2=0x7f85301022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:38.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.798+0000 7f8538800700 1 -- 172.21.15.43:0/833157623 shutdown_connections 2024-03-26T08:32:38.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.798+0000 7f8538800700 1 -- 172.21.15.43:0/833157623 wait complete. 2024-03-26T08:32:38.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.799+0000 7f8538800700 1 Processor -- start 2024-03-26T08:32:38.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.800+0000 7f8538800700 1 -- start start 2024-03-26T08:32:38.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.800+0000 7f8538800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8530104600 0x7f85301a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:38.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.800+0000 7f8538800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8530104fb0 0x7f85301a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:38.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.800+0000 7f8538800700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85301096e0 0x7f853019a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:38.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.800+0000 7f8538800700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85301145d0 con 0x7f8530104fb0 2024-03-26T08:32:38.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.800+0000 7f8538800700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8530114450 con 0x7f85301096e0 2024-03-26T08:32:38.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.800+0000 7f8538800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8530114750 con 0x7f8530104600 2024-03-26T08:32:38.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.800+0000 7f8535400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8530104fb0 0x7f85301a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:38.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.800+0000 7f8535400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8530104fb0 0x7f85301a0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42858/0 (socket says 172.21.15.43:42858) 2024-03-26T08:32:38.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.800+0000 7f8535400700 1 -- 172.21.15.43:0/3553983182 learned_addr learned my addr 172.21.15.43:0/3553983182 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:32:38.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.801+0000 7f8535e00700 1 --2- 172.21.15.43:0/3553983182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8530104600 0x7f85301a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:38.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.801+0000 7f8536800700 1 --2- 172.21.15.43:0/3553983182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85301096e0 0x7f853019a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:38.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.801+0000 7f8535400700 1 -- 172.21.15.43:0/3553983182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8530104600 msgr2=0x7f85301a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:38.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.801+0000 7f8535400700 1 --2- 172.21.15.43:0/3553983182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8530104600 0x7f85301a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:38.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.801+0000 7f8535400700 1 -- 172.21.15.43:0/3553983182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85301096e0 msgr2=0x7f853019a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:38.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.801+0000 7f8535400700 1 --2- 172.21.15.43:0/3553983182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85301096e0 0x7f853019a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:38.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.801+0000 7f8535400700 1 -- 172.21.15.43:0/3553983182 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853019b010 con 0x7f8530104fb0 2024-03-26T08:32:38.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.801+0000 7f8535400700 1 --2- 172.21.15.43:0/3553983182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8530104fb0 0x7f85301a0c30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f851c00d980 tx=0x7f851c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:38.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.802+0000 7f852aa00700 1 -- 172.21.15.43:0/3553983182 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c017070 con 0x7f8530104fb0 2024-03-26T08:32:38.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.802+0000 7f852aa00700 1 -- 172.21.15.43:0/3553983182 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f851c010460 con 0x7f8530104fb0 2024-03-26T08:32:38.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.802+0000 7f8538800700 1 -- 172.21.15.43:0/3553983182 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f853019b300 con 0x7f8530104fb0 2024-03-26T08:32:38.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.802+0000 7f852aa00700 1 -- 172.21.15.43:0/3553983182 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c00de60 con 0x7f8530104fb0 2024-03-26T08:32:38.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.802+0000 7f8538800700 1 -- 172.21.15.43:0/3553983182 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85301a75c0 con 0x7f8530104fb0 2024-03-26T08:32:38.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.803+0000 7f8538800700 1 -- 172.21.15.43:0/3553983182 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8530062920 con 0x7f8530104fb0 2024-03-26T08:32:38.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.805+0000 7f852aa00700 1 -- 172.21.15.43:0/3553983182 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f851c010610 con 0x7f8530104fb0 2024-03-26T08:32:38.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.806+0000 7f852aa00700 1 --2- 172.21.15.43:0/3553983182 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f85180a2920 0x7f85180a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:38.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.807+0000 7f852aa00700 1 -- 172.21.15.43:0/3553983182 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9071+0+0 (secure 0 0 0) 0x7f851c0ceea0 con 0x7f8530104fb0 2024-03-26T08:32:38.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.807+0000 7f8535e00700 1 --2- 172.21.15.43:0/3553983182 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f85180a2920 0x7f85180a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:38.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.807+0000 7f852aa00700 1 --2- 172.21.15.43:0/3553983182 >> [v2:172.21.15.52:6824/377628320,v1:172.21.15.52:6825/377628320] conn(0x7f85180ad970 0x7f85180afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:38.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.807+0000 7f852aa00700 1 -- 172.21.15.43:0/3553983182 --> [v2:172.21.15.52:6824/377628320,v1:172.21.15.52:6825/377628320] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f851c00ee50 con 0x7f85180ad970 2024-03-26T08:32:38.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.807+0000 7f852aa00700 1 -- 172.21.15.43:0/3553983182 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f851c0cf250 con 0x7f8530104fb0 2024-03-26T08:32:38.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.808+0000 7f8535e00700 1 --2- 172.21.15.43:0/3553983182 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f85180a2920 0x7f85180a4de0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8524005950 tx=0x7f85240058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:38.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.808+0000 7f8536800700 1 --2- 172.21.15.43:0/3553983182 >> [v2:172.21.15.52:6824/377628320,v1:172.21.15.52:6825/377628320] conn(0x7f85180ad970 0x7f85180afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:38.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.809+0000 7f8536800700 1 --2- 172.21.15.43:0/3553983182 >> [v2:172.21.15.52:6824/377628320,v1:172.21.15.52:6825/377628320] conn(0x7f85180ad970 0x7f85180afe10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:38.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.811+0000 7f852aa00700 1 -- 172.21.15.43:0/3553983182 <== osd.10 v2:172.21.15.52:6824/377628320 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f851c00ee50 con 0x7f85180ad970 2024-03-26T08:32:38.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.829+0000 7f8538800700 1 -- 172.21.15.43:0/3553983182 --> [v2:172.21.15.52:6824/377628320,v1:172.21.15.52:6825/377628320] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8530062da0 con 0x7f85180ad970 2024-03-26T08:32:38.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.830+0000 7f852aa00700 1 -- 172.21.15.43:0/3553983182 <== osd.10 v2:172.21.15.52:6824/377628320 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8530062da0 con 0x7f85180ad970 2024-03-26T08:32:38.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.831+0000 7f8538800700 1 -- 172.21.15.43:0/3553983182 >> [v2:172.21.15.52:6824/377628320,v1:172.21.15.52:6825/377628320] conn(0x7f85180ad970 msgr2=0x7f85180afe10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:38.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.831+0000 7f8538800700 1 --2- 172.21.15.43:0/3553983182 >> [v2:172.21.15.52:6824/377628320,v1:172.21.15.52:6825/377628320] conn(0x7f85180ad970 0x7f85180afe10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:38.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.831+0000 7f8538800700 1 -- 172.21.15.43:0/3553983182 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f85180a2920 msgr2=0x7f85180a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:38.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.831+0000 7f8538800700 1 --2- 172.21.15.43:0/3553983182 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f85180a2920 0x7f85180a4de0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8524005950 tx=0x7f85240058e0 comp rx=0 tx=0).stop 2024-03-26T08:32:38.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.831+0000 7f8538800700 1 -- 172.21.15.43:0/3553983182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8530104fb0 msgr2=0x7f85301a0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:38.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.831+0000 7f8538800700 1 --2- 172.21.15.43:0/3553983182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8530104fb0 0x7f85301a0c30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f851c00d980 tx=0x7f851c00dd40 comp rx=0 tx=0).stop 2024-03-26T08:32:38.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.831+0000 7f8536800700 1 -- 172.21.15.43:0/3553983182 reap_dead start 2024-03-26T08:32:38.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.832+0000 7f8538800700 1 -- 172.21.15.43:0/3553983182 shutdown_connections 2024-03-26T08:32:38.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.832+0000 7f8538800700 1 -- 172.21.15.43:0/3553983182 >> 172.21.15.43:0/3553983182 conn(0x7f85300ffe90 msgr2=0x7f8530100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:38.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.832+0000 7f8538800700 1 -- 172.21.15.43:0/3553983182 shutdown_connections 2024-03-26T08:32:38.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:34.832+0000 7f8538800700 1 -- 172.21.15.43:0/3553983182 wait complete. 2024-03-26T08:32:38.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:38 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-03-26T08:32:38.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:38 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-03-26T08:32:38.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:38 smithi043 ceph-mon[31929]: osdmap e93: 12 total, 12 up, 12 in 2024-03-26T08:32:38.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:38 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-03-26T08:32:38.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:38 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-03-26T08:32:38.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:38 smithi183 ceph-mon[35818]: osdmap e93: 12 total, 12 up, 12 in 2024-03-26T08:32:38.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:38 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-03-26T08:32:38.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:38 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-03-26T08:32:38.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:38 smithi052 ceph-mon[35071]: osdmap e93: 12 total, 12 up, 12 in 2024-03-26T08:32:38.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.816+0000 7f7655e00700 1 -- 172.21.15.43:0/3681013002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f765006a7e0 msgr2=0x7f765010f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:38.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.816+0000 7f7655e00700 1 --2- 172.21.15.43:0/3681013002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f765006a7e0 0x7f765010f020 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f764800b0d0 tx=0x7f764800b3e0 comp rx=0 tx=0).stop 2024-03-26T08:32:38.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.816+0000 7f7655e00700 1 -- 172.21.15.43:0/3681013002 shutdown_connections 2024-03-26T08:32:38.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.816+0000 7f7655e00700 1 --2- 172.21.15.43:0/3681013002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f765006a7e0 0x7f765010f020 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:38.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.816+0000 7f7655e00700 1 --2- 172.21.15.43:0/3681013002 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76501081e0 0x7f76501085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:38.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.816+0000 7f7655e00700 1 --2- 172.21.15.43:0/3681013002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7650069e40 0x7f765006a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:38.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.816+0000 7f7655e00700 1 -- 172.21.15.43:0/3681013002 >> 172.21.15.43:0/3681013002 conn(0x7f7650077fc0 msgr2=0x7f76500783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:38.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.816+0000 7f7655e00700 1 -- 172.21.15.43:0/3681013002 shutdown_connections 2024-03-26T08:32:38.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.816+0000 7f7655e00700 1 -- 172.21.15.43:0/3681013002 wait complete. 2024-03-26T08:32:38.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.817+0000 7f7655e00700 1 Processor -- start 2024-03-26T08:32:38.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.817+0000 7f7655e00700 1 -- start start 2024-03-26T08:32:38.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.818+0000 7f7655e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7650069e40 0x7f76501a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:38.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.818+0000 7f7655e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f765006a7e0 0x7f76501a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:38.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.818+0000 7f7655e00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76501081e0 0x7f765019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:38.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.818+0000 7f7655e00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7650112360 con 0x7f7650069e40 2024-03-26T08:32:38.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.818+0000 7f7655e00700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76501121e0 con 0x7f76501081e0 2024-03-26T08:32:38.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.818+0000 7f7655e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76501124e0 con 0x7f765006a7e0 2024-03-26T08:32:38.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.818+0000 7f764ea00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7650069e40 0x7f76501a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:38.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.818+0000 7f764ea00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7650069e40 0x7f76501a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42906/0 (socket says 172.21.15.43:42906) 2024-03-26T08:32:38.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.818+0000 7f764ea00700 1 -- 172.21.15.43:0/720424749 learned_addr learned my addr 172.21.15.43:0/720424749 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:32:38.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.818+0000 7f764e000700 1 --2- 172.21.15.43:0/720424749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f765006a7e0 0x7f76501a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:38.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.818+0000 7f764f400700 1 --2- 172.21.15.43:0/720424749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76501081e0 0x7f765019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:38.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.818+0000 7f764ea00700 1 -- 172.21.15.43:0/720424749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f765006a7e0 msgr2=0x7f76501a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:38.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.818+0000 7f764ea00700 1 --2- 172.21.15.43:0/720424749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f765006a7e0 0x7f76501a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:38.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.818+0000 7f764ea00700 1 -- 172.21.15.43:0/720424749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76501081e0 msgr2=0x7f765019a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:38.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.818+0000 7f764ea00700 1 --2- 172.21.15.43:0/720424749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76501081e0 0x7f765019a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:38.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.818+0000 7f764ea00700 1 -- 172.21.15.43:0/720424749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765019afc0 con 0x7f7650069e40 2024-03-26T08:32:38.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.819+0000 7f764f400700 1 --2- 172.21.15.43:0/720424749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76501081e0 0x7f765019a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:32:38.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.819+0000 7f764ea00700 1 --2- 172.21.15.43:0/720424749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7650069e40 0x7f76501a06a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f764000d980 tx=0x7f764000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:38.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.819+0000 7f7654a00700 1 -- 172.21.15.43:0/720424749 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7640016070 con 0x7f7650069e40 2024-03-26T08:32:38.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.819+0000 7f7654a00700 1 -- 172.21.15.43:0/720424749 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f76400044d0 con 0x7f7650069e40 2024-03-26T08:32:38.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.819+0000 7f7655e00700 1 -- 172.21.15.43:0/720424749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f765019b2b0 con 0x7f7650069e40 2024-03-26T08:32:38.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.819+0000 7f7654a00700 1 -- 172.21.15.43:0/720424749 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7640004cf0 con 0x7f7650069e40 2024-03-26T08:32:38.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.819+0000 7f7655e00700 1 -- 172.21.15.43:0/720424749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76501a7570 con 0x7f7650069e40 2024-03-26T08:32:38.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.820+0000 7f7655e00700 1 -- 172.21.15.43:0/720424749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f765010ddd0 con 0x7f7650069e40 2024-03-26T08:32:38.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.822+0000 7f7654a00700 1 -- 172.21.15.43:0/720424749 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f7640004020 con 0x7f7650069e40 2024-03-26T08:32:38.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.824+0000 7f7654a00700 1 --2- 172.21.15.43:0/720424749 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f76340a2920 0x7f76340a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:38.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.824+0000 7f7654a00700 1 -- 172.21.15.43:0/720424749 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f76400cf4a0 con 0x7f7650069e40 2024-03-26T08:32:38.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.824+0000 7f764e000700 1 --2- 172.21.15.43:0/720424749 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f76340a2920 0x7f76340a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:38.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.824+0000 7f7654a00700 1 --2- 172.21.15.43:0/720424749 >> [v2:172.21.15.183:6816/1023131725,v1:172.21.15.183:6817/1023131725] conn(0x7f76340adb00 0x7f76340affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:38.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.824+0000 7f7654a00700 1 -- 172.21.15.43:0/720424749 --> [v2:172.21.15.183:6816/1023131725,v1:172.21.15.183:6817/1023131725] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7640015ec0 con 0x7f76340adb00 2024-03-26T08:32:38.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.824+0000 7f7654a00700 1 -- 172.21.15.43:0/720424749 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76400cf850 con 0x7f7650069e40 2024-03-26T08:32:38.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.825+0000 7f764f400700 1 --2- 172.21.15.43:0/720424749 >> [v2:172.21.15.183:6816/1023131725,v1:172.21.15.183:6817/1023131725] conn(0x7f76340adb00 0x7f76340affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:38.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.825+0000 7f764e000700 1 --2- 172.21.15.43:0/720424749 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f76340a2920 0x7f76340a4de0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f763c005950 tx=0x7f763c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:38.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.826+0000 7f764f400700 1 --2- 172.21.15.43:0/720424749 >> [v2:172.21.15.183:6816/1023131725,v1:172.21.15.183:6817/1023131725] conn(0x7f76340adb00 0x7f76340affa0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:38.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.829+0000 7f7654a00700 1 -- 172.21.15.43:0/720424749 <== osd.8 v2:172.21.15.183:6816/1023131725 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7640015ec0 con 0x7f76340adb00 2024-03-26T08:32:38.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.847+0000 7f7655e00700 1 -- 172.21.15.43:0/720424749 --> [v2:172.21.15.183:6816/1023131725,v1:172.21.15.183:6817/1023131725] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7650103890 con 0x7f76340adb00 2024-03-26T08:32:38.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.848+0000 7f7654a00700 1 -- 172.21.15.43:0/720424749 <== osd.8 v2:172.21.15.183:6816/1023131725 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7650103890 con 0x7f76340adb00 2024-03-26T08:32:38.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.849+0000 7f7655e00700 1 -- 172.21.15.43:0/720424749 >> [v2:172.21.15.183:6816/1023131725,v1:172.21.15.183:6817/1023131725] conn(0x7f76340adb00 msgr2=0x7f76340affa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:38.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.849+0000 7f7655e00700 1 --2- 172.21.15.43:0/720424749 >> [v2:172.21.15.183:6816/1023131725,v1:172.21.15.183:6817/1023131725] conn(0x7f76340adb00 0x7f76340affa0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:38.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.849+0000 7f7655e00700 1 -- 172.21.15.43:0/720424749 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f76340a2920 msgr2=0x7f76340a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:38.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.849+0000 7f7655e00700 1 --2- 172.21.15.43:0/720424749 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f76340a2920 0x7f76340a4de0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f763c005950 tx=0x7f763c0058e0 comp rx=0 tx=0).stop 2024-03-26T08:32:38.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.849+0000 7f7655e00700 1 -- 172.21.15.43:0/720424749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7650069e40 msgr2=0x7f76501a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:38.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.849+0000 7f7655e00700 1 --2- 172.21.15.43:0/720424749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7650069e40 0x7f76501a06a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f764000d980 tx=0x7f764000dd40 comp rx=0 tx=0).stop 2024-03-26T08:32:38.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.849+0000 7f764f400700 1 -- 172.21.15.43:0/720424749 reap_dead start 2024-03-26T08:32:38.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.849+0000 7f7655e00700 1 -- 172.21.15.43:0/720424749 shutdown_connections 2024-03-26T08:32:38.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.849+0000 7f7655e00700 1 -- 172.21.15.43:0/720424749 >> 172.21.15.43:0/720424749 conn(0x7f7650077fc0 msgr2=0x7f765010d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:38.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.849+0000 7f7655e00700 1 -- 172.21.15.43:0/720424749 shutdown_connections 2024-03-26T08:32:38.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:38.849+0000 7f7655e00700 1 -- 172.21.15.43:0/720424749 wait complete. 2024-03-26T08:32:39.372 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:39 smithi043 ceph-mon[31929]: osdmap e94: 12 total, 12 up, 12 in 2024-03-26T08:32:39.372 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:39 smithi043 ceph-mon[31929]: pgmap v415: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-03-26T08:32:39.449 INFO:teuthology.orchestra.run.smithi043.stdout:313532612646 2024-03-26T08:32:39.450 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-26T08:32:39.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:39 smithi183 ceph-mon[35818]: osdmap e94: 12 total, 12 up, 12 in 2024-03-26T08:32:39.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:39 smithi183 ceph-mon[35818]: pgmap v415: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-03-26T08:32:39.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:39 smithi052 ceph-mon[35071]: osdmap e94: 12 total, 12 up, 12 in 2024-03-26T08:32:39.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:39 smithi052 ceph-mon[35071]: pgmap v415: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-03-26T08:32:39.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.692+0000 7fb050400700 1 -- 172.21.15.43:0/1680137603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb048074ce0 msgr2=0x7fb048112e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:39.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.692+0000 7fb050400700 1 --2- 172.21.15.43:0/1680137603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb048074ce0 0x7fb048112e20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb0340099c0 tx=0x7fb034009cd0 comp rx=0 tx=0).stop 2024-03-26T08:32:39.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.693+0000 7fb050400700 1 -- 172.21.15.43:0/1680137603 shutdown_connections 2024-03-26T08:32:39.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.693+0000 7fb050400700 1 --2- 172.21.15.43:0/1680137603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb048113360 0x7fb048115750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:39.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.693+0000 7fb050400700 1 --2- 172.21.15.43:0/1680137603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0480743c0 0x7fb0480747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:39.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.693+0000 7fb050400700 1 --2- 172.21.15.43:0/1680137603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb048074ce0 0x7fb048112e20 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:39.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.693+0000 7fb050400700 1 -- 172.21.15.43:0/1680137603 >> 172.21.15.43:0/1680137603 conn(0x7fb0480792f0 msgr2=0x7fb048079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:39.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.693+0000 7fb050400700 1 -- 172.21.15.43:0/1680137603 shutdown_connections 2024-03-26T08:32:39.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.693+0000 7fb050400700 1 -- 172.21.15.43:0/1680137603 wait complete. 2024-03-26T08:32:39.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.694+0000 7fb050400700 1 Processor -- start 2024-03-26T08:32:39.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.694+0000 7fb050400700 1 -- start start 2024-03-26T08:32:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.694+0000 7fb050400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0480743c0 0x7fb04819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.694+0000 7fb050400700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb048074ce0 0x7fb04819f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.694+0000 7fb050400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb048113360 0x7fb0481a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.694+0000 7fb050400700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb048118a60 con 0x7fb048113360 2024-03-26T08:32:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.694+0000 7fb050400700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0481188e0 con 0x7fb048074ce0 2024-03-26T08:32:39.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.694+0000 7fb050400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb048118be0 con 0x7fb0480743c0 2024-03-26T08:32:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.695+0000 7fb04e400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb048113360 0x7fb0481a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.695+0000 7fb04e400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb048113360 0x7fb0481a3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42936/0 (socket says 172.21.15.43:42936) 2024-03-26T08:32:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.695+0000 7fb04e400700 1 -- 172.21.15.43:0/3248554345 learned_addr learned my addr 172.21.15.43:0/3248554345 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:32:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.695+0000 7fb04d000700 1 --2- 172.21.15.43:0/3248554345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb048074ce0 0x7fb04819f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.695+0000 7fb04da00700 1 --2- 172.21.15.43:0/3248554345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0480743c0 0x7fb04819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.696+0000 7fb04e400700 1 -- 172.21.15.43:0/3248554345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0480743c0 msgr2=0x7fb04819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.696+0000 7fb04e400700 1 --2- 172.21.15.43:0/3248554345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0480743c0 0x7fb04819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.696+0000 7fb04e400700 1 -- 172.21.15.43:0/3248554345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb048074ce0 msgr2=0x7fb04819f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.696+0000 7fb04e400700 1 --2- 172.21.15.43:0/3248554345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb048074ce0 0x7fb04819f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.696+0000 7fb04e400700 1 -- 172.21.15.43:0/3248554345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0481a3fa0 con 0x7fb048113360 2024-03-26T08:32:39.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.696+0000 7fb04e400700 1 --2- 172.21.15.43:0/3248554345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb048113360 0x7fb0481a3860 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb040007ef0 tx=0x7fb04000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.696+0000 7fb046000700 1 -- 172.21.15.43:0/3248554345 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040004bb0 con 0x7fb048113360 2024-03-26T08:32:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.696+0000 7fb046000700 1 -- 172.21.15.43:0/3248554345 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb040004d50 con 0x7fb048113360 2024-03-26T08:32:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.697+0000 7fb046000700 1 -- 172.21.15.43:0/3248554345 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0400058e0 con 0x7fb048113360 2024-03-26T08:32:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.697+0000 7fb050400700 1 -- 172.21.15.43:0/3248554345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0481a4290 con 0x7fb048113360 2024-03-26T08:32:39.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.697+0000 7fb050400700 1 -- 172.21.15.43:0/3248554345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0481abb10 con 0x7fb048113360 2024-03-26T08:32:39.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.698+0000 7fb050400700 1 -- 172.21.15.43:0/3248554345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb04810f7a0 con 0x7fb048113360 2024-03-26T08:32:39.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.701+0000 7fb046000700 1 -- 172.21.15.43:0/3248554345 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fb04001a020 con 0x7fb048113360 2024-03-26T08:32:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.702+0000 7fb046000700 1 --2- 172.21.15.43:0/3248554345 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb0300a2920 0x7fb0300a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.702+0000 7fb046000700 1 -- 172.21.15.43:0/3248554345 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7fb0400ceca0 con 0x7fb048113360 2024-03-26T08:32:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.702+0000 7fb04da00700 1 --2- 172.21.15.43:0/3248554345 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb0300a2920 0x7fb0300a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.703+0000 7fb046000700 1 --2- 172.21.15.43:0/3248554345 >> [v2:172.21.15.183:6808/787132252,v1:172.21.15.183:6809/787132252] conn(0x7fb0300adb00 0x7fb0300affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.703+0000 7fb046000700 1 -- 172.21.15.43:0/3248554345 --> [v2:172.21.15.183:6808/787132252,v1:172.21.15.183:6809/787132252] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb04000be50 con 0x7fb0300adb00 2024-03-26T08:32:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.703+0000 7fb046000700 1 -- 172.21.15.43:0/3248554345 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0400cf0f0 con 0x7fb048113360 2024-03-26T08:32:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.703+0000 7fb04d000700 1 --2- 172.21.15.43:0/3248554345 >> [v2:172.21.15.183:6808/787132252,v1:172.21.15.183:6809/787132252] conn(0x7fb0300adb00 0x7fb0300affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:39.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.703+0000 7fb04da00700 1 --2- 172.21.15.43:0/3248554345 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb0300a2920 0x7fb0300a4de0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb03c005950 tx=0x7fb03c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:39.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.704+0000 7fb04d000700 1 --2- 172.21.15.43:0/3248554345 >> [v2:172.21.15.183:6808/787132252,v1:172.21.15.183:6809/787132252] conn(0x7fb0300adb00 0x7fb0300affa0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:39.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.706+0000 7fb046000700 1 -- 172.21.15.43:0/3248554345 <== osd.5 v2:172.21.15.183:6808/787132252 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb04000be50 con 0x7fb0300adb00 2024-03-26T08:32:39.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.724+0000 7fb050400700 1 -- 172.21.15.43:0/3248554345 --> [v2:172.21.15.183:6808/787132252,v1:172.21.15.183:6809/787132252] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb048075af0 con 0x7fb0300adb00 2024-03-26T08:32:39.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.726+0000 7fb046000700 1 -- 172.21.15.43:0/3248554345 <== osd.5 v2:172.21.15.183:6808/787132252 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb048075af0 con 0x7fb0300adb00 2024-03-26T08:32:39.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.726+0000 7fb050400700 1 -- 172.21.15.43:0/3248554345 >> [v2:172.21.15.183:6808/787132252,v1:172.21.15.183:6809/787132252] conn(0x7fb0300adb00 msgr2=0x7fb0300affa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:39.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.726+0000 7fb050400700 1 --2- 172.21.15.43:0/3248554345 >> [v2:172.21.15.183:6808/787132252,v1:172.21.15.183:6809/787132252] conn(0x7fb0300adb00 0x7fb0300affa0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:39.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.726+0000 7fb050400700 1 -- 172.21.15.43:0/3248554345 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb0300a2920 msgr2=0x7fb0300a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:39.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.726+0000 7fb050400700 1 --2- 172.21.15.43:0/3248554345 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb0300a2920 0x7fb0300a4de0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb03c005950 tx=0x7fb03c00b410 comp rx=0 tx=0).stop 2024-03-26T08:32:39.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.726+0000 7fb050400700 1 -- 172.21.15.43:0/3248554345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb048113360 msgr2=0x7fb0481a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:39.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.726+0000 7fb050400700 1 --2- 172.21.15.43:0/3248554345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb048113360 0x7fb0481a3860 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb040007ef0 tx=0x7fb04000d3b0 comp rx=0 tx=0).stop 2024-03-26T08:32:39.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.726+0000 7fb04e400700 1 -- 172.21.15.43:0/3248554345 reap_dead start 2024-03-26T08:32:39.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.727+0000 7fb050400700 1 -- 172.21.15.43:0/3248554345 shutdown_connections 2024-03-26T08:32:39.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.727+0000 7fb050400700 1 -- 172.21.15.43:0/3248554345 >> 172.21.15.43:0/3248554345 conn(0x7fb0480792f0 msgr2=0x7fb04810ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:39.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.727+0000 7fb050400700 1 -- 172.21.15.43:0/3248554345 shutdown_connections 2024-03-26T08:32:39.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:39.727+0000 7fb050400700 1 -- 172.21.15.43:0/3248554345 wait complete. 2024-03-26T08:32:40.385 INFO:teuthology.orchestra.run.smithi043.stdout:347892351008 2024-03-26T08:32:40.386 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-26T08:32:41.216 INFO:teuthology.orchestra.run.smithi043.stdout:257698037815 2024-03-26T08:32:41.217 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-26T08:32:41.593 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:32:41.604 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:32:41.611 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:32:41.611 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:32:41.635 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:32:41.637 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:32:41.671 INFO:teuthology.orchestra.run.smithi043.stdout:167503724619 2024-03-26T08:32:41.672 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-26T08:32:41.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:41 smithi183 ceph-mon[35818]: pgmap v416: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:32:41.802 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:41 smithi043 ceph-mon[31929]: pgmap v416: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:32:41.802 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:32:41.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:41 smithi052 ceph-mon[35071]: pgmap v416: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:32:41.946 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:32:43.770 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:43 smithi043 ceph-mon[31929]: pgmap v417: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:32:43.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:43 smithi183 ceph-mon[35818]: pgmap v417: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:32:43.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:43 smithi052 ceph-mon[35071]: pgmap v417: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:32:45.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:45 smithi043 ceph-mon[31929]: pgmap v418: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-03-26T08:32:45.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:45 smithi183 ceph-mon[35818]: pgmap v418: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-03-26T08:32:45.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:45 smithi052 ceph-mon[35071]: pgmap v418: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-03-26T08:32:45.926 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:32:45.928 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:32:47.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:47 smithi043 ceph-mon[31929]: pgmap v419: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-26T08:32:47.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:47 smithi183 ceph-mon[35818]: pgmap v419: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-26T08:32:47.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:47 smithi052 ceph-mon[35071]: pgmap v419: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-03-26T08:32:49.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:49 smithi043 ceph-mon[31929]: pgmap v420: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1013 B/s rd, 1 op/s 2024-03-26T08:32:49.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:49 smithi183 ceph-mon[35818]: pgmap v420: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1013 B/s rd, 1 op/s 2024-03-26T08:32:49.823 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:49 smithi052 ceph-mon[35071]: pgmap v420: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1013 B/s rd, 1 op/s 2024-03-26T08:32:51.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:51 smithi043 ceph-mon[31929]: pgmap v421: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:32:51.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.701+0000 7fd1e3e00700 1 -- 172.21.15.43:0/4273211529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e4103240 msgr2=0x7fd1e4103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:51.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.701+0000 7fd1e3e00700 1 --2- 172.21.15.43:0/4273211529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e4103240 0x7fd1e4103620 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009990 tx=0x7fd1d4009ca0 comp rx=0 tx=0).stop 2024-03-26T08:32:51.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.702+0000 7fd1e3e00700 1 -- 172.21.15.43:0/4273211529 shutdown_connections 2024-03-26T08:32:51.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.702+0000 7fd1e3e00700 1 --2- 172.21.15.43:0/4273211529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1e410c670 0x7fd1e410eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:51.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.702+0000 7fd1e3e00700 1 --2- 172.21.15.43:0/4273211529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e4103240 0x7fd1e4103620 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:51.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.702+0000 7fd1e3e00700 1 --2- 172.21.15.43:0/4273211529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1e4103b60 0x7fd1e410c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:51.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.702+0000 7fd1e3e00700 1 -- 172.21.15.43:0/4273211529 >> 172.21.15.43:0/4273211529 conn(0x7fd1e40fca70 msgr2=0x7fd1e40fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:51.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.702+0000 7fd1e3e00700 1 -- 172.21.15.43:0/4273211529 shutdown_connections 2024-03-26T08:32:51.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.702+0000 7fd1e3e00700 1 -- 172.21.15.43:0/4273211529 wait complete. 2024-03-26T08:32:51.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.703+0000 7fd1e3e00700 1 Processor -- start 2024-03-26T08:32:51.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.703+0000 7fd1e3e00700 1 -- start start 2024-03-26T08:32:51.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.703+0000 7fd1e3e00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1e4103240 0x7fd1e4100ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:51.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.703+0000 7fd1e3e00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1e4103b60 0x7fd1e41013e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:51.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.703+0000 7fd1e3e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e410c670 0x7fd1e4101920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:51.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.703+0000 7fd1e3e00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e41122e0 con 0x7fd1e410c670 2024-03-26T08:32:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.703+0000 7fd1e3e00700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e4112160 con 0x7fd1e4103240 2024-03-26T08:32:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.703+0000 7fd1e3e00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e4112460 con 0x7fd1e4103b60 2024-03-26T08:32:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.704+0000 7fd1e3400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e410c670 0x7fd1e4101920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.704+0000 7fd1e3400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e410c670 0x7fd1e4101920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45276/0 (socket says 172.21.15.43:45276) 2024-03-26T08:32:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.704+0000 7fd1e3400700 1 -- 172.21.15.43:0/333191289 learned_addr learned my addr 172.21.15.43:0/333191289 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:32:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.704+0000 7fd1e2a00700 1 --2- 172.21.15.43:0/333191289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1e4103240 0x7fd1e4100ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.704+0000 7fd1e2000700 1 --2- 172.21.15.43:0/333191289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1e4103b60 0x7fd1e41013e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.704+0000 7fd1e3400700 1 -- 172.21.15.43:0/333191289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1e4103b60 msgr2=0x7fd1e41013e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.704+0000 7fd1e3400700 1 --2- 172.21.15.43:0/333191289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1e4103b60 0x7fd1e41013e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.704+0000 7fd1e3400700 1 -- 172.21.15.43:0/333191289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1e4103240 msgr2=0x7fd1e4100ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.704+0000 7fd1e3400700 1 --2- 172.21.15.43:0/333191289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1e4103240 0x7fd1e4100ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.704+0000 7fd1e3400700 1 -- 172.21.15.43:0/333191289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e41a6d40 con 0x7fd1e410c670 2024-03-26T08:32:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.704+0000 7fd1e2000700 1 --2- 172.21.15.43:0/333191289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1e4103b60 0x7fd1e41013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:32:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.704+0000 7fd1e3400700 1 --2- 172.21.15.43:0/333191289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e410c670 0x7fd1e4101920 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc007ee0 tx=0x7fd1dc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.705+0000 7fd1db400700 1 -- 172.21.15.43:0/333191289 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc00dbd0 con 0x7fd1e410c670 2024-03-26T08:32:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.705+0000 7fd1db400700 1 -- 172.21.15.43:0/333191289 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd1dc00dd70 con 0x7fd1e410c670 2024-03-26T08:32:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.705+0000 7fd1db400700 1 -- 172.21.15.43:0/333191289 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc005a20 con 0x7fd1e410c670 2024-03-26T08:32:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.705+0000 7fd1e3e00700 1 -- 172.21.15.43:0/333191289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1e41a7030 con 0x7fd1e410c670 2024-03-26T08:32:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.705+0000 7fd1e3e00700 1 -- 172.21.15.43:0/333191289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e41a74f0 con 0x7fd1e410c670 2024-03-26T08:32:51.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.706+0000 7fd1e3e00700 1 -- 172.21.15.43:0/333191289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1e4068e50 con 0x7fd1e410c670 2024-03-26T08:32:51.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.708+0000 7fd1db400700 1 -- 172.21.15.43:0/333191289 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fd1dc01a020 con 0x7fd1e410c670 2024-03-26T08:32:51.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.710+0000 7fd1db400700 1 --2- 172.21.15.43:0/333191289 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fd1c80a29a0 0x7fd1c80a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:51.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.710+0000 7fd1db400700 1 -- 172.21.15.43:0/333191289 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7fd1dc0ce6d0 con 0x7fd1e410c670 2024-03-26T08:32:51.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.710+0000 7fd1e2a00700 1 --2- 172.21.15.43:0/333191289 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fd1c80a29a0 0x7fd1c80a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:51.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.711+0000 7fd1e2a00700 1 --2- 172.21.15.43:0/333191289 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fd1c80a29a0 0x7fd1c80a4e60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400b5c0 tx=0x7fd1d401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:51.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.715+0000 7fd1db400700 1 -- 172.21.15.43:0/333191289 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd1dc010070 con 0x7fd1e410c670 2024-03-26T08:32:51.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:51 smithi183 ceph-mon[35818]: pgmap v421: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:32:51.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:51 smithi052 ceph-mon[35071]: pgmap v421: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:32:51.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.950+0000 7fd1e3e00700 1 -- 172.21.15.43:0/333191289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd1e402dbf0 con 0x7fd1e410c670 2024-03-26T08:32:51.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.950+0000 7fd1db400700 1 -- 172.21.15.43:0/333191289 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd1dc09b230 con 0x7fd1e410c670 2024-03-26T08:32:51.943 INFO:teuthology.orchestra.run.smithi043.stdout:77309411427 2024-03-26T08:32:51.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.953+0000 7fd1e3e00700 1 -- 172.21.15.43:0/333191289 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fd1c80a29a0 msgr2=0x7fd1c80a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:51.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.953+0000 7fd1e3e00700 1 --2- 172.21.15.43:0/333191289 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fd1c80a29a0 0x7fd1c80a4e60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400b5c0 tx=0x7fd1d401a040 comp rx=0 tx=0).stop 2024-03-26T08:32:51.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.953+0000 7fd1e3e00700 1 -- 172.21.15.43:0/333191289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e410c670 msgr2=0x7fd1e4101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:51.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.953+0000 7fd1e3e00700 1 --2- 172.21.15.43:0/333191289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e410c670 0x7fd1e4101920 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc007ee0 tx=0x7fd1dc00d3b0 comp rx=0 tx=0).stop 2024-03-26T08:32:51.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.954+0000 7fd1e3e00700 1 -- 172.21.15.43:0/333191289 shutdown_connections 2024-03-26T08:32:51.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.954+0000 7fd1e3e00700 1 --2- 172.21.15.43:0/333191289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1e410c670 0x7fd1e4101920 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:51.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.954+0000 7fd1e3e00700 1 --2- 172.21.15.43:0/333191289 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fd1c80a29a0 0x7fd1c80a4e60 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:51.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.954+0000 7fd1e3e00700 1 --2- 172.21.15.43:0/333191289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1e4103240 0x7fd1e4100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:51.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.954+0000 7fd1e3e00700 1 --2- 172.21.15.43:0/333191289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1e4103b60 0x7fd1e41013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:51.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.954+0000 7fd1e3e00700 1 -- 172.21.15.43:0/333191289 >> 172.21.15.43:0/333191289 conn(0x7fd1e40fca70 msgr2=0x7fd1e40fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:51.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.954+0000 7fd1e3e00700 1 -- 172.21.15.43:0/333191289 shutdown_connections 2024-03-26T08:32:51.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:51.954+0000 7fd1e3e00700 1 -- 172.21.15.43:0/333191289 wait complete. 2024-03-26T08:32:52.622 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:52 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/333191289' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-26T08:32:52.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:52 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/333191289' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-26T08:32:52.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:52 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/333191289' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-26T08:32:53.515 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:53 smithi043 ceph-mon[31929]: pgmap v422: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:53.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:53 smithi183 ceph-mon[35818]: pgmap v422: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:53.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:53 smithi052 ceph-mon[35071]: pgmap v422: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:53.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.809+0000 7f3f99600700 1 -- 172.21.15.43:0/3596480358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f9406a7e0 msgr2=0x7f3f9410f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:53.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.809+0000 7f3f99600700 1 --2- 172.21.15.43:0/3596480358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f9406a7e0 0x7f3f9410f020 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00b0d0 tx=0x7f3f8c00b3e0 comp rx=0 tx=0).stop 2024-03-26T08:32:53.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.809+0000 7f3f99600700 1 -- 172.21.15.43:0/3596480358 shutdown_connections 2024-03-26T08:32:53.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.809+0000 7f3f99600700 1 --2- 172.21.15.43:0/3596480358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f9406a7e0 0x7f3f9410f020 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:53.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.809+0000 7f3f99600700 1 --2- 172.21.15.43:0/3596480358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f941081e0 0x7f3f941085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:53.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.809+0000 7f3f99600700 1 --2- 172.21.15.43:0/3596480358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f94069e40 0x7f3f9406a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:53.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.809+0000 7f3f99600700 1 -- 172.21.15.43:0/3596480358 >> 172.21.15.43:0/3596480358 conn(0x7f3f94077fc0 msgr2=0x7f3f940783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:53.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.809+0000 7f3f99600700 1 -- 172.21.15.43:0/3596480358 shutdown_connections 2024-03-26T08:32:53.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.809+0000 7f3f99600700 1 -- 172.21.15.43:0/3596480358 wait complete. 2024-03-26T08:32:53.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.810+0000 7f3f99600700 1 Processor -- start 2024-03-26T08:32:53.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.810+0000 7f3f99600700 1 -- start start 2024-03-26T08:32:53.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.811+0000 7f3f99600700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f94069e40 0x7f3f94101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:53.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.811+0000 7f3f99600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f9406a7e0 0x7f3f941016a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:53.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.811+0000 7f3f99600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f941081e0 0x7f3f94101be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:53.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.811+0000 7f3f99600700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f94112360 con 0x7f3f941081e0 2024-03-26T08:32:53.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.811+0000 7f3f99600700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f941121e0 con 0x7f3f94069e40 2024-03-26T08:32:53.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.811+0000 7f3f99600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f941124e0 con 0x7f3f9406a7e0 2024-03-26T08:32:53.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.811+0000 7f3f93400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f941081e0 0x7f3f94101be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:53.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.811+0000 7f3f93400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f941081e0 0x7f3f94101be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45290/0 (socket says 172.21.15.43:45290) 2024-03-26T08:32:53.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.811+0000 7f3f93400700 1 -- 172.21.15.43:0/1023601214 learned_addr learned my addr 172.21.15.43:0/1023601214 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:32:53.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.811+0000 7f3f92a00700 1 --2- 172.21.15.43:0/1023601214 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f94069e40 0x7f3f94101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:53.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.811+0000 7f3f8be00700 1 --2- 172.21.15.43:0/1023601214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f9406a7e0 0x7f3f941016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:53.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.812+0000 7f3f93400700 1 -- 172.21.15.43:0/1023601214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f9406a7e0 msgr2=0x7f3f941016a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:53.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.812+0000 7f3f93400700 1 --2- 172.21.15.43:0/1023601214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f9406a7e0 0x7f3f941016a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:53.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.812+0000 7f3f93400700 1 -- 172.21.15.43:0/1023601214 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f94069e40 msgr2=0x7f3f94101160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:53.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.812+0000 7f3f93400700 1 --2- 172.21.15.43:0/1023601214 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f94069e40 0x7f3f94101160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:53.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.812+0000 7f3f93400700 1 -- 172.21.15.43:0/1023601214 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f941a6df0 con 0x7f3f941081e0 2024-03-26T08:32:53.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.812+0000 7f3f93400700 1 --2- 172.21.15.43:0/1023601214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f941081e0 0x7f3f94101be0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c0079a0 tx=0x7f3f8c0079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:53.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.813+0000 7f3f8b400700 1 -- 172.21.15.43:0/1023601214 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c01f070 con 0x7f3f941081e0 2024-03-26T08:32:53.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.813+0000 7f3f99600700 1 -- 172.21.15.43:0/1023601214 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f941a7080 con 0x7f3f941081e0 2024-03-26T08:32:53.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.813+0000 7f3f8b400700 1 -- 172.21.15.43:0/1023601214 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3f8c003c00 con 0x7f3f941081e0 2024-03-26T08:32:53.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.813+0000 7f3f8b400700 1 -- 172.21.15.43:0/1023601214 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c01b430 con 0x7f3f941081e0 2024-03-26T08:32:53.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.814+0000 7f3f99600700 1 -- 172.21.15.43:0/1023601214 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f941a7530 con 0x7f3f941081e0 2024-03-26T08:32:53.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.815+0000 7f3f99600700 1 -- 172.21.15.43:0/1023601214 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f94068e50 con 0x7f3f941081e0 2024-03-26T08:32:53.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.816+0000 7f3f8b400700 1 -- 172.21.15.43:0/1023601214 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f3f8c022070 con 0x7f3f941081e0 2024-03-26T08:32:53.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.819+0000 7f3f8b400700 1 --2- 172.21.15.43:0/1023601214 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3f780a29a0 0x7f3f780a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:53.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.819+0000 7f3f8b400700 1 -- 172.21.15.43:0/1023601214 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f3f8c00f020 con 0x7f3f941081e0 2024-03-26T08:32:53.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.819+0000 7f3f92a00700 1 --2- 172.21.15.43:0/1023601214 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3f780a29a0 0x7f3f780a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:53.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.820+0000 7f3f92a00700 1 --2- 172.21.15.43:0/1023601214 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3f780a29a0 0x7f3f780a4e60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3f84009be0 tx=0x7f3f84009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:53.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:53.824+0000 7f3f8b400700 1 -- 172.21.15.43:0/1023601214 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3f8c029020 con 0x7f3f941081e0 2024-03-26T08:32:54.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.063+0000 7f3f99600700 1 -- 172.21.15.43:0/1023601214 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3f9402da30 con 0x7f3f941081e0 2024-03-26T08:32:54.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.064+0000 7f3f8b400700 1 -- 172.21.15.43:0/1023601214 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3f8c003db0 con 0x7f3f941081e0 2024-03-26T08:32:54.057 INFO:teuthology.orchestra.run.smithi043.stdout:55834574953 2024-03-26T08:32:54.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.067+0000 7f3f99600700 1 -- 172.21.15.43:0/1023601214 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3f780a29a0 msgr2=0x7f3f780a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:54.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.067+0000 7f3f99600700 1 --2- 172.21.15.43:0/1023601214 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3f780a29a0 0x7f3f780a4e60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3f84009be0 tx=0x7f3f84009260 comp rx=0 tx=0).stop 2024-03-26T08:32:54.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.068+0000 7f3f99600700 1 -- 172.21.15.43:0/1023601214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f941081e0 msgr2=0x7f3f94101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:54.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.068+0000 7f3f99600700 1 --2- 172.21.15.43:0/1023601214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f941081e0 0x7f3f94101be0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c0079a0 tx=0x7f3f8c0079d0 comp rx=0 tx=0).stop 2024-03-26T08:32:54.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.068+0000 7f3f99600700 1 -- 172.21.15.43:0/1023601214 shutdown_connections 2024-03-26T08:32:54.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.068+0000 7f3f99600700 1 --2- 172.21.15.43:0/1023601214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f941081e0 0x7f3f94101be0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:54.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.068+0000 7f3f99600700 1 --2- 172.21.15.43:0/1023601214 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3f780a29a0 0x7f3f780a4e60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:54.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.068+0000 7f3f99600700 1 --2- 172.21.15.43:0/1023601214 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f94069e40 0x7f3f94101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:54.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.068+0000 7f3f99600700 1 --2- 172.21.15.43:0/1023601214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f9406a7e0 0x7f3f941016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:54.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.068+0000 7f3f99600700 1 -- 172.21.15.43:0/1023601214 >> 172.21.15.43:0/1023601214 conn(0x7f3f94077fc0 msgr2=0x7f3f940ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:54.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.068+0000 7f3f99600700 1 -- 172.21.15.43:0/1023601214 shutdown_connections 2024-03-26T08:32:54.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.068+0000 7f3f99600700 1 -- 172.21.15.43:0/1023601214 wait complete. 2024-03-26T08:32:54.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:54 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1023601214' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-26T08:32:54.732 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:54 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1023601214' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-26T08:32:54.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:54 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1023601214' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-26T08:32:55.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.682+0000 7f1953200700 1 -- 172.21.15.43:0/342141614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1954113080 msgr2=0x7f1954115470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:55.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.682+0000 7f1953200700 1 --2- 172.21.15.43:0/342141614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1954113080 0x7f1954115470 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f194c00b0a0 tx=0x7f194c00b3b0 comp rx=0 tx=0).stop 2024-03-26T08:32:55.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.682+0000 7f1953200700 1 -- 172.21.15.43:0/342141614 shutdown_connections 2024-03-26T08:32:55.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.682+0000 7f1953200700 1 --2- 172.21.15.43:0/342141614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1954113080 0x7f1954115470 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:55.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.682+0000 7f1953200700 1 --2- 172.21.15.43:0/342141614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1954074530 0x7f1954074910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:55.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.682+0000 7f1953200700 1 --2- 172.21.15.43:0/342141614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1954074e50 0x7f1954112b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:55.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.682+0000 7f1953200700 1 -- 172.21.15.43:0/342141614 >> 172.21.15.43:0/342141614 conn(0x7f19540fdb60 msgr2=0x7f19540fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:55.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.682+0000 7f1953200700 1 -- 172.21.15.43:0/342141614 shutdown_connections 2024-03-26T08:32:55.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.682+0000 7f1953200700 1 -- 172.21.15.43:0/342141614 wait complete. 2024-03-26T08:32:55.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.684+0000 7f1953200700 1 Processor -- start 2024-03-26T08:32:55.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.684+0000 7f1953200700 1 -- start start 2024-03-26T08:32:55.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.684+0000 7f1953200700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1954074530 0x7f195419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:55.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.684+0000 7f1953200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1954074e50 0x7f195419f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:55.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.684+0000 7f1953200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1954113080 0x7f19541a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:55.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.684+0000 7f1953200700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1954118890 con 0x7f1954113080 2024-03-26T08:32:55.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.684+0000 7f1953200700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1954118710 con 0x7f1954074530 2024-03-26T08:32:55.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.684+0000 7f1953200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1954118a10 con 0x7f1954074e50 2024-03-26T08:32:55.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.685+0000 7f1952800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1954113080 0x7f19541a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:55.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.685+0000 7f1952800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1954113080 0x7f19541a3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46046/0 (socket says 172.21.15.43:46046) 2024-03-26T08:32:55.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.685+0000 7f1952800700 1 -- 172.21.15.43:0/2086469449 learned_addr learned my addr 172.21.15.43:0/2086469449 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:32:55.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.685+0000 7f1951e00700 1 --2- 172.21.15.43:0/2086469449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1954074530 0x7f195419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:55.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.685+0000 7f1951400700 1 --2- 172.21.15.43:0/2086469449 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1954074e50 0x7f195419f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:55.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.685+0000 7f1952800700 1 -- 172.21.15.43:0/2086469449 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1954074e50 msgr2=0x7f195419f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:55.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.685+0000 7f1952800700 1 --2- 172.21.15.43:0/2086469449 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1954074e50 0x7f195419f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:55.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.685+0000 7f1952800700 1 -- 172.21.15.43:0/2086469449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1954074530 msgr2=0x7f195419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:55.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.685+0000 7f1952800700 1 --2- 172.21.15.43:0/2086469449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1954074530 0x7f195419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:55.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.685+0000 7f1952800700 1 -- 172.21.15.43:0/2086469449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19541a3d40 con 0x7f1954113080 2024-03-26T08:32:55.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.685+0000 7f1951e00700 1 --2- 172.21.15.43:0/2086469449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1954074530 0x7f195419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:32:55.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.686+0000 7f1952800700 1 --2- 172.21.15.43:0/2086469449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1954113080 0x7f19541a3600 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f194c00b070 tx=0x7f194c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:55.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.686+0000 7f194aa00700 1 -- 172.21.15.43:0/2086469449 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c021070 con 0x7f1954113080 2024-03-26T08:32:55.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.686+0000 7f1953200700 1 -- 172.21.15.43:0/2086469449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19541a3fd0 con 0x7f1954113080 2024-03-26T08:32:55.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.686+0000 7f194aa00700 1 -- 172.21.15.43:0/2086469449 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f194c007e30 con 0x7f1954113080 2024-03-26T08:32:55.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.686+0000 7f194aa00700 1 -- 172.21.15.43:0/2086469449 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c004700 con 0x7f1954113080 2024-03-26T08:32:55.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.686+0000 7f1953200700 1 -- 172.21.15.43:0/2086469449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19541ab8b0 con 0x7f1954113080 2024-03-26T08:32:55.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.687+0000 7f1953200700 1 -- 172.21.15.43:0/2086469449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1954068e50 con 0x7f1954113080 2024-03-26T08:32:55.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.689+0000 7f194aa00700 1 -- 172.21.15.43:0/2086469449 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f194c007980 con 0x7f1954113080 2024-03-26T08:32:55.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.691+0000 7f194aa00700 1 --2- 172.21.15.43:0/2086469449 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f19380a2920 0x7f19380a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:55.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.691+0000 7f194aa00700 1 -- 172.21.15.43:0/2086469449 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f194c00f020 con 0x7f1954113080 2024-03-26T08:32:55.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.691+0000 7f1951e00700 1 --2- 172.21.15.43:0/2086469449 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f19380a2920 0x7f19380a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:55.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.692+0000 7f1951e00700 1 --2- 172.21.15.43:0/2086469449 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f19380a2920 0x7f19380a4de0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1944009cb0 tx=0x7f1944009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:55.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.695+0000 7f194aa00700 1 -- 172.21.15.43:0/2086469449 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f194c02a020 con 0x7f1954113080 2024-03-26T08:32:55.139 INFO:teuthology.orchestra.run.smithi043.stdout:292057776178 2024-03-26T08:32:55.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.940+0000 7f1953200700 1 -- 172.21.15.43:0/2086469449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f195402da30 con 0x7f1954113080 2024-03-26T08:32:55.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.941+0000 7f194aa00700 1 -- 172.21.15.43:0/2086469449 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f194c09d0b0 con 0x7f1954113080 2024-03-26T08:32:55.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.945+0000 7f1953200700 1 -- 172.21.15.43:0/2086469449 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f19380a2920 msgr2=0x7f19380a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:55.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.945+0000 7f1953200700 1 --2- 172.21.15.43:0/2086469449 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f19380a2920 0x7f19380a4de0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1944009cb0 tx=0x7f1944009260 comp rx=0 tx=0).stop 2024-03-26T08:32:55.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.945+0000 7f1953200700 1 -- 172.21.15.43:0/2086469449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1954113080 msgr2=0x7f19541a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:55.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.945+0000 7f1953200700 1 --2- 172.21.15.43:0/2086469449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1954113080 0x7f19541a3600 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f194c00b070 tx=0x7f194c004d10 comp rx=0 tx=0).stop 2024-03-26T08:32:55.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.946+0000 7f1953200700 1 -- 172.21.15.43:0/2086469449 shutdown_connections 2024-03-26T08:32:55.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.946+0000 7f1953200700 1 --2- 172.21.15.43:0/2086469449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1954113080 0x7f19541a3600 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:55.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.946+0000 7f1953200700 1 --2- 172.21.15.43:0/2086469449 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f19380a2920 0x7f19380a4de0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:55.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.946+0000 7f1953200700 1 --2- 172.21.15.43:0/2086469449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1954074530 0x7f195419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:55.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.946+0000 7f1953200700 1 --2- 172.21.15.43:0/2086469449 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1954074e50 0x7f195419f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:55.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.946+0000 7f1953200700 1 -- 172.21.15.43:0/2086469449 >> 172.21.15.43:0/2086469449 conn(0x7f19540fdb60 msgr2=0x7f19540fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:55.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.946+0000 7f1953200700 1 -- 172.21.15.43:0/2086469449 shutdown_connections 2024-03-26T08:32:55.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:54.946+0000 7f1953200700 1 -- 172.21.15.43:0/2086469449 wait complete. 2024-03-26T08:32:55.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:55 smithi043 ceph-mon[31929]: pgmap v423: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:55.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:55 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2086469449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-26T08:32:55.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:55 smithi183 ceph-mon[35818]: pgmap v423: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:55.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:55 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2086469449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-26T08:32:55.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:55 smithi052 ceph-mon[35071]: pgmap v423: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:55.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:55 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2086469449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-26T08:32:56.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.131+0000 7fa678000700 1 -- 172.21.15.43:0/284063776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa670109160 msgr2=0x7fa670115670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:56.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.131+0000 7fa678000700 1 --2- 172.21.15.43:0/284063776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa670109160 0x7fa670115670 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa66800b0a0 tx=0x7fa66800b3b0 comp rx=0 tx=0).stop 2024-03-26T08:32:56.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.132+0000 7fa678000700 1 -- 172.21.15.43:0/284063776 shutdown_connections 2024-03-26T08:32:56.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.132+0000 7fa678000700 1 --2- 172.21.15.43:0/284063776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa670109160 0x7fa670115670 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:56.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.132+0000 7fa678000700 1 --2- 172.21.15.43:0/284063776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa67010e7a0 0x7fa67010eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:56.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.132+0000 7fa678000700 1 --2- 172.21.15.43:0/284063776 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6701087a0 0x7fa670108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:56.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.132+0000 7fa678000700 1 -- 172.21.15.43:0/284063776 >> 172.21.15.43:0/284063776 conn(0x7fa670075960 msgr2=0x7fa670077d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:56.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.132+0000 7fa678000700 1 -- 172.21.15.43:0/284063776 shutdown_connections 2024-03-26T08:32:56.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.132+0000 7fa678000700 1 -- 172.21.15.43:0/284063776 wait complete. 2024-03-26T08:32:56.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.133+0000 7fa678000700 1 Processor -- start 2024-03-26T08:32:56.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.134+0000 7fa678000700 1 -- start start 2024-03-26T08:32:56.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.134+0000 7fa678000700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6701087a0 0x7fa67019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:56.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.134+0000 7fa678000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa670109160 0x7fa67019f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:56.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.134+0000 7fa678000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa67010e7a0 0x7fa6701a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:56.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.134+0000 7fa678000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa670118a30 con 0x7fa67010e7a0 2024-03-26T08:32:56.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.134+0000 7fa678000700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6701188b0 con 0x7fa6701087a0 2024-03-26T08:32:56.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.134+0000 7fa678000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa670118bb0 con 0x7fa670109160 2024-03-26T08:32:56.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.135+0000 7fa676000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa67010e7a0 0x7fa6701a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:56.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.135+0000 7fa676000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa67010e7a0 0x7fa6701a36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46052/0 (socket says 172.21.15.43:46052) 2024-03-26T08:32:56.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.135+0000 7fa676000700 1 -- 172.21.15.43:0/3172278110 learned_addr learned my addr 172.21.15.43:0/3172278110 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:32:56.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.135+0000 7fa674c00700 1 --2- 172.21.15.43:0/3172278110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa670109160 0x7fa67019f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.135+0000 7fa676000700 1 -- 172.21.15.43:0/3172278110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa670109160 msgr2=0x7fa67019f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.135+0000 7fa676000700 1 --2- 172.21.15.43:0/3172278110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa670109160 0x7fa67019f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.135+0000 7fa676000700 1 -- 172.21.15.43:0/3172278110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6701087a0 msgr2=0x7fa67019ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.135+0000 7fa676000700 1 --2- 172.21.15.43:0/3172278110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6701087a0 0x7fa67019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.135+0000 7fa676000700 1 -- 172.21.15.43:0/3172278110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6701a3e10 con 0x7fa67010e7a0 2024-03-26T08:32:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.135+0000 7fa676000700 1 --2- 172.21.15.43:0/3172278110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa67010e7a0 0x7fa6701a36d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa66800b070 tx=0x7fa668004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.136+0000 7fa66e000700 1 -- 172.21.15.43:0/3172278110 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa668021070 con 0x7fa67010e7a0 2024-03-26T08:32:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.136+0000 7fa66e000700 1 -- 172.21.15.43:0/3172278110 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa668007e30 con 0x7fa67010e7a0 2024-03-26T08:32:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.136+0000 7fa66e000700 1 -- 172.21.15.43:0/3172278110 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa668004780 con 0x7fa67010e7a0 2024-03-26T08:32:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.136+0000 7fa678000700 1 -- 172.21.15.43:0/3172278110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6701a40a0 con 0x7fa67010e7a0 2024-03-26T08:32:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.136+0000 7fa678000700 1 -- 172.21.15.43:0/3172278110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6701ab980 con 0x7fa67010e7a0 2024-03-26T08:32:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.137+0000 7fa678000700 1 -- 172.21.15.43:0/3172278110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa670109ee0 con 0x7fa67010e7a0 2024-03-26T08:32:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.139+0000 7fa66e000700 1 -- 172.21.15.43:0/3172278110 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7fa668007980 con 0x7fa67010e7a0 2024-03-26T08:32:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.140+0000 7fa66e000700 1 --2- 172.21.15.43:0/3172278110 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa6580a2920 0x7fa6580a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.140+0000 7fa66e000700 1 -- 172.21.15.43:0/3172278110 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7fa6680cffb0 con 0x7fa67010e7a0 2024-03-26T08:32:56.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.141+0000 7fa675600700 1 --2- 172.21.15.43:0/3172278110 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa6580a2920 0x7fa6580a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:56.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.141+0000 7fa675600700 1 --2- 172.21.15.43:0/3172278110 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa6580a2920 0x7fa6580a4de0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa664009b60 tx=0x7fa664009620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:56.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.145+0000 7fa66e000700 1 -- 172.21.15.43:0/3172278110 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa66809cba0 con 0x7fa67010e7a0 2024-03-26T08:32:56.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.385+0000 7fa678000700 1 -- 172.21.15.43:0/3172278110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fa67002da30 con 0x7fa67010e7a0 2024-03-26T08:32:56.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.385+0000 7fa66e000700 1 -- 172.21.15.43:0/3172278110 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa668009cb0 con 0x7fa67010e7a0 2024-03-26T08:32:56.378 INFO:teuthology.orchestra.run.smithi043.stdout:146028888148 2024-03-26T08:32:56.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.389+0000 7fa678000700 1 -- 172.21.15.43:0/3172278110 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa6580a2920 msgr2=0x7fa6580a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:56.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.389+0000 7fa678000700 1 --2- 172.21.15.43:0/3172278110 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa6580a2920 0x7fa6580a4de0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa664009b60 tx=0x7fa664009620 comp rx=0 tx=0).stop 2024-03-26T08:32:56.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.389+0000 7fa678000700 1 -- 172.21.15.43:0/3172278110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa67010e7a0 msgr2=0x7fa6701a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:56.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.389+0000 7fa678000700 1 --2- 172.21.15.43:0/3172278110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa67010e7a0 0x7fa6701a36d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa66800b070 tx=0x7fa668004cf0 comp rx=0 tx=0).stop 2024-03-26T08:32:56.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.389+0000 7fa678000700 1 -- 172.21.15.43:0/3172278110 shutdown_connections 2024-03-26T08:32:56.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.389+0000 7fa678000700 1 --2- 172.21.15.43:0/3172278110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa67010e7a0 0x7fa6701a36d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:56.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.389+0000 7fa678000700 1 --2- 172.21.15.43:0/3172278110 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa6580a2920 0x7fa6580a4de0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:56.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.389+0000 7fa678000700 1 --2- 172.21.15.43:0/3172278110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6701087a0 0x7fa67019ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:56.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.389+0000 7fa678000700 1 --2- 172.21.15.43:0/3172278110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa670109160 0x7fa67019f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:56.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.389+0000 7fa678000700 1 -- 172.21.15.43:0/3172278110 >> 172.21.15.43:0/3172278110 conn(0x7fa670075960 msgr2=0x7fa670075ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:56.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.389+0000 7fa678000700 1 -- 172.21.15.43:0/3172278110 shutdown_connections 2024-03-26T08:32:56.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:56.389+0000 7fa678000700 1 -- 172.21.15.43:0/3172278110 wait complete. 2024-03-26T08:32:56.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:56 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3172278110' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-26T08:32:56.630 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:56 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3172278110' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-26T08:32:56.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:56 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3172278110' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-26T08:32:57.753 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:57 smithi043 ceph-mon[31929]: pgmap v424: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:32:57.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:57 smithi183 ceph-mon[35818]: pgmap v424: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:32:57.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:57 smithi052 ceph-mon[35071]: pgmap v424: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:32:59.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.316+0000 7f43f3a00700 1 -- 172.21.15.43:0/4250919272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec107080 msgr2=0x7f43ec069e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:59.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.316+0000 7f43f3a00700 1 --2- 172.21.15.43:0/4250919272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec107080 0x7f43ec069e40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009990 tx=0x7f43d0009ca0 comp rx=0 tx=0).stop 2024-03-26T08:32:59.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.316+0000 7f43f3a00700 1 -- 172.21.15.43:0/4250919272 shutdown_connections 2024-03-26T08:32:59.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.316+0000 7f43f3a00700 1 --2- 172.21.15.43:0/4250919272 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43ec104b90 0x7f43ec105020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:59.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.316+0000 7f43f3a00700 1 --2- 172.21.15.43:0/4250919272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec107080 0x7f43ec069e40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:59.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.316+0000 7f43f3a00700 1 --2- 172.21.15.43:0/4250919272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43ec06a380 0x7f43ec06a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:59.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.316+0000 7f43f3a00700 1 -- 172.21.15.43:0/4250919272 >> 172.21.15.43:0/4250919272 conn(0x7f43ec077fc0 msgr2=0x7f43ec0783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:59.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.316+0000 7f43f3a00700 1 -- 172.21.15.43:0/4250919272 shutdown_connections 2024-03-26T08:32:59.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.316+0000 7f43f3a00700 1 -- 172.21.15.43:0/4250919272 wait complete. 2024-03-26T08:32:59.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.317+0000 7f43f3a00700 1 Processor -- start 2024-03-26T08:32:59.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.317+0000 7f43f3a00700 1 -- start start 2024-03-26T08:32:59.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.318+0000 7f43f3a00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43ec06a380 0x7f43ec19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:59.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.318+0000 7f43f3a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43ec104b90 0x7f43ec19f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:59.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.318+0000 7f43f3a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec107080 0x7f43ec1a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:59.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.318+0000 7f43f3a00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec118910 con 0x7f43ec107080 2024-03-26T08:32:59.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.318+0000 7f43f3a00700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec118790 con 0x7f43ec06a380 2024-03-26T08:32:59.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.318+0000 7f43f3a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec118a90 con 0x7f43ec104b90 2024-03-26T08:32:59.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.318+0000 7f43f1a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec107080 0x7f43ec1a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:59.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.318+0000 7f43f1a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec107080 0x7f43ec1a3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46058/0 (socket says 172.21.15.43:46058) 2024-03-26T08:32:59.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.318+0000 7f43f1a00700 1 -- 172.21.15.43:0/166155582 learned_addr learned my addr 172.21.15.43:0/166155582 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:32:59.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.318+0000 7f43f1000700 1 --2- 172.21.15.43:0/166155582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43ec06a380 0x7f43ec19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:59.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.319+0000 7f43e3e00700 1 --2- 172.21.15.43:0/166155582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43ec104b90 0x7f43ec19f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:59.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.319+0000 7f43f1a00700 1 -- 172.21.15.43:0/166155582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43ec104b90 msgr2=0x7f43ec19f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:59.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.319+0000 7f43f1a00700 1 --2- 172.21.15.43:0/166155582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43ec104b90 0x7f43ec19f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:59.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.319+0000 7f43f1a00700 1 -- 172.21.15.43:0/166155582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43ec06a380 msgr2=0x7f43ec19eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:59.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.319+0000 7f43f1a00700 1 --2- 172.21.15.43:0/166155582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43ec06a380 0x7f43ec19eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:59.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.319+0000 7f43f1a00700 1 -- 172.21.15.43:0/166155582 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ec1a3eb0 con 0x7f43ec107080 2024-03-26T08:32:59.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.319+0000 7f43e3e00700 1 --2- 172.21.15.43:0/166155582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43ec104b90 0x7f43ec19f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:32:59.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.319+0000 7f43f1000700 1 --2- 172.21.15.43:0/166155582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43ec06a380 0x7f43ec19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:32:59.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.319+0000 7f43f1a00700 1 --2- 172.21.15.43:0/166155582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec107080 0x7f43ec1a3770 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f43e400ada0 tx=0x7f43e400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:59.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.320+0000 7f43ea000700 1 -- 172.21.15.43:0/166155582 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e4017020 con 0x7f43ec107080 2024-03-26T08:32:59.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.320+0000 7f43ea000700 1 -- 172.21.15.43:0/166155582 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f43e400ce30 con 0x7f43ec107080 2024-03-26T08:32:59.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.320+0000 7f43f3a00700 1 -- 172.21.15.43:0/166155582 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43ec1a41a0 con 0x7f43ec107080 2024-03-26T08:32:59.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.320+0000 7f43ea000700 1 -- 172.21.15.43:0/166155582 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e4005460 con 0x7f43ec107080 2024-03-26T08:32:59.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.320+0000 7f43f3a00700 1 -- 172.21.15.43:0/166155582 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43ec1aba20 con 0x7f43ec107080 2024-03-26T08:32:59.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.321+0000 7f43f3a00700 1 -- 172.21.15.43:0/166155582 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43ec068e50 con 0x7f43ec107080 2024-03-26T08:32:59.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.323+0000 7f43ea000700 1 -- 172.21.15.43:0/166155582 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f43e400c980 con 0x7f43ec107080 2024-03-26T08:32:59.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.324+0000 7f43ea000700 1 --2- 172.21.15.43:0/166155582 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f43c80a2920 0x7f43c80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:59.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.324+0000 7f43ea000700 1 -- 172.21.15.43:0/166155582 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f43e40ce7e0 con 0x7f43ec107080 2024-03-26T08:32:59.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.325+0000 7f43f1000700 1 --2- 172.21.15.43:0/166155582 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f43c80a2920 0x7f43c80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:59.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.325+0000 7f43f1000700 1 --2- 172.21.15.43:0/166155582 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f43c80a2920 0x7f43c80a4de0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009750 tx=0x7f43d0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:59.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.329+0000 7f43ea000700 1 -- 172.21.15.43:0/166155582 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f43e4021020 con 0x7f43ec107080 2024-03-26T08:32:59.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.567+0000 7f43f3a00700 1 -- 172.21.15.43:0/166155582 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f43ec1abbf0 con 0x7f43ec107080 2024-03-26T08:32:59.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.568+0000 7f43ea000700 1 -- 172.21.15.43:0/166155582 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f43e409b3d0 con 0x7f43ec107080 2024-03-26T08:32:59.561 INFO:teuthology.orchestra.run.smithi043.stdout:111669149789 2024-03-26T08:32:59.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:32:59 smithi043 ceph-mon[31929]: pgmap v425: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:59.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.571+0000 7f43f3a00700 1 -- 172.21.15.43:0/166155582 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f43c80a2920 msgr2=0x7f43c80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:59.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.571+0000 7f43f3a00700 1 --2- 172.21.15.43:0/166155582 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f43c80a2920 0x7f43c80a4de0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009750 tx=0x7f43d0005fd0 comp rx=0 tx=0).stop 2024-03-26T08:32:59.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.571+0000 7f43f3a00700 1 -- 172.21.15.43:0/166155582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec107080 msgr2=0x7f43ec1a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:59.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.571+0000 7f43f3a00700 1 --2- 172.21.15.43:0/166155582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec107080 0x7f43ec1a3770 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f43e400ada0 tx=0x7f43e400c330 comp rx=0 tx=0).stop 2024-03-26T08:32:59.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.572+0000 7f43f3a00700 1 -- 172.21.15.43:0/166155582 shutdown_connections 2024-03-26T08:32:59.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.572+0000 7f43f3a00700 1 --2- 172.21.15.43:0/166155582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43ec107080 0x7f43ec1a3770 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:59.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.572+0000 7f43f3a00700 1 --2- 172.21.15.43:0/166155582 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f43c80a2920 0x7f43c80a4de0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:59.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.572+0000 7f43f3a00700 1 --2- 172.21.15.43:0/166155582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43ec06a380 0x7f43ec19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:59.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.572+0000 7f43f3a00700 1 --2- 172.21.15.43:0/166155582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43ec104b90 0x7f43ec19f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:59.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.572+0000 7f43f3a00700 1 -- 172.21.15.43:0/166155582 >> 172.21.15.43:0/166155582 conn(0x7f43ec077fc0 msgr2=0x7f43ec100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:59.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.572+0000 7f43f3a00700 1 -- 172.21.15.43:0/166155582 shutdown_connections 2024-03-26T08:32:59.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:32:59.572+0000 7f43f3a00700 1 -- 172.21.15.43:0/166155582 wait complete. 2024-03-26T08:32:59.799 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:32:59 smithi183 ceph-mon[35818]: pgmap v425: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:32:59.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:32:59 smithi052 ceph-mon[35071]: pgmap v425: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:00.000 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411422 got 77309411427 for osd.2 2024-03-26T08:33:00.000 DEBUG:teuthology.parallel:result is None 2024-03-26T08:33:00.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.712+0000 7f815c000700 1 -- 172.21.15.43:0/2007422973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8154069e40 msgr2=0x7f815406a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:00.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.712+0000 7f815c000700 1 --2- 172.21.15.43:0/2007422973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8154069e40 0x7f815406a2a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f81400099c0 tx=0x7f8140009cd0 comp rx=0 tx=0).stop 2024-03-26T08:33:00.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.712+0000 7f815c000700 1 -- 172.21.15.43:0/2007422973 shutdown_connections 2024-03-26T08:33:00.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.712+0000 7f815c000700 1 --2- 172.21.15.43:0/2007422973 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f815406a7e0 0x7f815410f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:00.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.712+0000 7f815c000700 1 --2- 172.21.15.43:0/2007422973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81541081e0 0x7f81541085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:00.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.712+0000 7f815c000700 1 --2- 172.21.15.43:0/2007422973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8154069e40 0x7f815406a2a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:00.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.712+0000 7f815c000700 1 -- 172.21.15.43:0/2007422973 >> 172.21.15.43:0/2007422973 conn(0x7f8154077fc0 msgr2=0x7f81540783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:00.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.713+0000 7f815c000700 1 -- 172.21.15.43:0/2007422973 shutdown_connections 2024-03-26T08:33:00.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.713+0000 7f815c000700 1 -- 172.21.15.43:0/2007422973 wait complete. 2024-03-26T08:33:00.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.715+0000 7f815c000700 1 Processor -- start 2024-03-26T08:33:00.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.715+0000 7f815c000700 1 -- start start 2024-03-26T08:33:00.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.715+0000 7f815c000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8154069e40 0x7f81541a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:00.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.716+0000 7f815c000700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f815406a7e0 0x7f81541a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:00.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.716+0000 7f815c000700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81541081e0 0x7f815419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:00.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.716+0000 7f815c000700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81541123a0 con 0x7f8154069e40 2024-03-26T08:33:00.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.716+0000 7f815c000700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8154112220 con 0x7f815406a7e0 2024-03-26T08:33:00.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.716+0000 7f815c000700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8154112520 con 0x7f81541081e0 2024-03-26T08:33:00.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.716+0000 7f8159600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8154069e40 0x7f81541a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:00.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.716+0000 7f8159600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8154069e40 0x7f81541a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46076/0 (socket says 172.21.15.43:46076) 2024-03-26T08:33:00.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.716+0000 7f8159600700 1 -- 172.21.15.43:0/3546408197 learned_addr learned my addr 172.21.15.43:0/3546408197 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.716+0000 7f8158c00700 1 --2- 172.21.15.43:0/3546408197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f815406a7e0 0x7f81541a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.716+0000 7f815a000700 1 --2- 172.21.15.43:0/3546408197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81541081e0 0x7f815419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.717+0000 7f8159600700 1 -- 172.21.15.43:0/3546408197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81541081e0 msgr2=0x7f815419a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.717+0000 7f8159600700 1 --2- 172.21.15.43:0/3546408197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81541081e0 0x7f815419a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.717+0000 7f8159600700 1 -- 172.21.15.43:0/3546408197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f815406a7e0 msgr2=0x7f81541a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.717+0000 7f8159600700 1 --2- 172.21.15.43:0/3546408197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f815406a7e0 0x7f81541a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.717+0000 7f8159600700 1 -- 172.21.15.43:0/3546408197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815419b080 con 0x7f8154069e40 2024-03-26T08:33:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.717+0000 7f8159600700 1 --2- 172.21.15.43:0/3546408197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8154069e40 0x7f81541a0760 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f814800b530 tx=0x7f814800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:00.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.718+0000 7f8152000700 1 -- 172.21.15.43:0/3546408197 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8148015020 con 0x7f8154069e40 2024-03-26T08:33:00.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.718+0000 7f8152000700 1 -- 172.21.15.43:0/3546408197 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8148011ce0 con 0x7f8154069e40 2024-03-26T08:33:00.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.718+0000 7f815c000700 1 -- 172.21.15.43:0/3546408197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f815419b370 con 0x7f8154069e40 2024-03-26T08:33:00.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.718+0000 7f8152000700 1 -- 172.21.15.43:0/3546408197 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8148010dd0 con 0x7f8154069e40 2024-03-26T08:33:00.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.718+0000 7f815c000700 1 -- 172.21.15.43:0/3546408197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81541a7630 con 0x7f8154069e40 2024-03-26T08:33:00.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.719+0000 7f815c000700 1 -- 172.21.15.43:0/3546408197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8154068e50 con 0x7f8154069e40 2024-03-26T08:33:00.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.721+0000 7f8152000700 1 -- 172.21.15.43:0/3546408197 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f8148018020 con 0x7f8154069e40 2024-03-26T08:33:00.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.723+0000 7f8152000700 1 --2- 172.21.15.43:0/3546408197 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f813c0a29a0 0x7f813c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:00.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.723+0000 7f8152000700 1 -- 172.21.15.43:0/3546408197 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f81480cea80 con 0x7f8154069e40 2024-03-26T08:33:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.723+0000 7f8158c00700 1 --2- 172.21.15.43:0/3546408197 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f813c0a29a0 0x7f813c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.724+0000 7f8158c00700 1 --2- 172.21.15.43:0/3546408197 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f813c0a29a0 0x7f813c0a4e60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8154103180 tx=0x7f8140005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:00.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.728+0000 7f8152000700 1 -- 172.21.15.43:0/3546408197 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f814801f020 con 0x7f8154069e40 2024-03-26T08:33:00.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:00 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/166155582' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-26T08:33:00.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:00 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/166155582' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-26T08:33:00.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:00 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/166155582' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-26T08:33:00.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.903+0000 7f971d400700 1 -- 172.21.15.43:0/3934999144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f971810eb90 msgr2=0x7f9718110f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:00.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.903+0000 7f971d400700 1 --2- 172.21.15.43:0/3934999144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f971810eb90 0x7f9718110f80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9710009990 tx=0x7f9710009ca0 comp rx=0 tx=0).stop 2024-03-26T08:33:00.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.904+0000 7f971d400700 1 -- 172.21.15.43:0/3934999144 shutdown_connections 2024-03-26T08:33:00.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.904+0000 7f971d400700 1 --2- 172.21.15.43:0/3934999144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f971810eb90 0x7f9718110f80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:00.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.904+0000 7f971d400700 1 --2- 172.21.15.43:0/3934999144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97180743c0 0x7f97180747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:00.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.904+0000 7f971d400700 1 --2- 172.21.15.43:0/3934999144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9718074ce0 0x7f971810e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:00.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.904+0000 7f971d400700 1 -- 172.21.15.43:0/3934999144 >> 172.21.15.43:0/3934999144 conn(0x7f97180792c0 msgr2=0x7f97180796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:00.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.904+0000 7f971d400700 1 -- 172.21.15.43:0/3934999144 shutdown_connections 2024-03-26T08:33:00.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.904+0000 7f971d400700 1 -- 172.21.15.43:0/3934999144 wait complete. 2024-03-26T08:33:00.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.905+0000 7f971d400700 1 Processor -- start 2024-03-26T08:33:00.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.905+0000 7f971d400700 1 -- start start 2024-03-26T08:33:00.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.906+0000 7f971d400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97180743c0 0x7f97181a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:00.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.906+0000 7f971d400700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9718074ce0 0x7f97181a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:00.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.906+0000 7f971d400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f971810eb90 0x7f971819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:00.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.906+0000 7f971d400700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97181145a0 con 0x7f97180743c0 2024-03-26T08:33:00.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.906+0000 7f971d400700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9718114420 con 0x7f9718074ce0 2024-03-26T08:33:00.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.906+0000 7f971d400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9718114720 con 0x7f971810eb90 2024-03-26T08:33:00.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.906+0000 7f9716800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97180743c0 0x7f97181a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:00.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.906+0000 7f9716800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97180743c0 0x7f97181a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46090/0 (socket says 172.21.15.43:46090) 2024-03-26T08:33:00.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.906+0000 7f9716800700 1 -- 172.21.15.43:0/1610747377 learned_addr learned my addr 172.21.15.43:0/1610747377 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:00.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.907+0000 7f9715e00700 1 --2- 172.21.15.43:0/1610747377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9718074ce0 0x7f97181a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:00.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.907+0000 7f9717200700 1 --2- 172.21.15.43:0/1610747377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f971810eb90 0x7f971819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:00.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.907+0000 7f9716800700 1 -- 172.21.15.43:0/1610747377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f971810eb90 msgr2=0x7f971819a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:00.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.907+0000 7f9716800700 1 --2- 172.21.15.43:0/1610747377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f971810eb90 0x7f971819a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:00.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.907+0000 7f9716800700 1 -- 172.21.15.43:0/1610747377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9718074ce0 msgr2=0x7f97181a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:00.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.907+0000 7f9716800700 1 --2- 172.21.15.43:0/1610747377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9718074ce0 0x7f97181a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:00.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.907+0000 7f9716800700 1 -- 172.21.15.43:0/1610747377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f971819afe0 con 0x7f97180743c0 2024-03-26T08:33:00.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.907+0000 7f9716800700 1 --2- 172.21.15.43:0/1610747377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97180743c0 0x7f97181a06c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f970800ba50 tx=0x7f970800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:00.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.908+0000 7f970f400700 1 -- 172.21.15.43:0/1610747377 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9708019070 con 0x7f97180743c0 2024-03-26T08:33:00.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.908+0000 7f970f400700 1 -- 172.21.15.43:0/1610747377 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9708014440 con 0x7f97180743c0 2024-03-26T08:33:00.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.908+0000 7f971d400700 1 -- 172.21.15.43:0/1610747377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f971819b2d0 con 0x7f97180743c0 2024-03-26T08:33:00.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.908+0000 7f970f400700 1 -- 172.21.15.43:0/1610747377 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97080075d0 con 0x7f97180743c0 2024-03-26T08:33:00.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.908+0000 7f971d400700 1 -- 172.21.15.43:0/1610747377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97181a7590 con 0x7f97180743c0 2024-03-26T08:33:00.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.909+0000 7f971d400700 1 -- 172.21.15.43:0/1610747377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f8005330 con 0x7f97180743c0 2024-03-26T08:33:00.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.911+0000 7f970f400700 1 -- 172.21.15.43:0/1610747377 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f970801c070 con 0x7f97180743c0 2024-03-26T08:33:00.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.913+0000 7f970f400700 1 --2- 172.21.15.43:0/1610747377 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f96fc0a29a0 0x7f96fc0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:00.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.914+0000 7f970f400700 1 -- 172.21.15.43:0/1610747377 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f97080ce230 con 0x7f97180743c0 2024-03-26T08:33:00.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.914+0000 7f9715e00700 1 --2- 172.21.15.43:0/1610747377 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f96fc0a29a0 0x7f96fc0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:00.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.914+0000 7f9715e00700 1 --2- 172.21.15.43:0/1610747377 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f96fc0a29a0 0x7f96fc0a4e60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9704005950 tx=0x7f97040058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:00.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.917+0000 7f970f400700 1 -- 172.21.15.43:0/1610747377 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f970809aea0 con 0x7f97180743c0 2024-03-26T08:33:00.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.989+0000 7f815c000700 1 -- 172.21.15.43:0/3546408197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f815419be50 con 0x7f8154069e40 2024-03-26T08:33:00.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.989+0000 7f8152000700 1 -- 172.21.15.43:0/3546408197 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f814809b6f0 con 0x7f8154069e40 2024-03-26T08:33:00.982 INFO:teuthology.orchestra.run.smithi043.stdout:34359738481 2024-03-26T08:33:00.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.992+0000 7f815c000700 1 -- 172.21.15.43:0/3546408197 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f813c0a29a0 msgr2=0x7f813c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:00.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.992+0000 7f815c000700 1 --2- 172.21.15.43:0/3546408197 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f813c0a29a0 0x7f813c0a4e60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8154103180 tx=0x7f8140005fb0 comp rx=0 tx=0).stop 2024-03-26T08:33:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.992+0000 7f815c000700 1 -- 172.21.15.43:0/3546408197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8154069e40 msgr2=0x7f81541a0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.992+0000 7f815c000700 1 --2- 172.21.15.43:0/3546408197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8154069e40 0x7f81541a0760 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f814800b530 tx=0x7f814800b840 comp rx=0 tx=0).stop 2024-03-26T08:33:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.993+0000 7f815c000700 1 -- 172.21.15.43:0/3546408197 shutdown_connections 2024-03-26T08:33:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.993+0000 7f815c000700 1 --2- 172.21.15.43:0/3546408197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81541081e0 0x7f815419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.993+0000 7f815c000700 1 --2- 172.21.15.43:0/3546408197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8154069e40 0x7f81541a0760 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.993+0000 7f815c000700 1 --2- 172.21.15.43:0/3546408197 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f813c0a29a0 0x7f813c0a4e60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.993+0000 7f815c000700 1 --2- 172.21.15.43:0/3546408197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f815406a7e0 0x7f81541a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.993+0000 7f815c000700 1 -- 172.21.15.43:0/3546408197 >> 172.21.15.43:0/3546408197 conn(0x7f8154077fc0 msgr2=0x7f815410d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.993+0000 7f815c000700 1 -- 172.21.15.43:0/3546408197 shutdown_connections 2024-03-26T08:33:00.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:00.993+0000 7f815c000700 1 -- 172.21.15.43:0/3546408197 wait complete. 2024-03-26T08:33:01.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:01.170+0000 7f971d400700 1 -- 172.21.15.43:0/1610747377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f96f80058f0 con 0x7f97180743c0 2024-03-26T08:33:01.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:01.170+0000 7f970f400700 1 -- 172.21.15.43:0/1610747377 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f970809a7a0 con 0x7f97180743c0 2024-03-26T08:33:01.163 INFO:teuthology.orchestra.run.smithi043.stdout:313532612652 2024-03-26T08:33:01.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:01.173+0000 7f971d400700 1 -- 172.21.15.43:0/1610747377 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f96fc0a29a0 msgr2=0x7f96fc0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:01.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:01.173+0000 7f971d400700 1 --2- 172.21.15.43:0/1610747377 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f96fc0a29a0 0x7f96fc0a4e60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9704005950 tx=0x7f97040058e0 comp rx=0 tx=0).stop 2024-03-26T08:33:01.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:01.173+0000 7f971d400700 1 -- 172.21.15.43:0/1610747377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97180743c0 msgr2=0x7f97181a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:01.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:01.173+0000 7f971d400700 1 --2- 172.21.15.43:0/1610747377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97180743c0 0x7f97181a06c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f970800ba50 tx=0x7f970800be10 comp rx=0 tx=0).stop 2024-03-26T08:33:01.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:01.174+0000 7f971d400700 1 -- 172.21.15.43:0/1610747377 shutdown_connections 2024-03-26T08:33:01.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:01.174+0000 7f971d400700 1 --2- 172.21.15.43:0/1610747377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f971810eb90 0x7f971819a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:01.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:01.174+0000 7f971d400700 1 --2- 172.21.15.43:0/1610747377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97180743c0 0x7f97181a06c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:01.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:01.174+0000 7f971d400700 1 --2- 172.21.15.43:0/1610747377 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f96fc0a29a0 0x7f96fc0a4e60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:01.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:01.174+0000 7f971d400700 1 --2- 172.21.15.43:0/1610747377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9718074ce0 0x7f97181a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:01.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:01.174+0000 7f971d400700 1 -- 172.21.15.43:0/1610747377 >> 172.21.15.43:0/1610747377 conn(0x7f97180792c0 msgr2=0x7f97180ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:01.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:01.174+0000 7f971d400700 1 -- 172.21.15.43:0/1610747377 shutdown_connections 2024-03-26T08:33:01.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:01.174+0000 7f971d400700 1 -- 172.21.15.43:0/1610747377 wait complete. 2024-03-26T08:33:01.648 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776173 got 292057776178 for osd.9 2024-03-26T08:33:01.649 DEBUG:teuthology.parallel:result is None 2024-03-26T08:33:01.793 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:01 smithi043 ceph-mon[31929]: pgmap v426: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:33:01.793 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:01 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3546408197' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-26T08:33:01.793 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:01 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1610747377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-26T08:33:01.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:01 smithi183 ceph-mon[35818]: pgmap v426: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:33:01.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:01 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3546408197' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-26T08:33:01.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:01 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1610747377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-26T08:33:01.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:01 smithi052 ceph-mon[35071]: pgmap v426: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:33:01.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:01 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3546408197' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-26T08:33:01.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:01 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1610747377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-26T08:33:02.333 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888144 got 146028888148 for osd.4 2024-03-26T08:33:02.334 DEBUG:teuthology.parallel:result is None 2024-03-26T08:33:02.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.555+0000 7f9071800700 1 -- 172.21.15.43:0/3061281371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f906c1074b0 msgr2=0x7f906c10f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:02.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.555+0000 7f9071800700 1 --2- 172.21.15.43:0/3061281371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f906c1074b0 0x7f906c10f080 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f906400b0d0 tx=0x7f906400b3e0 comp rx=0 tx=0).stop 2024-03-26T08:33:02.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.555+0000 7f9071800700 1 -- 172.21.15.43:0/3061281371 shutdown_connections 2024-03-26T08:33:02.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.555+0000 7f9071800700 1 --2- 172.21.15.43:0/3061281371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f906c1074b0 0x7f906c10f080 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:02.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.555+0000 7f9071800700 1 --2- 172.21.15.43:0/3061281371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f906c1023d0 0x7f906c1027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:02.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.555+0000 7f9071800700 1 --2- 172.21.15.43:0/3061281371 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f906c102d80 0x7f906c106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:02.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.555+0000 7f9071800700 1 -- 172.21.15.43:0/3061281371 >> 172.21.15.43:0/3061281371 conn(0x7f906c0fdc40 msgr2=0x7f906c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:02.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.555+0000 7f9071800700 1 -- 172.21.15.43:0/3061281371 shutdown_connections 2024-03-26T08:33:02.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.555+0000 7f9071800700 1 -- 172.21.15.43:0/3061281371 wait complete. 2024-03-26T08:33:02.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.556+0000 7f9071800700 1 Processor -- start 2024-03-26T08:33:02.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.556+0000 7f9071800700 1 -- start start 2024-03-26T08:33:02.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.557+0000 7f9071800700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f906c1023d0 0x7f906c1a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:02.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.557+0000 7f9071800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f906c102d80 0x7f906c1a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:02.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.557+0000 7f9071800700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f906c1074b0 0x7f906c19a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:02.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.557+0000 7f9071800700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c112410 con 0x7f906c1023d0 2024-03-26T08:33:02.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.557+0000 7f9071800700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c112290 con 0x7f906c1074b0 2024-03-26T08:33:02.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.557+0000 7f9071800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c112590 con 0x7f906c102d80 2024-03-26T08:33:02.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.557+0000 7f906aa00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f906c1023d0 0x7f906c1a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:02.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.557+0000 7f906aa00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f906c1023d0 0x7f906c1a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46110/0 (socket says 172.21.15.43:46110) 2024-03-26T08:33:02.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.558+0000 7f906aa00700 1 -- 172.21.15.43:0/1329862471 learned_addr learned my addr 172.21.15.43:0/1329862471 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:02.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.558+0000 7f906b400700 1 --2- 172.21.15.43:0/1329862471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f906c1074b0 0x7f906c19a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:02.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.558+0000 7f906a000700 1 --2- 172.21.15.43:0/1329862471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f906c102d80 0x7f906c1a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:02.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.558+0000 7f906b400700 1 -- 172.21.15.43:0/1329862471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f906c102d80 msgr2=0x7f906c1a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:02.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.558+0000 7f906b400700 1 --2- 172.21.15.43:0/1329862471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f906c102d80 0x7f906c1a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:02.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.558+0000 7f906b400700 1 -- 172.21.15.43:0/1329862471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f906c1023d0 msgr2=0x7f906c1a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:02.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.558+0000 7f906b400700 1 --2- 172.21.15.43:0/1329862471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f906c1023d0 0x7f906c1a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:02.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.558+0000 7f906b400700 1 -- 172.21.15.43:0/1329862471 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f906c19b0b0 con 0x7f906c1074b0 2024-03-26T08:33:02.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.558+0000 7f906a000700 1 --2- 172.21.15.43:0/1329862471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f906c102d80 0x7f906c1a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:33:02.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.558+0000 7f906aa00700 1 --2- 172.21.15.43:0/1329862471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f906c1023d0 0x7f906c1a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:33:02.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.559+0000 7f906b400700 1 --2- 172.21.15.43:0/1329862471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f906c1074b0 0x7f906c19a950 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f906400b820 tx=0x7f90640079a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:02.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.559+0000 7f9063400700 1 -- 172.21.15.43:0/1329862471 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9064021020 con 0x7f906c1074b0 2024-03-26T08:33:02.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.559+0000 7f9071800700 1 -- 172.21.15.43:0/1329862471 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f906c19b340 con 0x7f906c1074b0 2024-03-26T08:33:02.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.560+0000 7f9071800700 1 -- 172.21.15.43:0/1329862471 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f906c1a7660 con 0x7f906c1074b0 2024-03-26T08:33:02.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.560+0000 7f9063400700 1 -- 172.21.15.43:0/1329862471 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9064004c50 con 0x7f906c1074b0 2024-03-26T08:33:02.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.560+0000 7f9063400700 1 -- 172.21.15.43:0/1329862471 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9064005050 con 0x7f906c1074b0 2024-03-26T08:33:02.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.561+0000 7f9071800700 1 -- 172.21.15.43:0/1329862471 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f906c068e50 con 0x7f906c1074b0 2024-03-26T08:33:02.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.564+0000 7f9063400700 1 -- 172.21.15.43:0/1329862471 <== mon.2 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f9064024020 con 0x7f906c1074b0 2024-03-26T08:33:02.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.565+0000 7f9063400700 1 --2- 172.21.15.43:0/1329862471 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f90500a29a0 0x7f90500a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:02.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.565+0000 7f9063400700 1 -- 172.21.15.43:0/1329862471 <== mon.2 v2:172.21.15.52:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f90640cff40 con 0x7f906c1074b0 2024-03-26T08:33:02.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.565+0000 7f906aa00700 1 --2- 172.21.15.43:0/1329862471 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f90500a29a0 0x7f90500a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:02.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.566+0000 7f906aa00700 1 --2- 172.21.15.43:0/1329862471 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f90500a29a0 0x7f90500a4e60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f906c19ba90 tx=0x7f905c009290 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:02.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.579+0000 7f9063400700 1 -- 172.21.15.43:0/1329862471 <== mon.2 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f906402b070 con 0x7f906c1074b0 2024-03-26T08:33:02.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.816+0000 7f9071800700 1 -- 172.21.15.43:0/1329862471 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f906c1a7830 con 0x7f906c1074b0 2024-03-26T08:33:02.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.817+0000 7f9063400700 1 -- 172.21.15.43:0/1329862471 <== mon.2 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f906409cb90 con 0x7f906c1074b0 2024-03-26T08:33:02.811 INFO:teuthology.orchestra.run.smithi043.stdout:223338299457 2024-03-26T08:33:02.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.821+0000 7f9071800700 1 -- 172.21.15.43:0/1329862471 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f90500a29a0 msgr2=0x7f90500a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:02.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.821+0000 7f9071800700 1 --2- 172.21.15.43:0/1329862471 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f90500a29a0 0x7f90500a4e60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f906c19ba90 tx=0x7f905c009290 comp rx=0 tx=0).stop 2024-03-26T08:33:02.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.821+0000 7f9071800700 1 -- 172.21.15.43:0/1329862471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f906c1074b0 msgr2=0x7f906c19a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:02.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.821+0000 7f9071800700 1 --2- 172.21.15.43:0/1329862471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f906c1074b0 0x7f906c19a950 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f906400b820 tx=0x7f90640079a0 comp rx=0 tx=0).stop 2024-03-26T08:33:02.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.821+0000 7f9071800700 1 -- 172.21.15.43:0/1329862471 shutdown_connections 2024-03-26T08:33:02.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.821+0000 7f9071800700 1 --2- 172.21.15.43:0/1329862471 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f90500a29a0 0x7f90500a4e60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:02.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.821+0000 7f9071800700 1 --2- 172.21.15.43:0/1329862471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f906c1074b0 0x7f906c19a950 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:02.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.821+0000 7f9071800700 1 --2- 172.21.15.43:0/1329862471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f906c1023d0 0x7f906c1a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:02.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.821+0000 7f9071800700 1 --2- 172.21.15.43:0/1329862471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f906c102d80 0x7f906c1a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:02.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.821+0000 7f9071800700 1 -- 172.21.15.43:0/1329862471 >> 172.21.15.43:0/1329862471 conn(0x7f906c0fdc40 msgr2=0x7f906c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:02.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.821+0000 7f9071800700 1 -- 172.21.15.43:0/1329862471 shutdown_connections 2024-03-26T08:33:02.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:02.821+0000 7f9071800700 1 -- 172.21.15.43:0/1329862471 wait complete. 2024-03-26T08:33:03.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.689+0000 7f63bbc00700 1 -- 172.21.15.43:0/2821742403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63b4109160 msgr2=0x7f63b4115670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:03.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.689+0000 7f63bbc00700 1 --2- 172.21.15.43:0/2821742403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63b4109160 0x7f63b4115670 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00b0a0 tx=0x7f63ac00b3b0 comp rx=0 tx=0).stop 2024-03-26T08:33:03.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.690+0000 7f63bbc00700 1 -- 172.21.15.43:0/2821742403 shutdown_connections 2024-03-26T08:33:03.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.690+0000 7f63bbc00700 1 --2- 172.21.15.43:0/2821742403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63b4109160 0x7f63b4115670 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:03.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.690+0000 7f63bbc00700 1 --2- 172.21.15.43:0/2821742403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63b410e7a0 0x7f63b410eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:03.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.690+0000 7f63bbc00700 1 --2- 172.21.15.43:0/2821742403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63b41087a0 0x7f63b4108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:03.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.690+0000 7f63bbc00700 1 -- 172.21.15.43:0/2821742403 >> 172.21.15.43:0/2821742403 conn(0x7f63b4075960 msgr2=0x7f63b4077d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:03.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.690+0000 7f63bbc00700 1 -- 172.21.15.43:0/2821742403 shutdown_connections 2024-03-26T08:33:03.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.690+0000 7f63bbc00700 1 -- 172.21.15.43:0/2821742403 wait complete. 2024-03-26T08:33:03.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.691+0000 7f63bbc00700 1 Processor -- start 2024-03-26T08:33:03.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.691+0000 7f63bbc00700 1 -- start start 2024-03-26T08:33:03.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.691+0000 7f63bbc00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63b41087a0 0x7f63b419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:03.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.691+0000 7f63bbc00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63b4109160 0x7f63b419f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:03.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.691+0000 7f63bbc00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63b410e7a0 0x7f63b41a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:03.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.691+0000 7f63bbc00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b41189d0 con 0x7f63b41087a0 2024-03-26T08:33:03.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.691+0000 7f63bbc00700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b4118850 con 0x7f63b410e7a0 2024-03-26T08:33:03.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.691+0000 7f63bbc00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b4118b50 con 0x7f63b4109160 2024-03-26T08:33:03.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.692+0000 7f63b9200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63b41087a0 0x7f63b419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:03.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.692+0000 7f63b9200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63b41087a0 0x7f63b419ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46118/0 (socket says 172.21.15.43:46118) 2024-03-26T08:33:03.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.692+0000 7f63b9200700 1 -- 172.21.15.43:0/1502395893 learned_addr learned my addr 172.21.15.43:0/1502395893 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:03.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.692+0000 7f63b9c00700 1 --2- 172.21.15.43:0/1502395893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63b410e7a0 0x7f63b41a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:03.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.692+0000 7f63b3e00700 1 --2- 172.21.15.43:0/1502395893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63b4109160 0x7f63b419f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:03.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.692+0000 7f63b9200700 1 -- 172.21.15.43:0/1502395893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63b4109160 msgr2=0x7f63b419f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:03.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.692+0000 7f63b9200700 1 --2- 172.21.15.43:0/1502395893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63b4109160 0x7f63b419f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:03.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.692+0000 7f63b9200700 1 -- 172.21.15.43:0/1502395893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63b410e7a0 msgr2=0x7f63b41a36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:03.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.692+0000 7f63b9200700 1 --2- 172.21.15.43:0/1502395893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63b410e7a0 0x7f63b41a36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:03.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.692+0000 7f63b9200700 1 -- 172.21.15.43:0/1502395893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b41a3df0 con 0x7f63b41087a0 2024-03-26T08:33:03.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.692+0000 7f63b9c00700 1 --2- 172.21.15.43:0/1502395893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63b410e7a0 0x7f63b41a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:33:03.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.693+0000 7f63b9200700 1 --2- 172.21.15.43:0/1502395893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63b41087a0 0x7f63b419ede0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f63a800d980 tx=0x7f63a800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:03.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.693+0000 7f63b1600700 1 -- 172.21.15.43:0/1502395893 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a8016070 con 0x7f63b41087a0 2024-03-26T08:33:03.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.693+0000 7f63b1600700 1 -- 172.21.15.43:0/1502395893 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f63a80044d0 con 0x7f63b41087a0 2024-03-26T08:33:03.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.693+0000 7f63bbc00700 1 -- 172.21.15.43:0/1502395893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b41a40e0 con 0x7f63b41087a0 2024-03-26T08:33:03.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.693+0000 7f63b1600700 1 -- 172.21.15.43:0/1502395893 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a8004cf0 con 0x7f63b41087a0 2024-03-26T08:33:03.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.693+0000 7f63bbc00700 1 -- 172.21.15.43:0/1502395893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b41ab960 con 0x7f63b41087a0 2024-03-26T08:33:03.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.694+0000 7f63bbc00700 1 -- 172.21.15.43:0/1502395893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b4068e50 con 0x7f63b41087a0 2024-03-26T08:33:03.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.697+0000 7f63b1600700 1 -- 172.21.15.43:0/1502395893 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f63a8004020 con 0x7f63b41087a0 2024-03-26T08:33:03.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.698+0000 7f63b1600700 1 --2- 172.21.15.43:0/1502395893 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f639c0a2920 0x7f639c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:03.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.698+0000 7f63b1600700 1 -- 172.21.15.43:0/1502395893 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f63a80ced60 con 0x7f63b41087a0 2024-03-26T08:33:03.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.698+0000 7f63b3e00700 1 --2- 172.21.15.43:0/1502395893 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f639c0a2920 0x7f639c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:03.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.699+0000 7f63b3e00700 1 --2- 172.21.15.43:0/1502395893 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f639c0a2920 0x7f639c0a4de0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f63a0005fd0 tx=0x7f63a0005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:03.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.702+0000 7f63b1600700 1 -- 172.21.15.43:0/1502395893 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f63a801f020 con 0x7f63b41087a0 2024-03-26T08:33:03.795 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:03 smithi043 ceph-mon[31929]: pgmap v427: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:03.796 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:03 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1329862471' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-26T08:33:03.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:03 smithi183 ceph-mon[35818]: pgmap v427: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:03.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:03 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1329862471' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-26T08:33:03.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:03 smithi052 ceph-mon[35071]: pgmap v427: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:03.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:03 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1329862471' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-26T08:33:03.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.942+0000 7f63bbc00700 1 -- 172.21.15.43:0/1502395893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f63b402da30 con 0x7f63b41087a0 2024-03-26T08:33:03.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.943+0000 7f63b1600700 1 -- 172.21.15.43:0/1502395893 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f63a809b8c0 con 0x7f63b41087a0 2024-03-26T08:33:03.936 INFO:teuthology.orchestra.run.smithi043.stdout:347892351013 2024-03-26T08:33:03.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.946+0000 7f63bbc00700 1 -- 172.21.15.43:0/1502395893 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f639c0a2920 msgr2=0x7f639c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:03.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.946+0000 7f63bbc00700 1 --2- 172.21.15.43:0/1502395893 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f639c0a2920 0x7f639c0a4de0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f63a0005fd0 tx=0x7f63a0005f60 comp rx=0 tx=0).stop 2024-03-26T08:33:03.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.946+0000 7f63bbc00700 1 -- 172.21.15.43:0/1502395893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63b41087a0 msgr2=0x7f63b419ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:03.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.946+0000 7f63bbc00700 1 --2- 172.21.15.43:0/1502395893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63b41087a0 0x7f63b419ede0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f63a800d980 tx=0x7f63a800dd40 comp rx=0 tx=0).stop 2024-03-26T08:33:03.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.946+0000 7f63bbc00700 1 -- 172.21.15.43:0/1502395893 shutdown_connections 2024-03-26T08:33:03.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.946+0000 7f63bbc00700 1 --2- 172.21.15.43:0/1502395893 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f639c0a2920 0x7f639c0a4de0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:03.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.946+0000 7f63bbc00700 1 --2- 172.21.15.43:0/1502395893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63b410e7a0 0x7f63b41a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:03.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.946+0000 7f63bbc00700 1 --2- 172.21.15.43:0/1502395893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63b41087a0 0x7f63b419ede0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:03.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.946+0000 7f63bbc00700 1 --2- 172.21.15.43:0/1502395893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63b4109160 0x7f63b419f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:03.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.946+0000 7f63bbc00700 1 -- 172.21.15.43:0/1502395893 >> 172.21.15.43:0/1502395893 conn(0x7f63b4075960 msgr2=0x7f63b4075ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:03.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.947+0000 7f63bbc00700 1 -- 172.21.15.43:0/1502395893 shutdown_connections 2024-03-26T08:33:03.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:03.947+0000 7f63bbc00700 1 -- 172.21.15.43:0/1502395893 wait complete. 2024-03-26T08:33:03.981 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574946 got 55834574953 for osd.1 2024-03-26T08:33:03.981 DEBUG:teuthology.parallel:result is None 2024-03-26T08:33:04.603 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738473 got 34359738481 for osd.0 2024-03-26T08:33:04.603 DEBUG:teuthology.parallel:result is None 2024-03-26T08:33:04.779 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:04 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1502395893' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-26T08:33:04.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:04 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1502395893' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-26T08:33:04.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:04 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1502395893' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-26T08:33:05.328 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149782 got 111669149789 for osd.3 2024-03-26T08:33:05.328 DEBUG:teuthology.parallel:result is None 2024-03-26T08:33:05.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:05 smithi043 ceph-mon[31929]: pgmap v428: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:05.652 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:05 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3342602383' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-26T08:33:05.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:05 smithi183 ceph-mon[35818]: pgmap v428: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:05.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:05 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3342602383' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-26T08:33:05.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:05 smithi052 ceph-mon[35071]: pgmap v428: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:05.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:05 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3342602383' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-26T08:33:05.830 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299451 got 223338299457 for osd.7 2024-03-26T08:33:05.830 DEBUG:teuthology.parallel:result is None 2024-03-26T08:33:06.216 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612646 got 313532612652 for osd.10 2024-03-26T08:33:06.216 DEBUG:teuthology.parallel:result is None 2024-03-26T08:33:06.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.140+0000 7f0de1400700 1 -- 172.21.15.43:0/2433168129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ddc109160 msgr2=0x7f0ddc115670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:06.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.140+0000 7f0de1400700 1 --2- 172.21.15.43:0/2433168129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ddc109160 0x7f0ddc115670 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0dd400b0a0 tx=0x7f0dd400b3b0 comp rx=0 tx=0).stop 2024-03-26T08:33:06.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.140+0000 7f0de1400700 1 -- 172.21.15.43:0/2433168129 shutdown_connections 2024-03-26T08:33:06.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.140+0000 7f0de1400700 1 --2- 172.21.15.43:0/2433168129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ddc109160 0x7f0ddc115670 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:06.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.140+0000 7f0de1400700 1 --2- 172.21.15.43:0/2433168129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ddc10e7a0 0x7f0ddc10eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:06.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.140+0000 7f0de1400700 1 --2- 172.21.15.43:0/2433168129 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ddc1087a0 0x7f0ddc108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.140+0000 7f0de1400700 1 -- 172.21.15.43:0/2433168129 >> 172.21.15.43:0/2433168129 conn(0x7f0ddc075960 msgr2=0x7f0ddc077d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.140+0000 7f0de1400700 1 -- 172.21.15.43:0/2433168129 shutdown_connections 2024-03-26T08:33:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.140+0000 7f0de1400700 1 -- 172.21.15.43:0/2433168129 wait complete. 2024-03-26T08:33:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.142+0000 7f0de1400700 1 Processor -- start 2024-03-26T08:33:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.142+0000 7f0de1400700 1 -- start start 2024-03-26T08:33:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.142+0000 7f0de1400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ddc1087a0 0x7f0ddc1a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.142+0000 7f0de1400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ddc109160 0x7f0ddc1a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.142+0000 7f0de1400700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ddc10e7a0 0x7f0ddc19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.142+0000 7f0de1400700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc1189d0 con 0x7f0ddc109160 2024-03-26T08:33:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.142+0000 7f0de1400700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc118850 con 0x7f0ddc10e7a0 2024-03-26T08:33:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.142+0000 7f0de1400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc118b50 con 0x7f0ddc1087a0 2024-03-26T08:33:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.143+0000 7f0dd9e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ddc109160 0x7f0ddc1a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.143+0000 7f0dd9e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ddc109160 0x7f0ddc1a4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35848/0 (socket says 172.21.15.43:35848) 2024-03-26T08:33:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.143+0000 7f0dd9e00700 1 -- 172.21.15.43:0/3342602383 learned_addr learned my addr 172.21.15.43:0/3342602383 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:06.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.143+0000 7f0ddb200700 1 --2- 172.21.15.43:0/3342602383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ddc10e7a0 0x7f0ddc19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:06.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.143+0000 7f0dda800700 1 --2- 172.21.15.43:0/3342602383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ddc1087a0 0x7f0ddc1a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:06.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.143+0000 7f0dd9e00700 1 -- 172.21.15.43:0/3342602383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ddc1087a0 msgr2=0x7f0ddc1a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:06.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.143+0000 7f0dd9e00700 1 --2- 172.21.15.43:0/3342602383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ddc1087a0 0x7f0ddc1a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:06.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.143+0000 7f0dd9e00700 1 -- 172.21.15.43:0/3342602383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ddc10e7a0 msgr2=0x7f0ddc19eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:06.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.143+0000 7f0dd9e00700 1 --2- 172.21.15.43:0/3342602383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ddc10e7a0 0x7f0ddc19eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:06.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.143+0000 7f0dd9e00700 1 -- 172.21.15.43:0/3342602383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ddc19f2b0 con 0x7f0ddc109160 2024-03-26T08:33:06.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.143+0000 7f0ddb200700 1 --2- 172.21.15.43:0/3342602383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ddc10e7a0 0x7f0ddc19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:33:06.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.144+0000 7f0dd9e00700 1 --2- 172.21.15.43:0/3342602383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ddc109160 0x7f0ddc1a4e80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0dc800c9b0 tx=0x7f0dc800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:06.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.144+0000 7f0dd3400700 1 -- 172.21.15.43:0/3342602383 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8007b00 con 0x7f0ddc109160 2024-03-26T08:33:06.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.144+0000 7f0dd3400700 1 -- 172.21.15.43:0/3342602383 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0dc8007ca0 con 0x7f0ddc109160 2024-03-26T08:33:06.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.144+0000 7f0de1400700 1 -- 172.21.15.43:0/3342602383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ddc19f5a0 con 0x7f0ddc109160 2024-03-26T08:33:06.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.144+0000 7f0dd3400700 1 -- 172.21.15.43:0/3342602383 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8018730 con 0x7f0ddc109160 2024-03-26T08:33:06.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.144+0000 7f0de1400700 1 -- 172.21.15.43:0/3342602383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ddc1ab810 con 0x7f0ddc109160 2024-03-26T08:33:06.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.145+0000 7f0de1400700 1 -- 172.21.15.43:0/3342602383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ddc068e50 con 0x7f0ddc109160 2024-03-26T08:33:06.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.147+0000 7f0dd3400700 1 -- 172.21.15.43:0/3342602383 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 137341+0+0 (secure 0 0 0) 0x7f0dc800f450 con 0x7f0ddc109160 2024-03-26T08:33:06.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.149+0000 7f0dd3400700 1 --2- 172.21.15.43:0/3342602383 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0dc00a2920 0x7f0dc00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:06.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.149+0000 7f0dd3400700 1 -- 172.21.15.43:0/3342602383 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f0dc80cf340 con 0x7f0ddc109160 2024-03-26T08:33:06.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.149+0000 7f0dda800700 1 --2- 172.21.15.43:0/3342602383 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0dc00a2920 0x7f0dc00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:06.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.150+0000 7f0dda800700 1 --2- 172.21.15.43:0/3342602383 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0dc00a2920 0x7f0dc00a4de0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc005fd0 tx=0x7f0dcc00af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:06.325 INFO:teuthology.orchestra.run.smithi043.stdout:201863462984 2024-03-26T08:33:06.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.152+0000 7f0dd3400700 1 -- 172.21.15.43:0/3342602383 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0dc8014030 con 0x7f0ddc109160 2024-03-26T08:33:06.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.390+0000 7f0de1400700 1 -- 172.21.15.43:0/3342602383 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f0ddc19fff0 con 0x7f0ddc109160 2024-03-26T08:33:06.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.391+0000 7f0dd3400700 1 -- 172.21.15.43:0/3342602383 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0ddc19fff0 con 0x7f0ddc109160 2024-03-26T08:33:06.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.394+0000 7f0de1400700 1 -- 172.21.15.43:0/3342602383 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0dc00a2920 msgr2=0x7f0dc00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:06.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.394+0000 7f0de1400700 1 --2- 172.21.15.43:0/3342602383 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0dc00a2920 0x7f0dc00a4de0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc005fd0 tx=0x7f0dcc00af20 comp rx=0 tx=0).stop 2024-03-26T08:33:06.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.394+0000 7f0de1400700 1 -- 172.21.15.43:0/3342602383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ddc109160 msgr2=0x7f0ddc1a4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:06.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.394+0000 7f0de1400700 1 --2- 172.21.15.43:0/3342602383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ddc109160 0x7f0ddc1a4e80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0dc800c9b0 tx=0x7f0dc800cd70 comp rx=0 tx=0).stop 2024-03-26T08:33:06.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.394+0000 7f0de1400700 1 -- 172.21.15.43:0/3342602383 shutdown_connections 2024-03-26T08:33:06.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.394+0000 7f0de1400700 1 --2- 172.21.15.43:0/3342602383 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0dc00a2920 0x7f0dc00a4de0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:06.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.394+0000 7f0de1400700 1 --2- 172.21.15.43:0/3342602383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ddc10e7a0 0x7f0ddc19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:06.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.394+0000 7f0de1400700 1 --2- 172.21.15.43:0/3342602383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ddc1087a0 0x7f0ddc1a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:06.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.394+0000 7f0de1400700 1 --2- 172.21.15.43:0/3342602383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ddc109160 0x7f0ddc1a4e80 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:06.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.394+0000 7f0de1400700 1 -- 172.21.15.43:0/3342602383 >> 172.21.15.43:0/3342602383 conn(0x7f0ddc075960 msgr2=0x7f0ddc075de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:06.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.395+0000 7f0de1400700 1 -- 172.21.15.43:0/3342602383 shutdown_connections 2024-03-26T08:33:06.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:05.395+0000 7f0de1400700 1 -- 172.21.15.43:0/3342602383 wait complete. 2024-03-26T08:33:06.692 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:33:07.092 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462977 got 201863462984 for osd.6 2024-03-26T08:33:07.092 DEBUG:teuthology.parallel:result is None 2024-03-26T08:33:07.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:07 smithi052 ceph-mon[35071]: pgmap v429: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:33:07.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:07 smithi043 ceph-mon[31929]: pgmap v429: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:33:08.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:07 smithi183 ceph-mon[35818]: pgmap v429: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:33:08.378 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:33:08.403 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351008 got 347892351013 for osd.11 2024-03-26T08:33:08.403 DEBUG:teuthology.parallel:result is None 2024-03-26T08:33:08.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:08 smithi043 ceph-mon[31929]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-03-26T08:33:08.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:08 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:33:08.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:08 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:33:09.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:08 smithi183 ceph-mon[35818]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-03-26T08:33:09.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:08 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:33:09.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:08 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:33:09.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:08 smithi052 ceph-mon[35071]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-03-26T08:33:09.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:08 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:33:09.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:08 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:33:09.882 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:09 smithi043 ceph-mon[31929]: pgmap v430: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:09.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.947+0000 7f8c74a00700 1 -- 172.21.15.43:0/3815472590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c701045d0 msgr2=0x7f8c701049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:09.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.947+0000 7f8c74a00700 1 --2- 172.21.15.43:0/3815472590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c701045d0 0x7f8c701049b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8c600099c0 tx=0x7f8c60009cd0 comp rx=0 tx=0).stop 2024-03-26T08:33:09.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.947+0000 7f8c74a00700 1 -- 172.21.15.43:0/3815472590 shutdown_connections 2024-03-26T08:33:09.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.948+0000 7f8c74a00700 1 --2- 172.21.15.43:0/3815472590 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c701096b0 0x7f8c70111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:09.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.948+0000 7f8c74a00700 1 --2- 172.21.15.43:0/3815472590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c701045d0 0x7f8c701049b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:09.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.948+0000 7f8c74a00700 1 --2- 172.21.15.43:0/3815472590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c70104f80 0x7f8c70108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:09.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.948+0000 7f8c74a00700 1 -- 172.21.15.43:0/3815472590 >> 172.21.15.43:0/3815472590 conn(0x7f8c700ffea0 msgr2=0x7f8c701022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:09.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.948+0000 7f8c74a00700 1 -- 172.21.15.43:0/3815472590 shutdown_connections 2024-03-26T08:33:09.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.948+0000 7f8c74a00700 1 -- 172.21.15.43:0/3815472590 wait complete. 2024-03-26T08:33:09.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.949+0000 7f8c74a00700 1 Processor -- start 2024-03-26T08:33:09.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.949+0000 7f8c74a00700 1 -- start start 2024-03-26T08:33:09.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.949+0000 7f8c74a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c701045d0 0x7f8c701a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:09.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.949+0000 7f8c74a00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c70104f80 0x7f8c701a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:09.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.949+0000 7f8c74a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c701096b0 0x7f8c7019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:09.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.950+0000 7f8c74a00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c70114560 con 0x7f8c701045d0 2024-03-26T08:33:09.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.950+0000 7f8c74a00700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c701143e0 con 0x7f8c70104f80 2024-03-26T08:33:09.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.950+0000 7f8c74a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c701146e0 con 0x7f8c701096b0 2024-03-26T08:33:09.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.950+0000 7f8c6de00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c701045d0 0x7f8c701a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:09.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.950+0000 7f8c6de00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c701045d0 0x7f8c701a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35872/0 (socket says 172.21.15.43:35872) 2024-03-26T08:33:09.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.950+0000 7f8c6de00700 1 -- 172.21.15.43:0/342815657 learned_addr learned my addr 172.21.15.43:0/342815657 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:09.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.950+0000 7f8c6d400700 1 --2- 172.21.15.43:0/342815657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c70104f80 0x7f8c701a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:09.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.950+0000 7f8c6e800700 1 --2- 172.21.15.43:0/342815657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c701096b0 0x7f8c7019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:09.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.950+0000 7f8c6de00700 1 -- 172.21.15.43:0/342815657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c701096b0 msgr2=0x7f8c7019a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:09.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.950+0000 7f8c6de00700 1 --2- 172.21.15.43:0/342815657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c701096b0 0x7f8c7019a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:09.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.950+0000 7f8c6de00700 1 -- 172.21.15.43:0/342815657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c70104f80 msgr2=0x7f8c701a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:09.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.950+0000 7f8c6de00700 1 --2- 172.21.15.43:0/342815657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c70104f80 0x7f8c701a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:09.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.950+0000 7f8c6de00700 1 -- 172.21.15.43:0/342815657 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7019b0c0 con 0x7f8c701045d0 2024-03-26T08:33:09.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.951+0000 7f8c6de00700 1 --2- 172.21.15.43:0/342815657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c701045d0 0x7f8c701a0750 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8c600059d0 tx=0x7f8c600057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:09.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.951+0000 7f8c66a00700 1 -- 172.21.15.43:0/342815657 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c60027070 con 0x7f8c701045d0 2024-03-26T08:33:09.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.951+0000 7f8c66a00700 1 -- 172.21.15.43:0/342815657 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8c60003f50 con 0x7f8c701045d0 2024-03-26T08:33:09.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.951+0000 7f8c74a00700 1 -- 172.21.15.43:0/342815657 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c7019b350 con 0x7f8c701045d0 2024-03-26T08:33:09.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.951+0000 7f8c66a00700 1 -- 172.21.15.43:0/342815657 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c60022d80 con 0x7f8c701045d0 2024-03-26T08:33:09.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.951+0000 7f8c74a00700 1 -- 172.21.15.43:0/342815657 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c701a7620 con 0x7f8c701045d0 2024-03-26T08:33:09.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.952+0000 7f8c74a00700 1 -- 172.21.15.43:0/342815657 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c70105ab0 con 0x7f8c701045d0 2024-03-26T08:33:09.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.955+0000 7f8c66a00700 1 -- 172.21.15.43:0/342815657 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8c6002a070 con 0x7f8c701045d0 2024-03-26T08:33:09.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.956+0000 7f8c66a00700 1 --2- 172.21.15.43:0/342815657 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8c540a2890 0x7f8c540a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:09.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.956+0000 7f8c66a00700 1 -- 172.21.15.43:0/342815657 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f8c600cfc40 con 0x7f8c701045d0 2024-03-26T08:33:09.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.956+0000 7f8c6d400700 1 --2- 172.21.15.43:0/342815657 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8c540a2890 0x7f8c540a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:09.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.957+0000 7f8c6d400700 1 --2- 172.21.15.43:0/342815657 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8c540a2890 0x7f8c540a4d50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8c7010ad10 tx=0x7f8c5c00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:09.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:09.961+0000 7f8c66a00700 1 -- 172.21.15.43:0/342815657 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8c6005e740 con 0x7f8c701045d0 2024-03-26T08:33:10.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:09 smithi183 ceph-mon[35818]: pgmap v430: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:10.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:09 smithi052 ceph-mon[35071]: pgmap v430: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:10.193+0000 7f8c74a00700 1 -- 172.21.15.43:0/342815657 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f8c7019be40 con 0x7f8c701045d0 2024-03-26T08:33:10.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:10.194+0000 7f8c66a00700 1 -- 172.21.15.43:0/342815657 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8c60013070 con 0x7f8c701045d0 2024-03-26T08:33:10.187 INFO:teuthology.orchestra.run.smithi043.stdout:257698037820 2024-03-26T08:33:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:10.197+0000 7f8c74a00700 1 -- 172.21.15.43:0/342815657 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8c540a2890 msgr2=0x7f8c540a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:10.197+0000 7f8c74a00700 1 --2- 172.21.15.43:0/342815657 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8c540a2890 0x7f8c540a4d50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8c7010ad10 tx=0x7f8c5c00b3c0 comp rx=0 tx=0).stop 2024-03-26T08:33:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:10.197+0000 7f8c74a00700 1 -- 172.21.15.43:0/342815657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c701045d0 msgr2=0x7f8c701a0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:10.197+0000 7f8c74a00700 1 --2- 172.21.15.43:0/342815657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c701045d0 0x7f8c701a0750 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8c600059d0 tx=0x7f8c600057c0 comp rx=0 tx=0).stop 2024-03-26T08:33:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:10.197+0000 7f8c74a00700 1 -- 172.21.15.43:0/342815657 shutdown_connections 2024-03-26T08:33:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:10.197+0000 7f8c74a00700 1 --2- 172.21.15.43:0/342815657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c701096b0 0x7f8c7019a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:10.197+0000 7f8c74a00700 1 --2- 172.21.15.43:0/342815657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c701045d0 0x7f8c701a0750 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:10.197+0000 7f8c74a00700 1 --2- 172.21.15.43:0/342815657 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8c540a2890 0x7f8c540a4d50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:10.197+0000 7f8c74a00700 1 --2- 172.21.15.43:0/342815657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c70104f80 0x7f8c701a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:10.198+0000 7f8c74a00700 1 -- 172.21.15.43:0/342815657 >> 172.21.15.43:0/342815657 conn(0x7f8c700ffea0 msgr2=0x7f8c701011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:10.198+0000 7f8c74a00700 1 -- 172.21.15.43:0/342815657 shutdown_connections 2024-03-26T08:33:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:10.198+0000 7f8c74a00700 1 -- 172.21.15.43:0/342815657 wait complete. 2024-03-26T08:33:10.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:10 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:33:10.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:10 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:33:10.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:10 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:33:10.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:10 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/342815657' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-26T08:33:11.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:10 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:33:11.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:10 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:33:11.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:10 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:33:11.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:10 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/342815657' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-26T08:33:11.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:10 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:33:11.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:10 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:33:11.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:10 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:33:11.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:10 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/342815657' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-26T08:33:11.511 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037815 got 257698037820 for osd.8 2024-03-26T08:33:11.511 DEBUG:teuthology.parallel:result is None 2024-03-26T08:33:12.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:11 smithi043 ceph-mon[31929]: pgmap v431: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T08:33:12.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:11 smithi183 ceph-mon[35818]: pgmap v431: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T08:33:12.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:11 smithi052 ceph-mon[35071]: pgmap v431: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T08:33:12.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.113+0000 7f2c8ea00700 1 -- 172.21.15.43:0/2740328721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c881073b0 msgr2=0x7f2c8810ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:12.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.113+0000 7f2c8ea00700 1 --2- 172.21.15.43:0/2740328721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c881073b0 0x7f2c8810ef80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000b0d0 tx=0x7f2c8000b3e0 comp rx=0 tx=0).stop 2024-03-26T08:33:12.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.113+0000 7f2c8ea00700 1 -- 172.21.15.43:0/2740328721 shutdown_connections 2024-03-26T08:33:12.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.113+0000 7f2c8ea00700 1 --2- 172.21.15.43:0/2740328721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c881073b0 0x7f2c8810ef80 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:12.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.113+0000 7f2c8ea00700 1 --2- 172.21.15.43:0/2740328721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c881022d0 0x7f2c881026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:12.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.113+0000 7f2c8ea00700 1 --2- 172.21.15.43:0/2740328721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c88102c80 0x7f2c88106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:12.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.113+0000 7f2c8ea00700 1 -- 172.21.15.43:0/2740328721 >> 172.21.15.43:0/2740328721 conn(0x7f2c880fdb60 msgr2=0x7f2c880fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:12.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.113+0000 7f2c8ea00700 1 -- 172.21.15.43:0/2740328721 shutdown_connections 2024-03-26T08:33:12.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.113+0000 7f2c8ea00700 1 -- 172.21.15.43:0/2740328721 wait complete. 2024-03-26T08:33:12.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.114+0000 7f2c8ea00700 1 Processor -- start 2024-03-26T08:33:12.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.115+0000 7f2c8ea00700 1 -- start start 2024-03-26T08:33:12.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.115+0000 7f2c8ea00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c881022d0 0x7f2c881a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:12.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.115+0000 7f2c8ea00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c88102c80 0x7f2c881a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:12.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.115+0000 7f2c8ea00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c881073b0 0x7f2c8819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:12.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.115+0000 7f2c8ea00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c88112310 con 0x7f2c881073b0 2024-03-26T08:33:12.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.115+0000 7f2c8ea00700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c88112190 con 0x7f2c881022d0 2024-03-26T08:33:12.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.115+0000 7f2c8ea00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c88112490 con 0x7f2c88102c80 2024-03-26T08:33:12.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.116+0000 7f2c8e000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c881073b0 0x7f2c8819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:12.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.116+0000 7f2c8cc00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c88102c80 0x7f2c881a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:12.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.116+0000 7f2c8d600700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c881022d0 0x7f2c881a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:12.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.116+0000 7f2c8cc00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c88102c80 0x7f2c881a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.43:58244/0 (socket says 172.21.15.43:58244) 2024-03-26T08:33:12.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.116+0000 7f2c8e000700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c881073b0 0x7f2c8819a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35894/0 (socket says 172.21.15.43:35894) 2024-03-26T08:33:12.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.116+0000 7f2c8cc00700 1 -- 172.21.15.43:0/2203866117 learned_addr learned my addr 172.21.15.43:0/2203866117 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:12.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.116+0000 7f2c8e000700 1 -- 172.21.15.43:0/2203866117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c88102c80 msgr2=0x7f2c881a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:12.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.116+0000 7f2c8e000700 1 --2- 172.21.15.43:0/2203866117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c88102c80 0x7f2c881a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:12.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.116+0000 7f2c8e000700 1 -- 172.21.15.43:0/2203866117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c881022d0 msgr2=0x7f2c881a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:12.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.116+0000 7f2c8e000700 1 --2- 172.21.15.43:0/2203866117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c881022d0 0x7f2c881a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:12.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.116+0000 7f2c8e000700 1 -- 172.21.15.43:0/2203866117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8819b010 con 0x7f2c881073b0 2024-03-26T08:33:12.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.117+0000 7f2c8e000700 1 --2- 172.21.15.43:0/2203866117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c881073b0 0x7f2c8819a830 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000b820 tx=0x7f2c800079a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:12.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.117+0000 7f2c86000700 1 -- 172.21.15.43:0/2203866117 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c80021020 con 0x7f2c881073b0 2024-03-26T08:33:12.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.117+0000 7f2c86000700 1 -- 172.21.15.43:0/2203866117 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2c8001d470 con 0x7f2c881073b0 2024-03-26T08:33:12.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.117+0000 7f2c8ea00700 1 -- 172.21.15.43:0/2203866117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c8819b2a0 con 0x7f2c881073b0 2024-03-26T08:33:12.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.117+0000 7f2c86000700 1 -- 172.21.15.43:0/2203866117 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c800052b0 con 0x7f2c881073b0 2024-03-26T08:33:12.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.117+0000 7f2c8ea00700 1 -- 172.21.15.43:0/2203866117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c881a74c0 con 0x7f2c881073b0 2024-03-26T08:33:12.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.118+0000 7f2c8ea00700 1 -- 172.21.15.43:0/2203866117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c881037b0 con 0x7f2c881073b0 2024-03-26T08:33:12.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.120+0000 7f2c86000700 1 -- 172.21.15.43:0/2203866117 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f2c80024020 con 0x7f2c881073b0 2024-03-26T08:33:12.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.122+0000 7f2c86000700 1 --2- 172.21.15.43:0/2203866117 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2c700a2910 0x7f2c700a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:12.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.122+0000 7f2c86000700 1 -- 172.21.15.43:0/2203866117 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f2c8000f020 con 0x7f2c881073b0 2024-03-26T08:33:12.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.122+0000 7f2c8d600700 1 --2- 172.21.15.43:0/2203866117 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2c700a2910 0x7f2c700a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:12.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.123+0000 7f2c8d600700 1 --2- 172.21.15.43:0/2203866117 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2c700a2910 0x7f2c700a4dd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c009a10 tx=0x7f2c7c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:12.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.126+0000 7f2c86000700 1 -- 172.21.15.43:0/2203866117 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2c8009d2c0 con 0x7f2c881073b0 2024-03-26T08:33:12.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.353+0000 7f2c8ea00700 1 -- 172.21.15.43:0/2203866117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f2c8819bd10 con 0x7f2c881073b0 2024-03-26T08:33:12.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.354+0000 7f2c86000700 1 -- 172.21.15.43:0/2203866117 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2c8009cbc0 con 0x7f2c881073b0 2024-03-26T08:33:12.347 INFO:teuthology.orchestra.run.smithi043.stdout:167503724625 2024-03-26T08:33:12.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.357+0000 7f2c8ea00700 1 -- 172.21.15.43:0/2203866117 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2c700a2910 msgr2=0x7f2c700a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:12.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.357+0000 7f2c8ea00700 1 --2- 172.21.15.43:0/2203866117 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2c700a2910 0x7f2c700a4dd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c009a10 tx=0x7f2c7c009190 comp rx=0 tx=0).stop 2024-03-26T08:33:12.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.357+0000 7f2c8ea00700 1 -- 172.21.15.43:0/2203866117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c881073b0 msgr2=0x7f2c8819a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:12.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.357+0000 7f2c8ea00700 1 --2- 172.21.15.43:0/2203866117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c881073b0 0x7f2c8819a830 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000b820 tx=0x7f2c800079a0 comp rx=0 tx=0).stop 2024-03-26T08:33:12.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.358+0000 7f2c8ea00700 1 -- 172.21.15.43:0/2203866117 shutdown_connections 2024-03-26T08:33:12.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.358+0000 7f2c8ea00700 1 --2- 172.21.15.43:0/2203866117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c881073b0 0x7f2c8819a830 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:12.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.358+0000 7f2c8ea00700 1 --2- 172.21.15.43:0/2203866117 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f2c700a2910 0x7f2c700a4dd0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:12.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.358+0000 7f2c8ea00700 1 --2- 172.21.15.43:0/2203866117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2c881022d0 0x7f2c881a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:12.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.358+0000 7f2c8ea00700 1 --2- 172.21.15.43:0/2203866117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c88102c80 0x7f2c881a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:12.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.358+0000 7f2c8ea00700 1 -- 172.21.15.43:0/2203866117 >> 172.21.15.43:0/2203866117 conn(0x7f2c880fdb60 msgr2=0x7f2c8810d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:12.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.358+0000 7f2c8ea00700 1 -- 172.21.15.43:0/2203866117 shutdown_connections 2024-03-26T08:33:12.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:12.358+0000 7f2c8ea00700 1 -- 172.21.15.43:0/2203866117 wait complete. 2024-03-26T08:33:13.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:12 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2203866117' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-26T08:33:13.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:12 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2203866117' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-26T08:33:13.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:12 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2203866117' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-26T08:33:13.212 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724619 got 167503724625 for osd.5 2024-03-26T08:33:13.212 DEBUG:teuthology.parallel:result is None 2024-03-26T08:33:13.212 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-26T08:33:13.212 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph pg dump --format=json 2024-03-26T08:33:13.682 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:33:14.067 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:13 smithi043 ceph-mon[31929]: pgmap v432: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-03-26T08:33:14.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:13 smithi052 ceph-mon[35071]: pgmap v432: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-03-26T08:33:14.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:13 smithi183 ceph-mon[35818]: pgmap v432: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-03-26T08:33:15.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:15 smithi043 ceph-mon[31929]: pgmap v433: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-03-26T08:33:15.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.989+0000 7efd3e200700 1 -- 172.21.15.43:0/3109428989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd38107500 msgr2=0x7efd3810f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:15.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.989+0000 7efd3e200700 1 --2- 172.21.15.43:0/3109428989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd38107500 0x7efd3810f080 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efd3000b0d0 tx=0x7efd3000b3e0 comp rx=0 tx=0).stop 2024-03-26T08:33:15.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.989+0000 7efd3e200700 1 -- 172.21.15.43:0/3109428989 shutdown_connections 2024-03-26T08:33:15.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.989+0000 7efd3e200700 1 --2- 172.21.15.43:0/3109428989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd38107500 0x7efd3810f080 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:15.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.989+0000 7efd3e200700 1 --2- 172.21.15.43:0/3109428989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd3806aa90 0x7efd3806ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:15.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.989+0000 7efd3e200700 1 --2- 172.21.15.43:0/3109428989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd3806b440 0x7efd38106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:15.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.989+0000 7efd3e200700 1 -- 172.21.15.43:0/3109428989 >> 172.21.15.43:0/3109428989 conn(0x7efd38078000 msgr2=0x7efd38078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:15.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.989+0000 7efd3e200700 1 -- 172.21.15.43:0/3109428989 shutdown_connections 2024-03-26T08:33:15.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.989+0000 7efd3e200700 1 -- 172.21.15.43:0/3109428989 wait complete. 2024-03-26T08:33:15.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.990+0000 7efd3e200700 1 Processor -- start 2024-03-26T08:33:15.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.990+0000 7efd3e200700 1 -- start start 2024-03-26T08:33:15.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.991+0000 7efd3e200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd3806aa90 0x7efd3819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:15.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.991+0000 7efd3e200700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd3806b440 0x7efd3819eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:15.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.991+0000 7efd3e200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd38107500 0x7efd38198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:15.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.991+0000 7efd3e200700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd381123c0 con 0x7efd3806aa90 2024-03-26T08:33:15.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.991+0000 7efd3e200700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd38112240 con 0x7efd3806b440 2024-03-26T08:33:15.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.991+0000 7efd3e200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd38112540 con 0x7efd38107500 2024-03-26T08:33:15.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.991+0000 7efd37400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd3806aa90 0x7efd3819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:15.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.991+0000 7efd37400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd3806aa90 0x7efd3819e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44526/0 (socket says 172.21.15.43:44526) 2024-03-26T08:33:15.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.991+0000 7efd37400700 1 -- 172.21.15.43:0/2166075627 learned_addr learned my addr 172.21.15.43:0/2166075627 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:15.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.991+0000 7efd2ea00700 1 --2- 172.21.15.43:0/2166075627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd3806b440 0x7efd3819eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:15.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.991+0000 7efd37e00700 1 --2- 172.21.15.43:0/2166075627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd38107500 0x7efd38198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:15.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.992+0000 7efd37400700 1 -- 172.21.15.43:0/2166075627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd38107500 msgr2=0x7efd38198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:15.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.992+0000 7efd37400700 1 --2- 172.21.15.43:0/2166075627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd38107500 0x7efd38198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:15.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.992+0000 7efd37400700 1 -- 172.21.15.43:0/2166075627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd3806b440 msgr2=0x7efd3819eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:15.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.992+0000 7efd37400700 1 --2- 172.21.15.43:0/2166075627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd3806b440 0x7efd3819eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:15.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.992+0000 7efd37400700 1 -- 172.21.15.43:0/2166075627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd38198f60 con 0x7efd3806aa90 2024-03-26T08:33:15.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.992+0000 7efd37400700 1 --2- 172.21.15.43:0/2166075627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd3806aa90 0x7efd3819e5b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7efd2800e9a0 tx=0x7efd2800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:15.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.992+0000 7efd34c00700 1 -- 172.21.15.43:0/2166075627 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2800cbd0 con 0x7efd3806aa90 2024-03-26T08:33:15.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.992+0000 7efd34c00700 1 -- 172.21.15.43:0/2166075627 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efd2800cd70 con 0x7efd3806aa90 2024-03-26T08:33:15.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.992+0000 7efd3e200700 1 -- 172.21.15.43:0/2166075627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd38199250 con 0x7efd3806aa90 2024-03-26T08:33:15.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.992+0000 7efd34c00700 1 -- 172.21.15.43:0/2166075627 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd28018750 con 0x7efd3806aa90 2024-03-26T08:33:15.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.993+0000 7efd3e200700 1 -- 172.21.15.43:0/2166075627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd381a5530 con 0x7efd3806aa90 2024-03-26T08:33:15.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.994+0000 7efd3e200700 1 -- 172.21.15.43:0/2166075627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd3806a0e0 con 0x7efd3806aa90 2024-03-26T08:33:15.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.996+0000 7efd34c00700 1 -- 172.21.15.43:0/2166075627 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7efd28010430 con 0x7efd3806aa90 2024-03-26T08:33:15.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.997+0000 7efd34c00700 1 --2- 172.21.15.43:0/2166075627 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7efd1c0a2890 0x7efd1c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:15.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.997+0000 7efd34c00700 1 -- 172.21.15.43:0/2166075627 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7efd280cf150 con 0x7efd3806aa90 2024-03-26T08:33:15.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.997+0000 7efd2ea00700 1 --2- 172.21.15.43:0/2166075627 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7efd1c0a2890 0x7efd1c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:15.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:15.998+0000 7efd2ea00700 1 --2- 172.21.15.43:0/2166075627 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7efd1c0a2890 0x7efd1c0a4d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efd38199e40 tx=0x7efd2000b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:15.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:16.002+0000 7efd34c00700 1 -- 172.21.15.43:0/2166075627 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efd2809be10 con 0x7efd3806aa90 2024-03-26T08:33:16.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:15 smithi052 ceph-mon[35071]: pgmap v433: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-03-26T08:33:16.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:16.220+0000 7efd3e200700 1 -- 172.21.15.43:0/2166075627 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efd38199bb0 con 0x7efd1c0a2890 2024-03-26T08:33:16.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:16.234+0000 7efd34c00700 1 -- 172.21.15.43:0/2166075627 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340815 (secure 0 0 0) 0x7efd38199bb0 con 0x7efd1c0a2890 2024-03-26T08:33:16.236 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:33:16.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:16.243+0000 7efd3e200700 1 -- 172.21.15.43:0/2166075627 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7efd1c0a2890 msgr2=0x7efd1c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:16.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:16.243+0000 7efd3e200700 1 --2- 172.21.15.43:0/2166075627 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7efd1c0a2890 0x7efd1c0a4d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efd38199e40 tx=0x7efd2000b590 comp rx=0 tx=0).stop 2024-03-26T08:33:16.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:16.243+0000 7efd3e200700 1 -- 172.21.15.43:0/2166075627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd3806aa90 msgr2=0x7efd3819e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:16.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:16.243+0000 7efd3e200700 1 --2- 172.21.15.43:0/2166075627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd3806aa90 0x7efd3819e5b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7efd2800e9a0 tx=0x7efd2800ed60 comp rx=0 tx=0).stop 2024-03-26T08:33:16.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:16.243+0000 7efd3e200700 1 -- 172.21.15.43:0/2166075627 shutdown_connections 2024-03-26T08:33:16.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:16.243+0000 7efd3e200700 1 --2- 172.21.15.43:0/2166075627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd38107500 0x7efd38198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:16.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:16.243+0000 7efd3e200700 1 --2- 172.21.15.43:0/2166075627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd3806aa90 0x7efd3819e5b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:16.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:16.243+0000 7efd3e200700 1 --2- 172.21.15.43:0/2166075627 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7efd1c0a2890 0x7efd1c0a4d50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:16.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:16.243+0000 7efd3e200700 1 --2- 172.21.15.43:0/2166075627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd3806b440 0x7efd3819eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:16.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:16.243+0000 7efd3e200700 1 -- 172.21.15.43:0/2166075627 >> 172.21.15.43:0/2166075627 conn(0x7efd38078000 msgr2=0x7efd380ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:16.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:16.244+0000 7efd3e200700 1 -- 172.21.15.43:0/2166075627 shutdown_connections 2024-03-26T08:33:16.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:16.244+0000 7efd3e200700 1 -- 172.21.15.43:0/2166075627 wait complete. 2024-03-26T08:33:16.239 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-03-26T08:33:16.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:15 smithi183 ceph-mon[35818]: pgmap v433: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-03-26T08:33:17.064 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":433,"stamp":"2024-03-26T08:33:14.378130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653721,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":306,"num_read_kb":264,"num_write":269,"num_write_kb":3032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":711,"ondisk_log_size":711,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369864,"kb_used_data":12392,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571960,"statfs":{"total":1151940427776,"available":1151561687040,"internally_reserved":0,"allocated":12689408,"data_stored":8207609,"data_compressed":433440,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":11,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005407"},"pg_stats":[{"pgid":"3.3f","version":"94'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.346209+0000","last_change":"2024-03-26T08:31:22.394256+0000","last_active":"2024-03-26T08:33:06.346209+0000","last_peered":"2024-03-26T08:33:06.346209+0000","last_clean":"2024-03-26T08:33:06.346209+0000","last_became_active":"2024-03-26T08:31:22.393947+0000","last_became_peered":"2024-03-26T08:31:22.393947+0000","last_unstale":"2024-03-26T08:33:06.346209+0000","last_undegraded":"2024-03-26T08:33:06.346209+0000","last_fullsized":"2024-03-26T08:33:06.346209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:00:50.769853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"94'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.335167+0000","last_change":"2024-03-26T08:31:22.387549+0000","last_active":"2024-03-26T08:33:06.335167+0000","last_peered":"2024-03-26T08:33:06.335167+0000","last_clean":"2024-03-26T08:33:06.335167+0000","last_became_active":"2024-03-26T08:31:22.387127+0000","last_became_peered":"2024-03-26T08:31:22.387127+0000","last_unstale":"2024-03-26T08:33:06.335167+0000","last_undegraded":"2024-03-26T08:33:06.335167+0000","last_fullsized":"2024-03-26T08:33:06.335167+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:52:36.156193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.343998+0000","last_change":"2024-03-26T08:31:22.404275+0000","last_active":"2024-03-26T08:33:06.343998+0000","last_peered":"2024-03-26T08:33:06.343998+0000","last_clean":"2024-03-26T08:33:06.343998+0000","last_became_active":"2024-03-26T08:31:22.403909+0000","last_became_peered":"2024-03-26T08:31:22.403909+0000","last_unstale":"2024-03-26T08:33:06.343998+0000","last_undegraded":"2024-03-26T08:33:06.343998+0000","last_fullsized":"2024-03-26T08:33:06.343998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:55:56.295719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"94'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.331852+0000","last_change":"2024-03-26T08:31:22.381310+0000","last_active":"2024-03-26T08:33:06.331852+0000","last_peered":"2024-03-26T08:33:06.331852+0000","last_clean":"2024-03-26T08:33:06.331852+0000","last_became_active":"2024-03-26T08:31:22.380721+0000","last_became_peered":"2024-03-26T08:31:22.380721+0000","last_unstale":"2024-03-26T08:33:06.331852+0000","last_undegraded":"2024-03-26T08:33:06.331852+0000","last_fullsized":"2024-03-26T08:33:06.331852+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:42:30.325409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"94'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.328035+0000","last_change":"2024-03-26T08:31:22.396906+0000","last_active":"2024-03-26T08:33:06.328035+0000","last_peered":"2024-03-26T08:33:06.328035+0000","last_clean":"2024-03-26T08:33:06.328035+0000","last_became_active":"2024-03-26T08:31:22.392815+0000","last_became_peered":"2024-03-26T08:31:22.392815+0000","last_unstale":"2024-03-26T08:33:06.328035+0000","last_undegraded":"2024-03-26T08:33:06.328035+0000","last_fullsized":"2024-03-26T08:33:06.328035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:04:25.761459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"94'6","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.346325+0000","last_change":"2024-03-26T08:31:22.389374+0000","last_active":"2024-03-26T08:33:06.346325+0000","last_peered":"2024-03-26T08:33:06.346325+0000","last_clean":"2024-03-26T08:33:06.346325+0000","last_became_active":"2024-03-26T08:31:22.389012+0000","last_became_peered":"2024-03-26T08:31:22.389012+0000","last_unstale":"2024-03-26T08:33:06.346325+0000","last_undegraded":"2024-03-26T08:33:06.346325+0000","last_fullsized":"2024-03-26T08:33:06.346325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:23:35.696335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"94'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.342833+0000","last_change":"2024-03-26T08:31:22.378805+0000","last_active":"2024-03-26T08:33:06.342833+0000","last_peered":"2024-03-26T08:33:06.342833+0000","last_clean":"2024-03-26T08:33:06.342833+0000","last_became_active":"2024-03-26T08:31:22.378313+0000","last_became_peered":"2024-03-26T08:31:22.378313+0000","last_unstale":"2024-03-26T08:33:06.342833+0000","last_undegraded":"2024-03-26T08:33:06.342833+0000","last_fullsized":"2024-03-26T08:33:06.342833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:10:03.816988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"94'9","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.350604+0000","last_change":"2024-03-26T08:31:37.495699+0000","last_active":"2024-03-26T08:33:06.350604+0000","last_peered":"2024-03-26T08:33:06.350604+0000","last_clean":"2024-03-26T08:33:06.350604+0000","last_became_active":"2024-03-26T08:31:37.494458+0000","last_became_peered":"2024-03-26T08:31:37.494458+0000","last_unstale":"2024-03-26T08:33:06.350604+0000","last_undegraded":"2024-03-26T08:33:06.350604+0000","last_fullsized":"2024-03-26T08:33:06.350604+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:32:36.973898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,10],"acting":[9,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"94'11","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.344545+0000","last_change":"2024-03-26T08:32:38.574794+0000","last_active":"2024-03-26T08:33:06.344545+0000","last_peered":"2024-03-26T08:33:06.344545+0000","last_clean":"2024-03-26T08:33:06.344545+0000","last_became_active":"2024-03-26T08:32:38.574302+0000","last_became_peered":"2024-03-26T08:32:38.574302+0000","last_unstale":"2024-03-26T08:33:06.344545+0000","last_undegraded":"2024-03-26T08:33:06.344545+0000","last_fullsized":"2024-03-26T08:33:06.344545+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:01:01.176434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"94'16","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.346631+0000","last_change":"2024-03-26T08:32:38.313633+0000","last_active":"2024-03-26T08:33:06.346631+0000","last_peered":"2024-03-26T08:33:06.346631+0000","last_clean":"2024-03-26T08:33:06.346631+0000","last_became_active":"2024-03-26T08:32:38.312862+0000","last_became_peered":"2024-03-26T08:32:38.312862+0000","last_unstale":"2024-03-26T08:33:06.346631+0000","last_undegraded":"2024-03-26T08:33:06.346631+0000","last_fullsized":"2024-03-26T08:33:06.346631+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:54:24.226199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.349605+0000","last_change":"2024-03-26T08:31:22.388214+0000","last_active":"2024-03-26T08:33:06.349605+0000","last_peered":"2024-03-26T08:33:06.349605+0000","last_clean":"2024-03-26T08:33:06.349605+0000","last_became_active":"2024-03-26T08:31:22.387760+0000","last_became_peered":"2024-03-26T08:31:22.387760+0000","last_unstale":"2024-03-26T08:33:06.349605+0000","last_undegraded":"2024-03-26T08:33:06.349605+0000","last_fullsized":"2024-03-26T08:33:06.349605+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:25:22.498259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"94'7","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.335720+0000","last_change":"2024-03-26T08:31:22.390892+0000","last_active":"2024-03-26T08:33:06.335720+0000","last_peered":"2024-03-26T08:33:06.335720+0000","last_clean":"2024-03-26T08:33:06.335720+0000","last_became_active":"2024-03-26T08:31:22.390500+0000","last_became_peered":"2024-03-26T08:31:22.390500+0000","last_unstale":"2024-03-26T08:33:06.335720+0000","last_undegraded":"2024-03-26T08:33:06.335720+0000","last_fullsized":"2024-03-26T08:33:06.335720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:29:01.832399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"94'10","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.338385+0000","last_change":"2024-03-26T08:31:22.382442+0000","last_active":"2024-03-26T08:33:06.338385+0000","last_peered":"2024-03-26T08:33:06.338385+0000","last_clean":"2024-03-26T08:33:06.338385+0000","last_became_active":"2024-03-26T08:31:22.380346+0000","last_became_peered":"2024-03-26T08:31:22.380346+0000","last_unstale":"2024-03-26T08:33:06.338385+0000","last_undegraded":"2024-03-26T08:33:06.338385+0000","last_fullsized":"2024-03-26T08:33:06.338385+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:36:36.451504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.334236+0000","last_change":"2024-03-26T08:31:22.387401+0000","last_active":"2024-03-26T08:33:06.334236+0000","last_peered":"2024-03-26T08:33:06.334236+0000","last_clean":"2024-03-26T08:33:06.334236+0000","last_became_active":"2024-03-26T08:31:22.387020+0000","last_became_peered":"2024-03-26T08:31:22.387020+0000","last_unstale":"2024-03-26T08:33:06.334236+0000","last_undegraded":"2024-03-26T08:33:06.334236+0000","last_fullsized":"2024-03-26T08:33:06.334236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:50:15.073663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.342429+0000","last_change":"2024-03-26T08:31:22.397971+0000","last_active":"2024-03-26T08:33:06.342429+0000","last_peered":"2024-03-26T08:33:06.342429+0000","last_clean":"2024-03-26T08:33:06.342429+0000","last_became_active":"2024-03-26T08:31:22.395371+0000","last_became_peered":"2024-03-26T08:31:22.395371+0000","last_unstale":"2024-03-26T08:33:06.342429+0000","last_undegraded":"2024-03-26T08:33:06.342429+0000","last_fullsized":"2024-03-26T08:33:06.342429+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:59:51.053712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"94'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.348921+0000","last_change":"2024-03-26T08:31:22.394437+0000","last_active":"2024-03-26T08:33:06.348921+0000","last_peered":"2024-03-26T08:33:06.348921+0000","last_clean":"2024-03-26T08:33:06.348921+0000","last_became_active":"2024-03-26T08:31:22.386794+0000","last_became_peered":"2024-03-26T08:31:22.386794+0000","last_unstale":"2024-03-26T08:33:06.348921+0000","last_undegraded":"2024-03-26T08:33:06.348921+0000","last_fullsized":"2024-03-26T08:33:06.348921+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:56:01.686361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"94'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.346277+0000","last_change":"2024-03-26T08:31:22.397887+0000","last_active":"2024-03-26T08:33:06.346277+0000","last_peered":"2024-03-26T08:33:06.346277+0000","last_clean":"2024-03-26T08:33:06.346277+0000","last_became_active":"2024-03-26T08:31:22.397086+0000","last_became_peered":"2024-03-26T08:31:22.397086+0000","last_unstale":"2024-03-26T08:33:06.346277+0000","last_undegraded":"2024-03-26T08:33:06.346277+0000","last_fullsized":"2024-03-26T08:33:06.346277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:41:47.512904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"94'10","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.344271+0000","last_change":"2024-03-26T08:31:22.398868+0000","last_active":"2024-03-26T08:33:06.344271+0000","last_peered":"2024-03-26T08:33:06.344271+0000","last_clean":"2024-03-26T08:33:06.344271+0000","last_became_active":"2024-03-26T08:31:22.398536+0000","last_became_peered":"2024-03-26T08:31:22.398536+0000","last_unstale":"2024-03-26T08:33:06.344271+0000","last_undegraded":"2024-03-26T08:33:06.344271+0000","last_fullsized":"2024-03-26T08:33:06.344271+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:59:07.931120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"94'7","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.328913+0000","last_change":"2024-03-26T08:31:22.389805+0000","last_active":"2024-03-26T08:33:06.328913+0000","last_peered":"2024-03-26T08:33:06.328913+0000","last_clean":"2024-03-26T08:33:06.328913+0000","last_became_active":"2024-03-26T08:31:22.389410+0000","last_became_peered":"2024-03-26T08:31:22.389410+0000","last_unstale":"2024-03-26T08:33:06.328913+0000","last_undegraded":"2024-03-26T08:33:06.328913+0000","last_fullsized":"2024-03-26T08:33:06.328913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:36:29.934029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.310334+0000","last_change":"2024-03-26T08:31:26.437909+0000","last_active":"2024-03-26T08:32:38.310334+0000","last_peered":"2024-03-26T08:32:38.310334+0000","last_clean":"2024-03-26T08:32:38.310334+0000","last_became_active":"2024-03-26T08:31:26.437337+0000","last_became_peered":"2024-03-26T08:31:26.437337+0000","last_unstale":"2024-03-26T08:32:38.310334+0000","last_undegraded":"2024-03-26T08:32:38.310334+0000","last_fullsized":"2024-03-26T08:32:38.310334+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:01:15.856194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.273628+0000","last_change":"2024-03-26T08:31:21.367133+0000","last_active":"2024-03-26T08:32:38.273628+0000","last_peered":"2024-03-26T08:32:38.273628+0000","last_clean":"2024-03-26T08:32:38.273628+0000","last_became_active":"2024-03-26T08:31:21.366206+0000","last_became_peered":"2024-03-26T08:31:21.366206+0000","last_unstale":"2024-03-26T08:32:38.273628+0000","last_undegraded":"2024-03-26T08:32:38.273628+0000","last_fullsized":"2024-03-26T08:32:38.273628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:11:16.614464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"94'5","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.329519+0000","last_change":"2024-03-26T08:31:22.385984+0000","last_active":"2024-03-26T08:33:06.329519+0000","last_peered":"2024-03-26T08:33:06.329519+0000","last_clean":"2024-03-26T08:33:06.329519+0000","last_became_active":"2024-03-26T08:31:22.385512+0000","last_became_peered":"2024-03-26T08:31:22.385512+0000","last_unstale":"2024-03-26T08:33:06.329519+0000","last_undegraded":"2024-03-26T08:33:06.329519+0000","last_fullsized":"2024-03-26T08:33:06.329519+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:46:24.087975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.267156+0000","last_change":"2024-03-26T08:31:26.454001+0000","last_active":"2024-03-26T08:32:38.267156+0000","last_peered":"2024-03-26T08:32:38.267156+0000","last_clean":"2024-03-26T08:32:38.267156+0000","last_became_active":"2024-03-26T08:31:26.453474+0000","last_became_peered":"2024-03-26T08:31:26.453474+0000","last_unstale":"2024-03-26T08:32:38.267156+0000","last_undegraded":"2024-03-26T08:32:38.267156+0000","last_fullsized":"2024-03-26T08:32:38.267156+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:09:42.197500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.267437+0000","last_change":"2024-03-26T08:31:21.365012+0000","last_active":"2024-03-26T08:32:38.267437+0000","last_peered":"2024-03-26T08:32:38.267437+0000","last_clean":"2024-03-26T08:32:38.267437+0000","last_became_active":"2024-03-26T08:31:21.363981+0000","last_became_peered":"2024-03-26T08:31:21.363981+0000","last_unstale":"2024-03-26T08:32:38.267437+0000","last_undegraded":"2024-03-26T08:32:38.267437+0000","last_fullsized":"2024-03-26T08:32:38.267437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:12:36.581648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.346350+0000","last_change":"2024-03-26T08:31:22.387960+0000","last_active":"2024-03-26T08:33:06.346350+0000","last_peered":"2024-03-26T08:33:06.346350+0000","last_clean":"2024-03-26T08:33:06.346350+0000","last_became_active":"2024-03-26T08:31:22.387627+0000","last_became_peered":"2024-03-26T08:31:22.387627+0000","last_unstale":"2024-03-26T08:33:06.346350+0000","last_undegraded":"2024-03-26T08:33:06.346350+0000","last_fullsized":"2024-03-26T08:33:06.346350+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:20:09.265075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.269421+0000","last_change":"2024-03-26T08:31:26.450387+0000","last_active":"2024-03-26T08:32:38.269421+0000","last_peered":"2024-03-26T08:32:38.269421+0000","last_clean":"2024-03-26T08:32:38.269421+0000","last_became_active":"2024-03-26T08:31:26.449418+0000","last_became_peered":"2024-03-26T08:31:26.449418+0000","last_unstale":"2024-03-26T08:32:38.269421+0000","last_undegraded":"2024-03-26T08:32:38.269421+0000","last_fullsized":"2024-03-26T08:32:38.269421+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:51:59.022851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.268640+0000","last_change":"2024-03-26T08:31:21.373422+0000","last_active":"2024-03-26T08:32:38.268640+0000","last_peered":"2024-03-26T08:32:38.268640+0000","last_clean":"2024-03-26T08:32:38.268640+0000","last_became_active":"2024-03-26T08:31:21.372916+0000","last_became_peered":"2024-03-26T08:31:21.372916+0000","last_unstale":"2024-03-26T08:32:38.268640+0000","last_undegraded":"2024-03-26T08:32:38.268640+0000","last_fullsized":"2024-03-26T08:32:38.268640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:09:08.313365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"94'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.338926+0000","last_change":"2024-03-26T08:31:22.403385+0000","last_active":"2024-03-26T08:33:06.338926+0000","last_peered":"2024-03-26T08:33:06.338926+0000","last_clean":"2024-03-26T08:33:06.338926+0000","last_became_active":"2024-03-26T08:31:22.398021+0000","last_became_peered":"2024-03-26T08:31:22.398021+0000","last_unstale":"2024-03-26T08:33:06.338926+0000","last_undegraded":"2024-03-26T08:33:06.338926+0000","last_fullsized":"2024-03-26T08:33:06.338926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:56:06.691167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.267106+0000","last_change":"2024-03-26T08:31:26.449049+0000","last_active":"2024-03-26T08:32:38.267106+0000","last_peered":"2024-03-26T08:32:38.267106+0000","last_clean":"2024-03-26T08:32:38.267106+0000","last_became_active":"2024-03-26T08:31:26.448510+0000","last_became_peered":"2024-03-26T08:31:26.448510+0000","last_unstale":"2024-03-26T08:32:38.267106+0000","last_undegraded":"2024-03-26T08:32:38.267106+0000","last_fullsized":"2024-03-26T08:32:38.267106+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:24:58.699699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.267778+0000","last_change":"2024-03-26T08:31:21.367289+0000","last_active":"2024-03-26T08:32:38.267778+0000","last_peered":"2024-03-26T08:32:38.267778+0000","last_clean":"2024-03-26T08:32:38.267778+0000","last_became_active":"2024-03-26T08:31:21.366960+0000","last_became_peered":"2024-03-26T08:31:21.366960+0000","last_unstale":"2024-03-26T08:32:38.267778+0000","last_undegraded":"2024-03-26T08:32:38.267778+0000","last_fullsized":"2024-03-26T08:32:38.267778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:22:25.479383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"94'15","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.346079+0000","last_change":"2024-03-26T08:31:22.385815+0000","last_active":"2024-03-26T08:33:06.346079+0000","last_peered":"2024-03-26T08:33:06.346079+0000","last_clean":"2024-03-26T08:33:06.346079+0000","last_became_active":"2024-03-26T08:31:22.385503+0000","last_became_peered":"2024-03-26T08:31:22.385503+0000","last_unstale":"2024-03-26T08:33:06.346079+0000","last_undegraded":"2024-03-26T08:33:06.346079+0000","last_fullsized":"2024-03-26T08:33:06.346079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:33:35.768902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.311320+0000","last_change":"2024-03-26T08:31:26.449656+0000","last_active":"2024-03-26T08:32:38.311320+0000","last_peered":"2024-03-26T08:32:38.311320+0000","last_clean":"2024-03-26T08:32:38.311320+0000","last_became_active":"2024-03-26T08:31:26.448843+0000","last_became_peered":"2024-03-26T08:31:26.448843+0000","last_unstale":"2024-03-26T08:32:38.311320+0000","last_undegraded":"2024-03-26T08:32:38.311320+0000","last_fullsized":"2024-03-26T08:32:38.311320+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:11:28.135551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.456071+0000","last_change":"2024-03-26T08:31:21.373384+0000","last_active":"2024-03-26T08:32:38.456071+0000","last_peered":"2024-03-26T08:32:38.456071+0000","last_clean":"2024-03-26T08:32:38.456071+0000","last_became_active":"2024-03-26T08:31:21.371708+0000","last_became_peered":"2024-03-26T08:31:21.371708+0000","last_unstale":"2024-03-26T08:32:38.456071+0000","last_undegraded":"2024-03-26T08:32:38.456071+0000","last_fullsized":"2024-03-26T08:32:38.456071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:46:09.745738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.342417+0000","last_change":"2024-03-26T08:31:22.383990+0000","last_active":"2024-03-26T08:33:06.342417+0000","last_peered":"2024-03-26T08:33:06.342417+0000","last_clean":"2024-03-26T08:33:06.342417+0000","last_became_active":"2024-03-26T08:31:22.383580+0000","last_became_peered":"2024-03-26T08:31:22.383580+0000","last_unstale":"2024-03-26T08:33:06.342417+0000","last_undegraded":"2024-03-26T08:33:06.342417+0000","last_fullsized":"2024-03-26T08:33:06.342417+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:32:36.037407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.311026+0000","last_change":"2024-03-26T08:31:26.445710+0000","last_active":"2024-03-26T08:32:38.311026+0000","last_peered":"2024-03-26T08:32:38.311026+0000","last_clean":"2024-03-26T08:32:38.311026+0000","last_became_active":"2024-03-26T08:31:26.445189+0000","last_became_peered":"2024-03-26T08:31:26.445189+0000","last_unstale":"2024-03-26T08:32:38.311026+0000","last_undegraded":"2024-03-26T08:32:38.311026+0000","last_fullsized":"2024-03-26T08:32:38.311026+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:31:23.033192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.272979+0000","last_change":"2024-03-26T08:31:21.362175+0000","last_active":"2024-03-26T08:32:38.272979+0000","last_peered":"2024-03-26T08:32:38.272979+0000","last_clean":"2024-03-26T08:32:38.272979+0000","last_became_active":"2024-03-26T08:31:21.361805+0000","last_became_peered":"2024-03-26T08:31:21.361805+0000","last_unstale":"2024-03-26T08:32:38.272979+0000","last_undegraded":"2024-03-26T08:32:38.272979+0000","last_fullsized":"2024-03-26T08:32:38.272979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:38:26.537955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"94'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.338999+0000","last_change":"2024-03-26T08:31:22.380489+0000","last_active":"2024-03-26T08:33:06.338999+0000","last_peered":"2024-03-26T08:33:06.338999+0000","last_clean":"2024-03-26T08:33:06.338999+0000","last_became_active":"2024-03-26T08:31:22.380139+0000","last_became_peered":"2024-03-26T08:31:22.380139+0000","last_unstale":"2024-03-26T08:33:06.338999+0000","last_undegraded":"2024-03-26T08:33:06.338999+0000","last_fullsized":"2024-03-26T08:33:06.338999+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:09:18.550790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.572421+0000","last_change":"2024-03-26T08:31:26.454535+0000","last_active":"2024-03-26T08:32:38.572421+0000","last_peered":"2024-03-26T08:32:38.572421+0000","last_clean":"2024-03-26T08:32:38.572421+0000","last_became_active":"2024-03-26T08:31:26.454106+0000","last_became_peered":"2024-03-26T08:31:26.454106+0000","last_unstale":"2024-03-26T08:32:38.572421+0000","last_undegraded":"2024-03-26T08:32:38.572421+0000","last_fullsized":"2024-03-26T08:32:38.572421+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:57:28.870653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.571042+0000","last_change":"2024-03-26T08:31:21.347236+0000","last_active":"2024-03-26T08:32:38.571042+0000","last_peered":"2024-03-26T08:32:38.571042+0000","last_clean":"2024-03-26T08:32:38.571042+0000","last_became_active":"2024-03-26T08:31:21.346770+0000","last_became_peered":"2024-03-26T08:31:21.346770+0000","last_unstale":"2024-03-26T08:32:38.571042+0000","last_undegraded":"2024-03-26T08:32:38.571042+0000","last_fullsized":"2024-03-26T08:32:38.571042+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:17:22.927636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"94'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.336185+0000","last_change":"2024-03-26T08:31:22.390623+0000","last_active":"2024-03-26T08:33:06.336185+0000","last_peered":"2024-03-26T08:33:06.336185+0000","last_clean":"2024-03-26T08:33:06.336185+0000","last_became_active":"2024-03-26T08:31:22.389512+0000","last_became_peered":"2024-03-26T08:31:22.389512+0000","last_unstale":"2024-03-26T08:33:06.336185+0000","last_undegraded":"2024-03-26T08:33:06.336185+0000","last_fullsized":"2024-03-26T08:33:06.336185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:04:31.317386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.572594+0000","last_change":"2024-03-26T08:31:26.445277+0000","last_active":"2024-03-26T08:32:38.572594+0000","last_peered":"2024-03-26T08:32:38.572594+0000","last_clean":"2024-03-26T08:32:38.572594+0000","last_became_active":"2024-03-26T08:31:26.443558+0000","last_became_peered":"2024-03-26T08:31:26.443558+0000","last_unstale":"2024-03-26T08:32:38.572594+0000","last_undegraded":"2024-03-26T08:32:38.572594+0000","last_fullsized":"2024-03-26T08:32:38.572594+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:20:16.688699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.454859+0000","last_change":"2024-03-26T08:31:21.371523+0000","last_active":"2024-03-26T08:32:38.454859+0000","last_peered":"2024-03-26T08:32:38.454859+0000","last_clean":"2024-03-26T08:32:38.454859+0000","last_became_active":"2024-03-26T08:31:21.371136+0000","last_became_peered":"2024-03-26T08:31:21.371136+0000","last_unstale":"2024-03-26T08:32:38.454859+0000","last_undegraded":"2024-03-26T08:32:38.454859+0000","last_fullsized":"2024-03-26T08:32:38.454859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:06:55.083579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"94'11","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.347390+0000","last_change":"2024-03-26T08:31:22.386237+0000","last_active":"2024-03-26T08:33:06.347390+0000","last_peered":"2024-03-26T08:33:06.347390+0000","last_clean":"2024-03-26T08:33:06.347390+0000","last_became_active":"2024-03-26T08:31:22.385874+0000","last_became_peered":"2024-03-26T08:31:22.385874+0000","last_unstale":"2024-03-26T08:33:06.347390+0000","last_undegraded":"2024-03-26T08:33:06.347390+0000","last_fullsized":"2024-03-26T08:33:06.347390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:14:58.278611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.265815+0000","last_change":"2024-03-26T08:31:26.452464+0000","last_active":"2024-03-26T08:32:38.265815+0000","last_peered":"2024-03-26T08:32:38.265815+0000","last_clean":"2024-03-26T08:32:38.265815+0000","last_became_active":"2024-03-26T08:31:26.451862+0000","last_became_peered":"2024-03-26T08:31:26.451862+0000","last_unstale":"2024-03-26T08:32:38.265815+0000","last_undegraded":"2024-03-26T08:32:38.265815+0000","last_fullsized":"2024-03-26T08:32:38.265815+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:48:29.616247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.274651+0000","last_change":"2024-03-26T08:31:21.365283+0000","last_active":"2024-03-26T08:32:38.274651+0000","last_peered":"2024-03-26T08:32:38.274651+0000","last_clean":"2024-03-26T08:32:38.274651+0000","last_became_active":"2024-03-26T08:31:21.364902+0000","last_became_peered":"2024-03-26T08:31:21.364902+0000","last_unstale":"2024-03-26T08:32:38.274651+0000","last_undegraded":"2024-03-26T08:32:38.274651+0000","last_fullsized":"2024-03-26T08:32:38.274651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:55:39.394355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"94'10","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.344341+0000","last_change":"2024-03-26T08:31:22.391645+0000","last_active":"2024-03-26T08:33:06.344341+0000","last_peered":"2024-03-26T08:33:06.344341+0000","last_clean":"2024-03-26T08:33:06.344341+0000","last_became_active":"2024-03-26T08:31:22.391280+0000","last_became_peered":"2024-03-26T08:31:22.391280+0000","last_unstale":"2024-03-26T08:33:06.344341+0000","last_undegraded":"2024-03-26T08:33:06.344341+0000","last_fullsized":"2024-03-26T08:33:06.344341+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:31:48.963287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.272411+0000","last_change":"2024-03-26T08:31:26.446215+0000","last_active":"2024-03-26T08:32:38.272411+0000","last_peered":"2024-03-26T08:32:38.272411+0000","last_clean":"2024-03-26T08:32:38.272411+0000","last_became_active":"2024-03-26T08:31:26.445765+0000","last_became_peered":"2024-03-26T08:31:26.445765+0000","last_unstale":"2024-03-26T08:32:38.272411+0000","last_undegraded":"2024-03-26T08:32:38.272411+0000","last_fullsized":"2024-03-26T08:32:38.272411+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:57:14.809610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.572547+0000","last_change":"2024-03-26T08:31:21.373345+0000","last_active":"2024-03-26T08:32:38.572547+0000","last_peered":"2024-03-26T08:32:38.572547+0000","last_clean":"2024-03-26T08:32:38.572547+0000","last_became_active":"2024-03-26T08:31:21.372906+0000","last_became_peered":"2024-03-26T08:31:21.372906+0000","last_unstale":"2024-03-26T08:32:38.572547+0000","last_undegraded":"2024-03-26T08:32:38.572547+0000","last_fullsized":"2024-03-26T08:32:38.572547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:20:00.518815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.339863+0000","last_change":"2024-03-26T08:31:22.393433+0000","last_active":"2024-03-26T08:33:06.339863+0000","last_peered":"2024-03-26T08:33:06.339863+0000","last_clean":"2024-03-26T08:33:06.339863+0000","last_became_active":"2024-03-26T08:31:22.393100+0000","last_became_peered":"2024-03-26T08:31:22.393100+0000","last_unstale":"2024-03-26T08:33:06.339863+0000","last_undegraded":"2024-03-26T08:33:06.339863+0000","last_fullsized":"2024-03-26T08:33:06.339863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:33:30.418492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.268506+0000","last_change":"2024-03-26T08:31:26.449975+0000","last_active":"2024-03-26T08:32:38.268506+0000","last_peered":"2024-03-26T08:32:38.268506+0000","last_clean":"2024-03-26T08:32:38.268506+0000","last_became_active":"2024-03-26T08:31:26.448345+0000","last_became_peered":"2024-03-26T08:31:26.448345+0000","last_unstale":"2024-03-26T08:32:38.268506+0000","last_undegraded":"2024-03-26T08:32:38.268506+0000","last_fullsized":"2024-03-26T08:32:38.268506+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:01:24.765939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.457882+0000","last_change":"2024-03-26T08:31:21.364195+0000","last_active":"2024-03-26T08:32:38.457882+0000","last_peered":"2024-03-26T08:32:38.457882+0000","last_clean":"2024-03-26T08:32:38.457882+0000","last_became_active":"2024-03-26T08:31:21.363887+0000","last_became_peered":"2024-03-26T08:31:21.363887+0000","last_unstale":"2024-03-26T08:32:38.457882+0000","last_undegraded":"2024-03-26T08:32:38.457882+0000","last_fullsized":"2024-03-26T08:32:38.457882+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:03:03.015581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"94'11","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.339076+0000","last_change":"2024-03-26T08:31:37.493245+0000","last_active":"2024-03-26T08:33:06.339076+0000","last_peered":"2024-03-26T08:33:06.339076+0000","last_clean":"2024-03-26T08:33:06.339076+0000","last_became_active":"2024-03-26T08:31:37.492509+0000","last_became_peered":"2024-03-26T08:31:37.492509+0000","last_unstale":"2024-03-26T08:33:06.339076+0000","last_undegraded":"2024-03-26T08:33:06.339076+0000","last_fullsized":"2024-03-26T08:33:06.339076+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:28:56.355010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.272450+0000","last_change":"2024-03-26T08:31:26.449405+0000","last_active":"2024-03-26T08:32:38.272450+0000","last_peered":"2024-03-26T08:32:38.272450+0000","last_clean":"2024-03-26T08:32:38.272450+0000","last_became_active":"2024-03-26T08:31:26.448934+0000","last_became_peered":"2024-03-26T08:31:26.448934+0000","last_unstale":"2024-03-26T08:32:38.272450+0000","last_undegraded":"2024-03-26T08:32:38.272450+0000","last_fullsized":"2024-03-26T08:32:38.272450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:12:22.797816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'31","reported_seq":235,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:09.837431+0000","last_change":"2024-03-26T08:31:21.372300+0000","last_active":"2024-03-26T08:33:09.837431+0000","last_peered":"2024-03-26T08:33:09.837431+0000","last_clean":"2024-03-26T08:33:09.837431+0000","last_became_active":"2024-03-26T08:31:21.370103+0000","last_became_peered":"2024-03-26T08:31:21.370103+0000","last_unstale":"2024-03-26T08:33:09.837431+0000","last_undegraded":"2024-03-26T08:33:09.837431+0000","last_fullsized":"2024-03-26T08:33:09.837431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:54:52.657768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62321,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":174,"num_read_kb":149,"num_write":31,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.348143+0000","last_change":"2024-03-26T08:31:22.373102+0000","last_active":"2024-03-26T08:33:06.348143+0000","last_peered":"2024-03-26T08:33:06.348143+0000","last_clean":"2024-03-26T08:33:06.348143+0000","last_became_active":"2024-03-26T08:31:22.372792+0000","last_became_peered":"2024-03-26T08:31:22.372792+0000","last_unstale":"2024-03-26T08:33:06.348143+0000","last_undegraded":"2024-03-26T08:33:06.348143+0000","last_fullsized":"2024-03-26T08:33:06.348143+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:58:58.274941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.454102+0000","last_change":"2024-03-26T08:31:26.448172+0000","last_active":"2024-03-26T08:32:38.454102+0000","last_peered":"2024-03-26T08:32:38.454102+0000","last_clean":"2024-03-26T08:32:38.454102+0000","last_became_active":"2024-03-26T08:31:26.447550+0000","last_became_peered":"2024-03-26T08:31:26.447550+0000","last_unstale":"2024-03-26T08:32:38.454102+0000","last_undegraded":"2024-03-26T08:32:38.454102+0000","last_fullsized":"2024-03-26T08:32:38.454102+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:56:19.806076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.310789+0000","last_change":"2024-03-26T08:31:21.363946+0000","last_active":"2024-03-26T08:32:38.310789+0000","last_peered":"2024-03-26T08:32:38.310789+0000","last_clean":"2024-03-26T08:32:38.310789+0000","last_became_active":"2024-03-26T08:31:21.363566+0000","last_became_peered":"2024-03-26T08:31:21.363566+0000","last_unstale":"2024-03-26T08:32:38.310789+0000","last_undegraded":"2024-03-26T08:32:38.310789+0000","last_fullsized":"2024-03-26T08:32:38.310789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:32:51.714454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"94'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.349700+0000","last_change":"2024-03-26T08:31:22.375813+0000","last_active":"2024-03-26T08:33:06.349700+0000","last_peered":"2024-03-26T08:33:06.349700+0000","last_clean":"2024-03-26T08:33:06.349700+0000","last_became_active":"2024-03-26T08:31:22.375437+0000","last_became_peered":"2024-03-26T08:31:22.375437+0000","last_unstale":"2024-03-26T08:33:06.349700+0000","last_undegraded":"2024-03-26T08:33:06.349700+0000","last_fullsized":"2024-03-26T08:33:06.349700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:48:25.167143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.571501+0000","last_change":"2024-03-26T08:31:26.452079+0000","last_active":"2024-03-26T08:32:38.571501+0000","last_peered":"2024-03-26T08:32:38.571501+0000","last_clean":"2024-03-26T08:32:38.571501+0000","last_became_active":"2024-03-26T08:31:26.451146+0000","last_became_peered":"2024-03-26T08:31:26.451146+0000","last_unstale":"2024-03-26T08:32:38.571501+0000","last_undegraded":"2024-03-26T08:32:38.571501+0000","last_fullsized":"2024-03-26T08:32:38.571501+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:08:57.803180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.456006+0000","last_change":"2024-03-26T08:31:21.371222+0000","last_active":"2024-03-26T08:32:38.456006+0000","last_peered":"2024-03-26T08:32:38.456006+0000","last_clean":"2024-03-26T08:32:38.456006+0000","last_became_active":"2024-03-26T08:31:21.370843+0000","last_became_peered":"2024-03-26T08:31:21.370843+0000","last_unstale":"2024-03-26T08:32:38.456006+0000","last_undegraded":"2024-03-26T08:32:38.456006+0000","last_fullsized":"2024-03-26T08:32:38.456006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:32:00.518713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"94'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.350801+0000","last_change":"2024-03-26T08:31:22.379781+0000","last_active":"2024-03-26T08:33:06.350801+0000","last_peered":"2024-03-26T08:33:06.350801+0000","last_clean":"2024-03-26T08:33:06.350801+0000","last_became_active":"2024-03-26T08:31:22.379357+0000","last_became_peered":"2024-03-26T08:31:22.379357+0000","last_unstale":"2024-03-26T08:33:06.350801+0000","last_undegraded":"2024-03-26T08:33:06.350801+0000","last_fullsized":"2024-03-26T08:33:06.350801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:21:52.306343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.267928+0000","last_change":"2024-03-26T08:31:37.498490+0000","last_active":"2024-03-26T08:32:38.267928+0000","last_peered":"2024-03-26T08:32:38.267928+0000","last_clean":"2024-03-26T08:32:38.267928+0000","last_became_active":"2024-03-26T08:31:37.496608+0000","last_became_peered":"2024-03-26T08:31:37.496608+0000","last_unstale":"2024-03-26T08:32:38.267928+0000","last_undegraded":"2024-03-26T08:32:38.267928+0000","last_fullsized":"2024-03-26T08:32:38.267928+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:32:11.653087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.267117+0000","last_change":"2024-03-26T08:31:21.366945+0000","last_active":"2024-03-26T08:32:38.267117+0000","last_peered":"2024-03-26T08:32:38.267117+0000","last_clean":"2024-03-26T08:32:38.267117+0000","last_became_active":"2024-03-26T08:31:21.366548+0000","last_became_peered":"2024-03-26T08:31:21.366548+0000","last_unstale":"2024-03-26T08:32:38.267117+0000","last_undegraded":"2024-03-26T08:32:38.267117+0000","last_fullsized":"2024-03-26T08:32:38.267117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:13:52.037163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.347687+0000","last_change":"2024-03-26T08:31:22.387575+0000","last_active":"2024-03-26T08:33:06.347687+0000","last_peered":"2024-03-26T08:33:06.347687+0000","last_clean":"2024-03-26T08:33:06.347687+0000","last_became_active":"2024-03-26T08:31:22.383815+0000","last_became_peered":"2024-03-26T08:31:22.383815+0000","last_unstale":"2024-03-26T08:33:06.347687+0000","last_undegraded":"2024-03-26T08:33:06.347687+0000","last_fullsized":"2024-03-26T08:33:06.347687+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:10:20.318597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.570150+0000","last_change":"2024-03-26T08:31:26.440745+0000","last_active":"2024-03-26T08:32:38.570150+0000","last_peered":"2024-03-26T08:32:38.570150+0000","last_clean":"2024-03-26T08:32:38.570150+0000","last_became_active":"2024-03-26T08:31:26.440199+0000","last_became_peered":"2024-03-26T08:31:26.440199+0000","last_unstale":"2024-03-26T08:32:38.570150+0000","last_undegraded":"2024-03-26T08:32:38.570150+0000","last_fullsized":"2024-03-26T08:32:38.570150+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:25:11.407274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.266699+0000","last_change":"2024-03-26T08:31:21.369227+0000","last_active":"2024-03-26T08:32:38.266699+0000","last_peered":"2024-03-26T08:32:38.266699+0000","last_clean":"2024-03-26T08:32:38.266699+0000","last_became_active":"2024-03-26T08:31:21.368709+0000","last_became_peered":"2024-03-26T08:31:21.368709+0000","last_unstale":"2024-03-26T08:32:38.266699+0000","last_undegraded":"2024-03-26T08:32:38.266699+0000","last_fullsized":"2024-03-26T08:32:38.266699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:30:04.714715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"94'6","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.330464+0000","last_change":"2024-03-26T08:31:22.384199+0000","last_active":"2024-03-26T08:33:06.330464+0000","last_peered":"2024-03-26T08:33:06.330464+0000","last_clean":"2024-03-26T08:33:06.330464+0000","last_became_active":"2024-03-26T08:31:22.383854+0000","last_became_peered":"2024-03-26T08:31:22.383854+0000","last_unstale":"2024-03-26T08:33:06.330464+0000","last_undegraded":"2024-03-26T08:33:06.330464+0000","last_fullsized":"2024-03-26T08:33:06.330464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:33:24.240505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.453908+0000","last_change":"2024-03-26T08:31:26.447657+0000","last_active":"2024-03-26T08:32:38.453908+0000","last_peered":"2024-03-26T08:32:38.453908+0000","last_clean":"2024-03-26T08:32:38.453908+0000","last_became_active":"2024-03-26T08:31:26.444812+0000","last_became_peered":"2024-03-26T08:31:26.444812+0000","last_unstale":"2024-03-26T08:32:38.453908+0000","last_undegraded":"2024-03-26T08:32:38.453908+0000","last_fullsized":"2024-03-26T08:32:38.453908+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:58:21.422318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.574367+0000","last_change":"2024-03-26T08:31:37.510665+0000","last_active":"2024-03-26T08:32:38.574367+0000","last_peered":"2024-03-26T08:32:38.574367+0000","last_clean":"2024-03-26T08:32:38.574367+0000","last_became_active":"2024-03-26T08:31:37.504109+0000","last_became_peered":"2024-03-26T08:31:37.504109+0000","last_unstale":"2024-03-26T08:32:38.574367+0000","last_undegraded":"2024-03-26T08:32:38.574367+0000","last_fullsized":"2024-03-26T08:32:38.574367+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:38:21.446881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"94'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.331161+0000","last_change":"2024-03-26T08:31:37.502088+0000","last_active":"2024-03-26T08:33:06.331161+0000","last_peered":"2024-03-26T08:33:06.331161+0000","last_clean":"2024-03-26T08:33:06.331161+0000","last_became_active":"2024-03-26T08:31:37.499767+0000","last_became_peered":"2024-03-26T08:31:37.499767+0000","last_unstale":"2024-03-26T08:33:06.331161+0000","last_undegraded":"2024-03-26T08:33:06.331161+0000","last_fullsized":"2024-03-26T08:33:06.331161+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:55:32.652231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.267066+0000","last_change":"2024-03-26T08:31:26.479654+0000","last_active":"2024-03-26T08:32:38.267066+0000","last_peered":"2024-03-26T08:32:38.267066+0000","last_clean":"2024-03-26T08:32:38.267066+0000","last_became_active":"2024-03-26T08:31:26.478948+0000","last_became_peered":"2024-03-26T08:31:26.478948+0000","last_unstale":"2024-03-26T08:32:38.267066+0000","last_undegraded":"2024-03-26T08:32:38.267066+0000","last_fullsized":"2024-03-26T08:32:38.267066+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:33:58.445531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.272835+0000","last_change":"2024-03-26T08:31:21.365203+0000","last_active":"2024-03-26T08:32:38.272835+0000","last_peered":"2024-03-26T08:32:38.272835+0000","last_clean":"2024-03-26T08:32:38.272835+0000","last_became_active":"2024-03-26T08:31:21.364816+0000","last_became_peered":"2024-03-26T08:31:21.364816+0000","last_unstale":"2024-03-26T08:32:38.272835+0000","last_undegraded":"2024-03-26T08:32:38.272835+0000","last_fullsized":"2024-03-26T08:32:38.272835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:59:50.157011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"94'4","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.341384+0000","last_change":"2024-03-26T08:31:22.392014+0000","last_active":"2024-03-26T08:33:06.341384+0000","last_peered":"2024-03-26T08:33:06.341384+0000","last_clean":"2024-03-26T08:33:06.341384+0000","last_became_active":"2024-03-26T08:31:22.391627+0000","last_became_peered":"2024-03-26T08:31:22.391627+0000","last_unstale":"2024-03-26T08:33:06.341384+0000","last_undegraded":"2024-03-26T08:33:06.341384+0000","last_fullsized":"2024-03-26T08:33:06.341384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:59:12.544248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.570150+0000","last_change":"2024-03-26T08:31:26.444754+0000","last_active":"2024-03-26T08:32:38.570150+0000","last_peered":"2024-03-26T08:32:38.570150+0000","last_clean":"2024-03-26T08:32:38.570150+0000","last_became_active":"2024-03-26T08:31:26.444331+0000","last_became_peered":"2024-03-26T08:31:26.444331+0000","last_unstale":"2024-03-26T08:32:38.570150+0000","last_undegraded":"2024-03-26T08:32:38.570150+0000","last_fullsized":"2024-03-26T08:32:38.570150+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:27:58.430695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.572968+0000","last_change":"2024-03-26T08:31:21.371028+0000","last_active":"2024-03-26T08:32:38.572968+0000","last_peered":"2024-03-26T08:32:38.572968+0000","last_clean":"2024-03-26T08:32:38.572968+0000","last_became_active":"2024-03-26T08:31:21.370631+0000","last_became_peered":"2024-03-26T08:31:21.370631+0000","last_unstale":"2024-03-26T08:32:38.572968+0000","last_undegraded":"2024-03-26T08:32:38.572968+0000","last_fullsized":"2024-03-26T08:32:38.572968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:33:25.546255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"94'7","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.335399+0000","last_change":"2024-03-26T08:31:22.388791+0000","last_active":"2024-03-26T08:33:06.335399+0000","last_peered":"2024-03-26T08:33:06.335399+0000","last_clean":"2024-03-26T08:33:06.335399+0000","last_became_active":"2024-03-26T08:31:22.388342+0000","last_became_peered":"2024-03-26T08:31:22.388342+0000","last_unstale":"2024-03-26T08:33:06.335399+0000","last_undegraded":"2024-03-26T08:33:06.335399+0000","last_fullsized":"2024-03-26T08:33:06.335399+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:26:30.537880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.570444+0000","last_change":"2024-03-26T08:31:26.478919+0000","last_active":"2024-03-26T08:32:38.570444+0000","last_peered":"2024-03-26T08:32:38.570444+0000","last_clean":"2024-03-26T08:32:38.570444+0000","last_became_active":"2024-03-26T08:31:26.478330+0000","last_became_peered":"2024-03-26T08:31:26.478330+0000","last_unstale":"2024-03-26T08:32:38.570444+0000","last_undegraded":"2024-03-26T08:32:38.570444+0000","last_fullsized":"2024-03-26T08:32:38.570444+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:37:36.421982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"94'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.335085+0000","last_change":"2024-03-26T08:31:22.384299+0000","last_active":"2024-03-26T08:33:06.335085+0000","last_peered":"2024-03-26T08:33:06.335085+0000","last_clean":"2024-03-26T08:33:06.335085+0000","last_became_active":"2024-03-26T08:31:22.383955+0000","last_became_peered":"2024-03-26T08:31:22.383955+0000","last_unstale":"2024-03-26T08:33:06.335085+0000","last_undegraded":"2024-03-26T08:33:06.335085+0000","last_fullsized":"2024-03-26T08:33:06.335085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:08:53.065524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.570916+0000","last_change":"2024-03-26T08:31:21.358464+0000","last_active":"2024-03-26T08:32:38.570916+0000","last_peered":"2024-03-26T08:32:38.570916+0000","last_clean":"2024-03-26T08:32:38.570916+0000","last_became_active":"2024-03-26T08:31:21.356853+0000","last_became_peered":"2024-03-26T08:31:21.356853+0000","last_unstale":"2024-03-26T08:32:38.570916+0000","last_undegraded":"2024-03-26T08:32:38.570916+0000","last_fullsized":"2024-03-26T08:32:38.570916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:08:21.395306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"92'1","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.572677+0000","last_change":"2024-03-26T08:31:37.502144+0000","last_active":"2024-03-26T08:32:38.572677+0000","last_peered":"2024-03-26T08:32:38.572677+0000","last_clean":"2024-03-26T08:32:38.572677+0000","last_became_active":"2024-03-26T08:31:37.501465+0000","last_became_peered":"2024-03-26T08:31:37.501465+0000","last_unstale":"2024-03-26T08:32:38.572677+0000","last_undegraded":"2024-03-26T08:32:38.572677+0000","last_fullsized":"2024-03-26T08:32:38.572677+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:30:18.771898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"94'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.332952+0000","last_change":"2024-03-26T08:31:22.403380+0000","last_active":"2024-03-26T08:33:06.332952+0000","last_peered":"2024-03-26T08:33:06.332952+0000","last_clean":"2024-03-26T08:33:06.332952+0000","last_became_active":"2024-03-26T08:31:22.400991+0000","last_became_peered":"2024-03-26T08:31:22.400991+0000","last_unstale":"2024-03-26T08:33:06.332952+0000","last_undegraded":"2024-03-26T08:33:06.332952+0000","last_fullsized":"2024-03-26T08:33:06.332952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:04:38.842919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.268183+0000","last_change":"2024-03-26T08:31:21.358570+0000","last_active":"2024-03-26T08:32:38.268183+0000","last_peered":"2024-03-26T08:32:38.268183+0000","last_clean":"2024-03-26T08:32:38.268183+0000","last_became_active":"2024-03-26T08:31:21.356603+0000","last_became_peered":"2024-03-26T08:31:21.356603+0000","last_unstale":"2024-03-26T08:32:38.268183+0000","last_undegraded":"2024-03-26T08:32:38.268183+0000","last_fullsized":"2024-03-26T08:32:38.268183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:10:28.829223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":356,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.573161+0000","last_change":"2024-03-26T08:29:39.048593+0000","last_active":"2024-03-26T08:32:38.573161+0000","last_peered":"2024-03-26T08:32:38.573161+0000","last_clean":"2024-03-26T08:32:38.573161+0000","last_became_active":"2024-03-26T08:29:39.047909+0000","last_became_peered":"2024-03-26T08:29:39.047909+0000","last_unstale":"2024-03-26T08:32:38.573161+0000","last_undegraded":"2024-03-26T08:32:38.573161+0000","last_fullsized":"2024-03-26T08:32:38.573161+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:24:55.011726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:24:55.011726+0000","last_clean_scrub_stamp":"2024-03-26T08:24:55.011726+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:46:55.510857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.458439+0000","last_change":"2024-03-26T08:31:26.478740+0000","last_active":"2024-03-26T08:32:38.458439+0000","last_peered":"2024-03-26T08:32:38.458439+0000","last_clean":"2024-03-26T08:32:38.458439+0000","last_became_active":"2024-03-26T08:31:26.478218+0000","last_became_peered":"2024-03-26T08:31:26.478218+0000","last_unstale":"2024-03-26T08:32:38.458439+0000","last_undegraded":"2024-03-26T08:32:38.458439+0000","last_fullsized":"2024-03-26T08:32:38.458439+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:02:54.506805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"94'5","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.331680+0000","last_change":"2024-03-26T08:31:22.381931+0000","last_active":"2024-03-26T08:33:06.331680+0000","last_peered":"2024-03-26T08:33:06.331680+0000","last_clean":"2024-03-26T08:33:06.331680+0000","last_became_active":"2024-03-26T08:31:22.379021+0000","last_became_peered":"2024-03-26T08:31:22.379021+0000","last_unstale":"2024-03-26T08:33:06.331680+0000","last_undegraded":"2024-03-26T08:33:06.331680+0000","last_fullsized":"2024-03-26T08:33:06.331680+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:07:31.555154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.572111+0000","last_change":"2024-03-26T08:31:21.375753+0000","last_active":"2024-03-26T08:32:38.572111+0000","last_peered":"2024-03-26T08:32:38.572111+0000","last_clean":"2024-03-26T08:32:38.572111+0000","last_became_active":"2024-03-26T08:31:21.375340+0000","last_became_peered":"2024-03-26T08:31:21.375340+0000","last_unstale":"2024-03-26T08:32:38.572111+0000","last_undegraded":"2024-03-26T08:32:38.572111+0000","last_fullsized":"2024-03-26T08:32:38.572111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:21:14.241774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.571348+0000","last_change":"2024-03-26T08:31:26.454026+0000","last_active":"2024-03-26T08:32:38.571348+0000","last_peered":"2024-03-26T08:32:38.571348+0000","last_clean":"2024-03-26T08:32:38.571348+0000","last_became_active":"2024-03-26T08:31:26.453494+0000","last_became_peered":"2024-03-26T08:31:26.453494+0000","last_unstale":"2024-03-26T08:32:38.571348+0000","last_undegraded":"2024-03-26T08:32:38.571348+0000","last_fullsized":"2024-03-26T08:32:38.571348+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:30:10.388280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.268131+0000","last_change":"2024-03-26T08:31:21.358930+0000","last_active":"2024-03-26T08:32:38.268131+0000","last_peered":"2024-03-26T08:32:38.268131+0000","last_clean":"2024-03-26T08:32:38.268131+0000","last_became_active":"2024-03-26T08:31:21.357573+0000","last_became_peered":"2024-03-26T08:31:21.357573+0000","last_unstale":"2024-03-26T08:32:38.268131+0000","last_undegraded":"2024-03-26T08:32:38.268131+0000","last_fullsized":"2024-03-26T08:32:38.268131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:36:41.572993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"94'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.343400+0000","last_change":"2024-03-26T08:31:22.378520+0000","last_active":"2024-03-26T08:33:06.343400+0000","last_peered":"2024-03-26T08:33:06.343400+0000","last_clean":"2024-03-26T08:33:06.343400+0000","last_became_active":"2024-03-26T08:31:22.378081+0000","last_became_peered":"2024-03-26T08:31:22.378081+0000","last_unstale":"2024-03-26T08:33:06.343400+0000","last_undegraded":"2024-03-26T08:33:06.343400+0000","last_fullsized":"2024-03-26T08:33:06.343400+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:09:34.748182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.265421+0000","last_change":"2024-03-26T08:31:26.479545+0000","last_active":"2024-03-26T08:32:38.265421+0000","last_peered":"2024-03-26T08:32:38.265421+0000","last_clean":"2024-03-26T08:32:38.265421+0000","last_became_active":"2024-03-26T08:31:26.479029+0000","last_became_peered":"2024-03-26T08:31:26.479029+0000","last_unstale":"2024-03-26T08:32:38.265421+0000","last_undegraded":"2024-03-26T08:32:38.265421+0000","last_fullsized":"2024-03-26T08:32:38.265421+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:49:03.795752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.273412+0000","last_change":"2024-03-26T08:31:21.360510+0000","last_active":"2024-03-26T08:32:38.273412+0000","last_peered":"2024-03-26T08:32:38.273412+0000","last_clean":"2024-03-26T08:32:38.273412+0000","last_became_active":"2024-03-26T08:31:21.359593+0000","last_became_peered":"2024-03-26T08:31:21.359593+0000","last_unstale":"2024-03-26T08:32:38.273412+0000","last_undegraded":"2024-03-26T08:32:38.273412+0000","last_fullsized":"2024-03-26T08:32:38.273412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:37:33.752689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"94'5","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.348139+0000","last_change":"2024-03-26T08:31:22.390181+0000","last_active":"2024-03-26T08:33:06.348139+0000","last_peered":"2024-03-26T08:33:06.348139+0000","last_clean":"2024-03-26T08:33:06.348139+0000","last_became_active":"2024-03-26T08:31:22.389686+0000","last_became_peered":"2024-03-26T08:31:22.389686+0000","last_unstale":"2024-03-26T08:33:06.348139+0000","last_undegraded":"2024-03-26T08:33:06.348139+0000","last_fullsized":"2024-03-26T08:33:06.348139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:48:09.913134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.309800+0000","last_change":"2024-03-26T08:31:26.449160+0000","last_active":"2024-03-26T08:32:38.309800+0000","last_peered":"2024-03-26T08:32:38.309800+0000","last_clean":"2024-03-26T08:32:38.309800+0000","last_became_active":"2024-03-26T08:31:26.448570+0000","last_became_peered":"2024-03-26T08:31:26.448570+0000","last_unstale":"2024-03-26T08:32:38.309800+0000","last_undegraded":"2024-03-26T08:32:38.309800+0000","last_fullsized":"2024-03-26T08:32:38.309800+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:05:21.258982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.268339+0000","last_change":"2024-03-26T08:31:21.365723+0000","last_active":"2024-03-26T08:32:38.268339+0000","last_peered":"2024-03-26T08:32:38.268339+0000","last_clean":"2024-03-26T08:32:38.268339+0000","last_became_active":"2024-03-26T08:31:21.364893+0000","last_became_peered":"2024-03-26T08:31:21.364893+0000","last_unstale":"2024-03-26T08:32:38.268339+0000","last_undegraded":"2024-03-26T08:32:38.268339+0000","last_fullsized":"2024-03-26T08:32:38.268339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:24:17.210093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.342502+0000","last_change":"2024-03-26T08:31:22.396341+0000","last_active":"2024-03-26T08:33:06.342502+0000","last_peered":"2024-03-26T08:33:06.342502+0000","last_clean":"2024-03-26T08:33:06.342502+0000","last_became_active":"2024-03-26T08:31:22.395956+0000","last_became_peered":"2024-03-26T08:31:22.395956+0000","last_unstale":"2024-03-26T08:33:06.342502+0000","last_undegraded":"2024-03-26T08:33:06.342502+0000","last_fullsized":"2024-03-26T08:33:06.342502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:08:51.194370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.267538+0000","last_change":"2024-03-26T08:31:37.497971+0000","last_active":"2024-03-26T08:32:38.267538+0000","last_peered":"2024-03-26T08:32:38.267538+0000","last_clean":"2024-03-26T08:32:38.267538+0000","last_became_active":"2024-03-26T08:31:37.497379+0000","last_became_peered":"2024-03-26T08:31:37.497379+0000","last_unstale":"2024-03-26T08:32:38.267538+0000","last_undegraded":"2024-03-26T08:32:38.267538+0000","last_fullsized":"2024-03-26T08:32:38.267538+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:35:55.266756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.534861+0000","last_change":"2024-03-26T08:31:37.493616+0000","last_active":"2024-03-26T08:32:38.534861+0000","last_peered":"2024-03-26T08:32:38.534861+0000","last_clean":"2024-03-26T08:32:38.534861+0000","last_became_active":"2024-03-26T08:31:37.492288+0000","last_became_peered":"2024-03-26T08:31:37.492288+0000","last_unstale":"2024-03-26T08:32:38.534861+0000","last_undegraded":"2024-03-26T08:32:38.534861+0000","last_fullsized":"2024-03-26T08:32:38.534861+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:49:10.413751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"94'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.275996+0000","last_change":"2024-03-26T08:31:22.384347+0000","last_active":"2024-03-26T08:33:06.275996+0000","last_peered":"2024-03-26T08:33:06.275996+0000","last_clean":"2024-03-26T08:33:06.275996+0000","last_became_active":"2024-03-26T08:31:22.380639+0000","last_became_peered":"2024-03-26T08:31:22.380639+0000","last_unstale":"2024-03-26T08:33:06.275996+0000","last_undegraded":"2024-03-26T08:33:06.275996+0000","last_fullsized":"2024-03-26T08:33:06.275996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:39:07.778005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.273214+0000","last_change":"2024-03-26T08:31:26.455706+0000","last_active":"2024-03-26T08:32:38.273214+0000","last_peered":"2024-03-26T08:32:38.273214+0000","last_clean":"2024-03-26T08:32:38.273214+0000","last_became_active":"2024-03-26T08:31:26.454591+0000","last_became_peered":"2024-03-26T08:31:26.454591+0000","last_unstale":"2024-03-26T08:32:38.273214+0000","last_undegraded":"2024-03-26T08:32:38.273214+0000","last_fullsized":"2024-03-26T08:32:38.273214+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:07:30.612220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.266005+0000","last_change":"2024-03-26T08:31:21.357734+0000","last_active":"2024-03-26T08:32:38.266005+0000","last_peered":"2024-03-26T08:32:38.266005+0000","last_clean":"2024-03-26T08:32:38.266005+0000","last_became_active":"2024-03-26T08:31:21.357361+0000","last_became_peered":"2024-03-26T08:31:21.357361+0000","last_unstale":"2024-03-26T08:32:38.266005+0000","last_undegraded":"2024-03-26T08:32:38.266005+0000","last_fullsized":"2024-03-26T08:32:38.266005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:26:57.454577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"94'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.338137+0000","last_change":"2024-03-26T08:31:22.383118+0000","last_active":"2024-03-26T08:33:06.338137+0000","last_peered":"2024-03-26T08:33:06.338137+0000","last_clean":"2024-03-26T08:33:06.338137+0000","last_became_active":"2024-03-26T08:31:22.382773+0000","last_became_peered":"2024-03-26T08:31:22.382773+0000","last_unstale":"2024-03-26T08:33:06.338137+0000","last_undegraded":"2024-03-26T08:33:06.338137+0000","last_fullsized":"2024-03-26T08:33:06.338137+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:50:34.889914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"92'1","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.268151+0000","last_change":"2024-03-26T08:31:26.451467+0000","last_active":"2024-03-26T08:32:38.268151+0000","last_peered":"2024-03-26T08:32:38.268151+0000","last_clean":"2024-03-26T08:32:38.268151+0000","last_became_active":"2024-03-26T08:31:26.451022+0000","last_became_peered":"2024-03-26T08:31:26.451022+0000","last_unstale":"2024-03-26T08:32:38.268151+0000","last_undegraded":"2024-03-26T08:32:38.268151+0000","last_fullsized":"2024-03-26T08:32:38.268151+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:19:50.442779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.310275+0000","last_change":"2024-03-26T08:31:21.361596+0000","last_active":"2024-03-26T08:32:38.310275+0000","last_peered":"2024-03-26T08:32:38.310275+0000","last_clean":"2024-03-26T08:32:38.310275+0000","last_became_active":"2024-03-26T08:31:21.357502+0000","last_became_peered":"2024-03-26T08:31:21.357502+0000","last_unstale":"2024-03-26T08:32:38.310275+0000","last_undegraded":"2024-03-26T08:32:38.310275+0000","last_fullsized":"2024-03-26T08:32:38.310275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:05:33.951346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"94'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.339334+0000","last_change":"2024-03-26T08:31:22.397344+0000","last_active":"2024-03-26T08:33:06.339334+0000","last_peered":"2024-03-26T08:33:06.339334+0000","last_clean":"2024-03-26T08:33:06.339334+0000","last_became_active":"2024-03-26T08:31:22.396936+0000","last_became_peered":"2024-03-26T08:31:22.396936+0000","last_unstale":"2024-03-26T08:33:06.339334+0000","last_undegraded":"2024-03-26T08:33:06.339334+0000","last_fullsized":"2024-03-26T08:33:06.339334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:07:49.412098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.267394+0000","last_change":"2024-03-26T08:31:26.479060+0000","last_active":"2024-03-26T08:32:38.267394+0000","last_peered":"2024-03-26T08:32:38.267394+0000","last_clean":"2024-03-26T08:32:38.267394+0000","last_became_active":"2024-03-26T08:31:26.478519+0000","last_became_peered":"2024-03-26T08:31:26.478519+0000","last_unstale":"2024-03-26T08:32:38.267394+0000","last_undegraded":"2024-03-26T08:32:38.267394+0000","last_fullsized":"2024-03-26T08:32:38.267394+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:37:20.528845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.267142+0000","last_change":"2024-03-26T08:31:21.374128+0000","last_active":"2024-03-26T08:32:38.267142+0000","last_peered":"2024-03-26T08:32:38.267142+0000","last_clean":"2024-03-26T08:32:38.267142+0000","last_became_active":"2024-03-26T08:31:21.372934+0000","last_became_peered":"2024-03-26T08:31:21.372934+0000","last_unstale":"2024-03-26T08:32:38.267142+0000","last_undegraded":"2024-03-26T08:32:38.267142+0000","last_fullsized":"2024-03-26T08:32:38.267142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:25:15.919855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"94'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.349410+0000","last_change":"2024-03-26T08:31:22.381847+0000","last_active":"2024-03-26T08:33:06.349410+0000","last_peered":"2024-03-26T08:33:06.349410+0000","last_clean":"2024-03-26T08:33:06.349410+0000","last_became_active":"2024-03-26T08:31:22.379956+0000","last_became_peered":"2024-03-26T08:31:22.379956+0000","last_unstale":"2024-03-26T08:33:06.349410+0000","last_undegraded":"2024-03-26T08:33:06.349410+0000","last_fullsized":"2024-03-26T08:33:06.349410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:16:38.048495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.534107+0000","last_change":"2024-03-26T08:31:26.455503+0000","last_active":"2024-03-26T08:32:38.534107+0000","last_peered":"2024-03-26T08:32:38.534107+0000","last_clean":"2024-03-26T08:32:38.534107+0000","last_became_active":"2024-03-26T08:31:26.455023+0000","last_became_peered":"2024-03-26T08:31:26.455023+0000","last_unstale":"2024-03-26T08:32:38.534107+0000","last_undegraded":"2024-03-26T08:32:38.534107+0000","last_fullsized":"2024-03-26T08:32:38.534107+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:06:47.565407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.311173+0000","last_change":"2024-03-26T08:31:21.362832+0000","last_active":"2024-03-26T08:32:38.311173+0000","last_peered":"2024-03-26T08:32:38.311173+0000","last_clean":"2024-03-26T08:32:38.311173+0000","last_became_active":"2024-03-26T08:31:21.362411+0000","last_became_peered":"2024-03-26T08:31:21.362411+0000","last_unstale":"2024-03-26T08:32:38.311173+0000","last_undegraded":"2024-03-26T08:32:38.311173+0000","last_fullsized":"2024-03-26T08:32:38.311173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:34:07.646011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.349836+0000","last_change":"2024-03-26T08:31:22.403106+0000","last_active":"2024-03-26T08:33:06.349836+0000","last_peered":"2024-03-26T08:33:06.349836+0000","last_clean":"2024-03-26T08:33:06.349836+0000","last_became_active":"2024-03-26T08:31:22.401172+0000","last_became_peered":"2024-03-26T08:31:22.401172+0000","last_unstale":"2024-03-26T08:33:06.349836+0000","last_undegraded":"2024-03-26T08:33:06.349836+0000","last_fullsized":"2024-03-26T08:33:06.349836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:31:42.654817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.572016+0000","last_change":"2024-03-26T08:31:37.495759+0000","last_active":"2024-03-26T08:32:38.572016+0000","last_peered":"2024-03-26T08:32:38.572016+0000","last_clean":"2024-03-26T08:32:38.572016+0000","last_became_active":"2024-03-26T08:31:37.493610+0000","last_became_peered":"2024-03-26T08:31:37.493610+0000","last_unstale":"2024-03-26T08:32:38.572016+0000","last_undegraded":"2024-03-26T08:32:38.572016+0000","last_fullsized":"2024-03-26T08:32:38.572016+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:17:05.345033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.458336+0000","last_change":"2024-03-26T08:31:21.369396+0000","last_active":"2024-03-26T08:32:38.458336+0000","last_peered":"2024-03-26T08:32:38.458336+0000","last_clean":"2024-03-26T08:32:38.458336+0000","last_became_active":"2024-03-26T08:31:21.364011+0000","last_became_peered":"2024-03-26T08:31:21.364011+0000","last_unstale":"2024-03-26T08:32:38.458336+0000","last_undegraded":"2024-03-26T08:32:38.458336+0000","last_fullsized":"2024-03-26T08:32:38.458336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:23:47.347158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.350016+0000","last_change":"2024-03-26T08:31:22.389586+0000","last_active":"2024-03-26T08:33:06.350016+0000","last_peered":"2024-03-26T08:33:06.350016+0000","last_clean":"2024-03-26T08:33:06.350016+0000","last_became_active":"2024-03-26T08:31:22.389200+0000","last_became_peered":"2024-03-26T08:31:22.389200+0000","last_unstale":"2024-03-26T08:33:06.350016+0000","last_undegraded":"2024-03-26T08:33:06.350016+0000","last_fullsized":"2024-03-26T08:33:06.350016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:54:30.076744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.454375+0000","last_change":"2024-03-26T08:31:26.452274+0000","last_active":"2024-03-26T08:32:38.454375+0000","last_peered":"2024-03-26T08:32:38.454375+0000","last_clean":"2024-03-26T08:32:38.454375+0000","last_became_active":"2024-03-26T08:31:26.451670+0000","last_became_peered":"2024-03-26T08:31:26.451670+0000","last_unstale":"2024-03-26T08:32:38.454375+0000","last_undegraded":"2024-03-26T08:32:38.454375+0000","last_fullsized":"2024-03-26T08:32:38.454375+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:48:36.028810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.573718+0000","last_change":"2024-03-26T08:31:21.356259+0000","last_active":"2024-03-26T08:32:38.573718+0000","last_peered":"2024-03-26T08:32:38.573718+0000","last_clean":"2024-03-26T08:32:38.573718+0000","last_became_active":"2024-03-26T08:31:21.355724+0000","last_became_peered":"2024-03-26T08:31:21.355724+0000","last_unstale":"2024-03-26T08:32:38.573718+0000","last_undegraded":"2024-03-26T08:32:38.573718+0000","last_fullsized":"2024-03-26T08:32:38.573718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:30:05.621417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.340153+0000","last_change":"2024-03-26T08:31:22.370869+0000","last_active":"2024-03-26T08:33:06.340153+0000","last_peered":"2024-03-26T08:33:06.340153+0000","last_clean":"2024-03-26T08:33:06.340153+0000","last_became_active":"2024-03-26T08:31:22.369797+0000","last_became_peered":"2024-03-26T08:31:22.369797+0000","last_unstale":"2024-03-26T08:33:06.340153+0000","last_undegraded":"2024-03-26T08:33:06.340153+0000","last_fullsized":"2024-03-26T08:33:06.340153+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:59:33.064619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"94'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.336607+0000","last_change":"2024-03-26T08:31:22.377927+0000","last_active":"2024-03-26T08:33:06.336607+0000","last_peered":"2024-03-26T08:33:06.336607+0000","last_clean":"2024-03-26T08:33:06.336607+0000","last_became_active":"2024-03-26T08:31:22.377559+0000","last_became_peered":"2024-03-26T08:31:22.377559+0000","last_unstale":"2024-03-26T08:33:06.336607+0000","last_undegraded":"2024-03-26T08:33:06.336607+0000","last_fullsized":"2024-03-26T08:33:06.336607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:06:05.544179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"94'12","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.340397+0000","last_change":"2024-03-26T08:31:22.380287+0000","last_active":"2024-03-26T08:33:06.340397+0000","last_peered":"2024-03-26T08:33:06.340397+0000","last_clean":"2024-03-26T08:33:06.340397+0000","last_became_active":"2024-03-26T08:31:22.379948+0000","last_became_peered":"2024-03-26T08:31:22.379948+0000","last_unstale":"2024-03-26T08:33:06.340397+0000","last_undegraded":"2024-03-26T08:33:06.340397+0000","last_fullsized":"2024-03-26T08:33:06.340397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:57:12.636151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"94'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.333723+0000","last_change":"2024-03-26T08:31:22.390109+0000","last_active":"2024-03-26T08:33:06.333723+0000","last_peered":"2024-03-26T08:33:06.333723+0000","last_clean":"2024-03-26T08:33:06.333723+0000","last_became_active":"2024-03-26T08:31:22.389766+0000","last_became_peered":"2024-03-26T08:31:22.389766+0000","last_unstale":"2024-03-26T08:33:06.333723+0000","last_undegraded":"2024-03-26T08:33:06.333723+0000","last_fullsized":"2024-03-26T08:33:06.333723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:24:27.454598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.341634+0000","last_change":"2024-03-26T08:31:22.391882+0000","last_active":"2024-03-26T08:33:06.341634+0000","last_peered":"2024-03-26T08:33:06.341634+0000","last_clean":"2024-03-26T08:33:06.341634+0000","last_became_active":"2024-03-26T08:31:22.391395+0000","last_became_peered":"2024-03-26T08:31:22.391395+0000","last_unstale":"2024-03-26T08:33:06.341634+0000","last_undegraded":"2024-03-26T08:33:06.341634+0000","last_fullsized":"2024-03-26T08:33:06.341634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:45:06.685287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"94'4","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.333414+0000","last_change":"2024-03-26T08:31:22.385993+0000","last_active":"2024-03-26T08:33:06.333414+0000","last_peered":"2024-03-26T08:33:06.333414+0000","last_clean":"2024-03-26T08:33:06.333414+0000","last_became_active":"2024-03-26T08:31:22.385625+0000","last_became_peered":"2024-03-26T08:31:22.385625+0000","last_unstale":"2024-03-26T08:33:06.333414+0000","last_undegraded":"2024-03-26T08:33:06.333414+0000","last_fullsized":"2024-03-26T08:33:06.333414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:16:20.731718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"94'6","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.334408+0000","last_change":"2024-03-26T08:31:22.379588+0000","last_active":"2024-03-26T08:33:06.334408+0000","last_peered":"2024-03-26T08:33:06.334408+0000","last_clean":"2024-03-26T08:33:06.334408+0000","last_became_active":"2024-03-26T08:31:22.379215+0000","last_became_peered":"2024-03-26T08:31:22.379215+0000","last_unstale":"2024-03-26T08:33:06.334408+0000","last_undegraded":"2024-03-26T08:33:06.334408+0000","last_fullsized":"2024-03-26T08:33:06.334408+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:23:20.339737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"94'15","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.345962+0000","last_change":"2024-03-26T08:31:22.404354+0000","last_active":"2024-03-26T08:33:06.345962+0000","last_peered":"2024-03-26T08:33:06.345962+0000","last_clean":"2024-03-26T08:33:06.345962+0000","last_became_active":"2024-03-26T08:31:22.403973+0000","last_became_peered":"2024-03-26T08:31:22.403973+0000","last_unstale":"2024-03-26T08:33:06.345962+0000","last_undegraded":"2024-03-26T08:33:06.345962+0000","last_fullsized":"2024-03-26T08:33:06.345962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:36:54.360147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"94'2","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.318506+0000","last_change":"2024-03-26T08:31:22.379642+0000","last_active":"2024-03-26T08:33:06.318506+0000","last_peered":"2024-03-26T08:33:06.318506+0000","last_clean":"2024-03-26T08:33:06.318506+0000","last_became_active":"2024-03-26T08:31:22.379241+0000","last_became_peered":"2024-03-26T08:31:22.379241+0000","last_unstale":"2024-03-26T08:33:06.318506+0000","last_undegraded":"2024-03-26T08:33:06.318506+0000","last_fullsized":"2024-03-26T08:33:06.318506+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:21:51.117616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"94'6","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.344111+0000","last_change":"2024-03-26T08:31:22.378809+0000","last_active":"2024-03-26T08:33:06.344111+0000","last_peered":"2024-03-26T08:33:06.344111+0000","last_clean":"2024-03-26T08:33:06.344111+0000","last_became_active":"2024-03-26T08:31:22.378388+0000","last_became_peered":"2024-03-26T08:31:22.378388+0000","last_unstale":"2024-03-26T08:33:06.344111+0000","last_undegraded":"2024-03-26T08:33:06.344111+0000","last_fullsized":"2024-03-26T08:33:06.344111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:33:21.583423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"94'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.340907+0000","last_change":"2024-03-26T08:31:22.389230+0000","last_active":"2024-03-26T08:33:06.340907+0000","last_peered":"2024-03-26T08:33:06.340907+0000","last_clean":"2024-03-26T08:33:06.340907+0000","last_became_active":"2024-03-26T08:31:22.388870+0000","last_became_peered":"2024-03-26T08:31:22.388870+0000","last_unstale":"2024-03-26T08:33:06.340907+0000","last_undegraded":"2024-03-26T08:33:06.340907+0000","last_fullsized":"2024-03-26T08:33:06.340907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:40:20.377122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"94'10","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.351060+0000","last_change":"2024-03-26T08:31:22.396232+0000","last_active":"2024-03-26T08:33:06.351060+0000","last_peered":"2024-03-26T08:33:06.351060+0000","last_clean":"2024-03-26T08:33:06.351060+0000","last_became_active":"2024-03-26T08:31:22.392940+0000","last_became_peered":"2024-03-26T08:31:22.392940+0000","last_unstale":"2024-03-26T08:33:06.351060+0000","last_undegraded":"2024-03-26T08:33:06.351060+0000","last_fullsized":"2024-03-26T08:33:06.351060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:01:15.460526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"94'11","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.348113+0000","last_change":"2024-03-26T08:31:22.370880+0000","last_active":"2024-03-26T08:33:06.348113+0000","last_peered":"2024-03-26T08:33:06.348113+0000","last_clean":"2024-03-26T08:33:06.348113+0000","last_became_active":"2024-03-26T08:31:22.370546+0000","last_became_peered":"2024-03-26T08:31:22.370546+0000","last_unstale":"2024-03-26T08:33:06.348113+0000","last_undegraded":"2024-03-26T08:33:06.348113+0000","last_fullsized":"2024-03-26T08:33:06.348113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:46:11.657763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"94'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.341245+0000","last_change":"2024-03-26T08:31:22.387573+0000","last_active":"2024-03-26T08:33:06.341245+0000","last_peered":"2024-03-26T08:33:06.341245+0000","last_clean":"2024-03-26T08:33:06.341245+0000","last_became_active":"2024-03-26T08:31:22.387142+0000","last_became_peered":"2024-03-26T08:31:22.387142+0000","last_unstale":"2024-03-26T08:33:06.341245+0000","last_undegraded":"2024-03-26T08:33:06.341245+0000","last_fullsized":"2024-03-26T08:33:06.341245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:13:32.307195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281284,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63119,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":180,"num_read_kb":155,"num_write":86,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":189357,"data_compressed":5637,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351015,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":521983,"data_compressed":26706,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612654,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32304,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712848,"statfs":{"total":95995035648,"available":95961956352,"internally_reserved":0,"allocated":1155072,"data_stored":1174844,"data_compressed":64362,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776182,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":522543,"data_compressed":26706,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037822,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":522543,"data_compressed":26706,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299460,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":1117007,"data_compressed":62483,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462986,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":517865,"data_compressed":26706,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":580186,"data_compressed":28585,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738483,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":517955,"data_compressed":26706,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411432,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1093632,"data_stored":580220,"data_compressed":28585,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149792,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":517969,"data_compressed":26706,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888152,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":521983,"data_compressed":26706,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724626,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":1112511,"data_compressed":62483,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34133,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34133,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34133,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35777,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35777,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35777,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35777,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35777,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62321,"data_compressed":1879,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62355,"data_compressed":1879,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62515,"data_compressed":1879,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T08:33:17.070 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph pg dump --format=json 2024-03-26T08:33:17.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:16 smithi052 ceph-mon[35071]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:33:17.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:16 smithi043 ceph-mon[31929]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:33:17.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:16 smithi183 ceph-mon[35818]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:33:17.596 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:33:18.003 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:17 smithi043 ceph-mon[31929]: pgmap v434: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T08:33:18.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:17 smithi052 ceph-mon[35071]: pgmap v434: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T08:33:18.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:17 smithi183 ceph-mon[35818]: pgmap v434: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T08:33:20.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.082+0000 7f2487400700 1 -- 172.21.15.43:0/260871678 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c003c50 con 0x7f24881081c0 2024-03-26T08:33:20.074 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:19 smithi043 ceph-mon[31929]: pgmap v435: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-03-26T08:33:20.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.082+0000 7f248fa00700 1 -- 172.21.15.43:0/260871678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24881081c0 msgr2=0x7f24881085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:20.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.082+0000 7f248fa00700 1 --2- 172.21.15.43:0/260871678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24881081c0 0x7f24881085a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f247c0099c0 tx=0x7f247c009cd0 comp rx=0 tx=0).stop 2024-03-26T08:33:20.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.082+0000 7f248fa00700 1 -- 172.21.15.43:0/260871678 shutdown_connections 2024-03-26T08:33:20.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.082+0000 7f248fa00700 1 --2- 172.21.15.43:0/260871678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2488102ba0 0x7f248810efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:20.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.082+0000 7f248fa00700 1 --2- 172.21.15.43:0/260871678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24881081c0 0x7f24881085a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:20.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.082+0000 7f248fa00700 1 --2- 172.21.15.43:0/260871678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24881021e0 0x7f2488102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:20.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.082+0000 7f248fa00700 1 -- 172.21.15.43:0/260871678 >> 172.21.15.43:0/260871678 conn(0x7f24880792c0 msgr2=0x7f24880796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:20.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.083+0000 7f248fa00700 1 -- 172.21.15.43:0/260871678 shutdown_connections 2024-03-26T08:33:20.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.083+0000 7f248fa00700 1 -- 172.21.15.43:0/260871678 wait complete. 2024-03-26T08:33:20.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.084+0000 7f248fa00700 1 Processor -- start 2024-03-26T08:33:20.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.084+0000 7f248fa00700 1 -- start start 2024-03-26T08:33:20.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.085+0000 7f248fa00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24881021e0 0x7f24881a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:20.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.085+0000 7f248fa00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2488102ba0 0x7f24881a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:20.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.085+0000 7f248fa00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24881081c0 0x7f248819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:20.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.085+0000 7f248fa00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2488112390 con 0x7f2488102ba0 2024-03-26T08:33:20.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.085+0000 7f248fa00700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2488112210 con 0x7f24881021e0 2024-03-26T08:33:20.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.085+0000 7f248fa00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2488112510 con 0x7f24881081c0 2024-03-26T08:33:20.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.085+0000 7f2487e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2488102ba0 0x7f24881a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:20.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.085+0000 7f2487e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2488102ba0 0x7f24881a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44536/0 (socket says 172.21.15.43:44536) 2024-03-26T08:33:20.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.085+0000 7f2487e00700 1 -- 172.21.15.43:0/1183571957 learned_addr learned my addr 172.21.15.43:0/1183571957 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:20.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.085+0000 7f248d000700 1 --2- 172.21.15.43:0/1183571957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24881021e0 0x7f24881a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:20.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.085+0000 7f248da00700 1 --2- 172.21.15.43:0/1183571957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24881081c0 0x7f248819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:20.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.086+0000 7f2487e00700 1 -- 172.21.15.43:0/1183571957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24881081c0 msgr2=0x7f248819a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:20.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.086+0000 7f2487e00700 1 --2- 172.21.15.43:0/1183571957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24881081c0 0x7f248819a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:20.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.086+0000 7f2487e00700 1 -- 172.21.15.43:0/1183571957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24881021e0 msgr2=0x7f24881a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:20.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.086+0000 7f2487e00700 1 --2- 172.21.15.43:0/1183571957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24881021e0 0x7f24881a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:20.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.086+0000 7f2487e00700 1 -- 172.21.15.43:0/1183571957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248819aff0 con 0x7f2488102ba0 2024-03-26T08:33:20.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.086+0000 7f2487e00700 1 --2- 172.21.15.43:0/1183571957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2488102ba0 0x7f24881a0b80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f247400c9b0 tx=0x7f247400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:20.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.086+0000 7f2485600700 1 -- 172.21.15.43:0/1183571957 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2474007b00 con 0x7f2488102ba0 2024-03-26T08:33:20.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.086+0000 7f2485600700 1 -- 172.21.15.43:0/1183571957 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2474007ca0 con 0x7f2488102ba0 2024-03-26T08:33:20.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.086+0000 7f248fa00700 1 -- 172.21.15.43:0/1183571957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f248819b2e0 con 0x7f2488102ba0 2024-03-26T08:33:20.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.086+0000 7f2485600700 1 -- 172.21.15.43:0/1183571957 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2474018730 con 0x7f2488102ba0 2024-03-26T08:33:20.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.086+0000 7f248fa00700 1 -- 172.21.15.43:0/1183571957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24881a75c0 con 0x7f2488102ba0 2024-03-26T08:33:20.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.088+0000 7f248fa00700 1 -- 172.21.15.43:0/1183571957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2488103870 con 0x7f2488102ba0 2024-03-26T08:33:20.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.090+0000 7f2485600700 1 -- 172.21.15.43:0/1183571957 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f247400f450 con 0x7f2488102ba0 2024-03-26T08:33:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.091+0000 7f2485600700 1 --2- 172.21.15.43:0/1183571957 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f24700a2890 0x7f24700a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.091+0000 7f2485600700 1 -- 172.21.15.43:0/1183571957 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f24740ceed0 con 0x7f2488102ba0 2024-03-26T08:33:20.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.091+0000 7f248d000700 1 --2- 172.21.15.43:0/1183571957 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f24700a2890 0x7f24700a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:20.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.092+0000 7f248d000700 1 --2- 172.21.15.43:0/1183571957 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f24700a2890 0x7f24700a4d50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f247c009750 tx=0x7f247c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:20.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.095+0000 7f2485600700 1 -- 172.21.15.43:0/1183571957 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2474014030 con 0x7f2488102ba0 2024-03-26T08:33:20.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:19 smithi183 ceph-mon[35818]: pgmap v435: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-03-26T08:33:20.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.320+0000 7f248fa00700 1 -- 172.21.15.43:0/1183571957 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f24881011b0 con 0x7f24700a2890 2024-03-26T08:33:20.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:19 smithi052 ceph-mon[35071]: pgmap v435: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-03-26T08:33:20.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.338+0000 7f2485600700 1 -- 172.21.15.43:0/1183571957 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340715 (secure 0 0 0) 0x7f24881011b0 con 0x7f24700a2890 2024-03-26T08:33:20.332 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:33:20.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.348+0000 7f248fa00700 1 -- 172.21.15.43:0/1183571957 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f24700a2890 msgr2=0x7f24700a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:20.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.348+0000 7f248fa00700 1 --2- 172.21.15.43:0/1183571957 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f24700a2890 0x7f24700a4d50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f247c009750 tx=0x7f247c005fd0 comp rx=0 tx=0).stop 2024-03-26T08:33:20.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.348+0000 7f248fa00700 1 -- 172.21.15.43:0/1183571957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2488102ba0 msgr2=0x7f24881a0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:20.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.348+0000 7f248fa00700 1 --2- 172.21.15.43:0/1183571957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2488102ba0 0x7f24881a0b80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f247400c9b0 tx=0x7f247400cd70 comp rx=0 tx=0).stop 2024-03-26T08:33:20.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.349+0000 7f248fa00700 1 -- 172.21.15.43:0/1183571957 shutdown_connections 2024-03-26T08:33:20.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.349+0000 7f248fa00700 1 --2- 172.21.15.43:0/1183571957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24881081c0 0x7f248819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:20.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.349+0000 7f248fa00700 1 --2- 172.21.15.43:0/1183571957 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f24700a2890 0x7f24700a4d50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:20.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.349+0000 7f248fa00700 1 --2- 172.21.15.43:0/1183571957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24881021e0 0x7f24881a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:20.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.349+0000 7f248fa00700 1 --2- 172.21.15.43:0/1183571957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2488102ba0 0x7f24881a0b80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:20.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.349+0000 7f248fa00700 1 -- 172.21.15.43:0/1183571957 >> 172.21.15.43:0/1183571957 conn(0x7f24880792c0 msgr2=0x7f2488100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:20.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.349+0000 7f248fa00700 1 -- 172.21.15.43:0/1183571957 shutdown_connections 2024-03-26T08:33:20.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:20.349+0000 7f248fa00700 1 -- 172.21.15.43:0/1183571957 wait complete. 2024-03-26T08:33:20.342 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-03-26T08:33:21.187 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":435,"stamp":"2024-03-26T08:33:18.380212+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653721,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":316,"num_read_kb":269,"num_write":269,"num_write_kb":3032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":711,"ondisk_log_size":711,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369864,"kb_used_data":12392,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571960,"statfs":{"total":1151940427776,"available":1151561687040,"internally_reserved":0,"allocated":12689408,"data_stored":8207609,"data_compressed":433440,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":11,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005526"},"pg_stats":[{"pgid":"3.3f","version":"94'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.346209+0000","last_change":"2024-03-26T08:31:22.394256+0000","last_active":"2024-03-26T08:33:06.346209+0000","last_peered":"2024-03-26T08:33:06.346209+0000","last_clean":"2024-03-26T08:33:06.346209+0000","last_became_active":"2024-03-26T08:31:22.393947+0000","last_became_peered":"2024-03-26T08:31:22.393947+0000","last_unstale":"2024-03-26T08:33:06.346209+0000","last_undegraded":"2024-03-26T08:33:06.346209+0000","last_fullsized":"2024-03-26T08:33:06.346209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:00:50.769853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"94'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.335167+0000","last_change":"2024-03-26T08:31:22.387549+0000","last_active":"2024-03-26T08:33:06.335167+0000","last_peered":"2024-03-26T08:33:06.335167+0000","last_clean":"2024-03-26T08:33:06.335167+0000","last_became_active":"2024-03-26T08:31:22.387127+0000","last_became_peered":"2024-03-26T08:31:22.387127+0000","last_unstale":"2024-03-26T08:33:06.335167+0000","last_undegraded":"2024-03-26T08:33:06.335167+0000","last_fullsized":"2024-03-26T08:33:06.335167+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:52:36.156193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.343998+0000","last_change":"2024-03-26T08:31:22.404275+0000","last_active":"2024-03-26T08:33:06.343998+0000","last_peered":"2024-03-26T08:33:06.343998+0000","last_clean":"2024-03-26T08:33:06.343998+0000","last_became_active":"2024-03-26T08:31:22.403909+0000","last_became_peered":"2024-03-26T08:31:22.403909+0000","last_unstale":"2024-03-26T08:33:06.343998+0000","last_undegraded":"2024-03-26T08:33:06.343998+0000","last_fullsized":"2024-03-26T08:33:06.343998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:55:56.295719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"94'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.331852+0000","last_change":"2024-03-26T08:31:22.381310+0000","last_active":"2024-03-26T08:33:06.331852+0000","last_peered":"2024-03-26T08:33:06.331852+0000","last_clean":"2024-03-26T08:33:06.331852+0000","last_became_active":"2024-03-26T08:31:22.380721+0000","last_became_peered":"2024-03-26T08:31:22.380721+0000","last_unstale":"2024-03-26T08:33:06.331852+0000","last_undegraded":"2024-03-26T08:33:06.331852+0000","last_fullsized":"2024-03-26T08:33:06.331852+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:42:30.325409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"94'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.328035+0000","last_change":"2024-03-26T08:31:22.396906+0000","last_active":"2024-03-26T08:33:06.328035+0000","last_peered":"2024-03-26T08:33:06.328035+0000","last_clean":"2024-03-26T08:33:06.328035+0000","last_became_active":"2024-03-26T08:31:22.392815+0000","last_became_peered":"2024-03-26T08:31:22.392815+0000","last_unstale":"2024-03-26T08:33:06.328035+0000","last_undegraded":"2024-03-26T08:33:06.328035+0000","last_fullsized":"2024-03-26T08:33:06.328035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:04:25.761459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"94'6","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.346325+0000","last_change":"2024-03-26T08:31:22.389374+0000","last_active":"2024-03-26T08:33:06.346325+0000","last_peered":"2024-03-26T08:33:06.346325+0000","last_clean":"2024-03-26T08:33:06.346325+0000","last_became_active":"2024-03-26T08:31:22.389012+0000","last_became_peered":"2024-03-26T08:31:22.389012+0000","last_unstale":"2024-03-26T08:33:06.346325+0000","last_undegraded":"2024-03-26T08:33:06.346325+0000","last_fullsized":"2024-03-26T08:33:06.346325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:23:35.696335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"94'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.342833+0000","last_change":"2024-03-26T08:31:22.378805+0000","last_active":"2024-03-26T08:33:06.342833+0000","last_peered":"2024-03-26T08:33:06.342833+0000","last_clean":"2024-03-26T08:33:06.342833+0000","last_became_active":"2024-03-26T08:31:22.378313+0000","last_became_peered":"2024-03-26T08:31:22.378313+0000","last_unstale":"2024-03-26T08:33:06.342833+0000","last_undegraded":"2024-03-26T08:33:06.342833+0000","last_fullsized":"2024-03-26T08:33:06.342833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:10:03.816988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"94'9","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.350604+0000","last_change":"2024-03-26T08:31:37.495699+0000","last_active":"2024-03-26T08:33:06.350604+0000","last_peered":"2024-03-26T08:33:06.350604+0000","last_clean":"2024-03-26T08:33:06.350604+0000","last_became_active":"2024-03-26T08:31:37.494458+0000","last_became_peered":"2024-03-26T08:31:37.494458+0000","last_unstale":"2024-03-26T08:33:06.350604+0000","last_undegraded":"2024-03-26T08:33:06.350604+0000","last_fullsized":"2024-03-26T08:33:06.350604+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:32:36.973898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,10],"acting":[9,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"94'11","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.344545+0000","last_change":"2024-03-26T08:32:38.574794+0000","last_active":"2024-03-26T08:33:06.344545+0000","last_peered":"2024-03-26T08:33:06.344545+0000","last_clean":"2024-03-26T08:33:06.344545+0000","last_became_active":"2024-03-26T08:32:38.574302+0000","last_became_peered":"2024-03-26T08:32:38.574302+0000","last_unstale":"2024-03-26T08:33:06.344545+0000","last_undegraded":"2024-03-26T08:33:06.344545+0000","last_fullsized":"2024-03-26T08:33:06.344545+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:01:01.176434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"94'16","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.346631+0000","last_change":"2024-03-26T08:32:38.313633+0000","last_active":"2024-03-26T08:33:06.346631+0000","last_peered":"2024-03-26T08:33:06.346631+0000","last_clean":"2024-03-26T08:33:06.346631+0000","last_became_active":"2024-03-26T08:32:38.312862+0000","last_became_peered":"2024-03-26T08:32:38.312862+0000","last_unstale":"2024-03-26T08:33:06.346631+0000","last_undegraded":"2024-03-26T08:33:06.346631+0000","last_fullsized":"2024-03-26T08:33:06.346631+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:54:24.226199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.349605+0000","last_change":"2024-03-26T08:31:22.388214+0000","last_active":"2024-03-26T08:33:06.349605+0000","last_peered":"2024-03-26T08:33:06.349605+0000","last_clean":"2024-03-26T08:33:06.349605+0000","last_became_active":"2024-03-26T08:31:22.387760+0000","last_became_peered":"2024-03-26T08:31:22.387760+0000","last_unstale":"2024-03-26T08:33:06.349605+0000","last_undegraded":"2024-03-26T08:33:06.349605+0000","last_fullsized":"2024-03-26T08:33:06.349605+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:25:22.498259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"94'7","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.335720+0000","last_change":"2024-03-26T08:31:22.390892+0000","last_active":"2024-03-26T08:33:06.335720+0000","last_peered":"2024-03-26T08:33:06.335720+0000","last_clean":"2024-03-26T08:33:06.335720+0000","last_became_active":"2024-03-26T08:31:22.390500+0000","last_became_peered":"2024-03-26T08:31:22.390500+0000","last_unstale":"2024-03-26T08:33:06.335720+0000","last_undegraded":"2024-03-26T08:33:06.335720+0000","last_fullsized":"2024-03-26T08:33:06.335720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:29:01.832399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"94'10","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.338385+0000","last_change":"2024-03-26T08:31:22.382442+0000","last_active":"2024-03-26T08:33:06.338385+0000","last_peered":"2024-03-26T08:33:06.338385+0000","last_clean":"2024-03-26T08:33:06.338385+0000","last_became_active":"2024-03-26T08:31:22.380346+0000","last_became_peered":"2024-03-26T08:31:22.380346+0000","last_unstale":"2024-03-26T08:33:06.338385+0000","last_undegraded":"2024-03-26T08:33:06.338385+0000","last_fullsized":"2024-03-26T08:33:06.338385+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:36:36.451504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.334236+0000","last_change":"2024-03-26T08:31:22.387401+0000","last_active":"2024-03-26T08:33:06.334236+0000","last_peered":"2024-03-26T08:33:06.334236+0000","last_clean":"2024-03-26T08:33:06.334236+0000","last_became_active":"2024-03-26T08:31:22.387020+0000","last_became_peered":"2024-03-26T08:31:22.387020+0000","last_unstale":"2024-03-26T08:33:06.334236+0000","last_undegraded":"2024-03-26T08:33:06.334236+0000","last_fullsized":"2024-03-26T08:33:06.334236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:50:15.073663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.342429+0000","last_change":"2024-03-26T08:31:22.397971+0000","last_active":"2024-03-26T08:33:06.342429+0000","last_peered":"2024-03-26T08:33:06.342429+0000","last_clean":"2024-03-26T08:33:06.342429+0000","last_became_active":"2024-03-26T08:31:22.395371+0000","last_became_peered":"2024-03-26T08:31:22.395371+0000","last_unstale":"2024-03-26T08:33:06.342429+0000","last_undegraded":"2024-03-26T08:33:06.342429+0000","last_fullsized":"2024-03-26T08:33:06.342429+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:59:51.053712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"94'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.348921+0000","last_change":"2024-03-26T08:31:22.394437+0000","last_active":"2024-03-26T08:33:06.348921+0000","last_peered":"2024-03-26T08:33:06.348921+0000","last_clean":"2024-03-26T08:33:06.348921+0000","last_became_active":"2024-03-26T08:31:22.386794+0000","last_became_peered":"2024-03-26T08:31:22.386794+0000","last_unstale":"2024-03-26T08:33:06.348921+0000","last_undegraded":"2024-03-26T08:33:06.348921+0000","last_fullsized":"2024-03-26T08:33:06.348921+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:56:01.686361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"94'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.346277+0000","last_change":"2024-03-26T08:31:22.397887+0000","last_active":"2024-03-26T08:33:06.346277+0000","last_peered":"2024-03-26T08:33:06.346277+0000","last_clean":"2024-03-26T08:33:06.346277+0000","last_became_active":"2024-03-26T08:31:22.397086+0000","last_became_peered":"2024-03-26T08:31:22.397086+0000","last_unstale":"2024-03-26T08:33:06.346277+0000","last_undegraded":"2024-03-26T08:33:06.346277+0000","last_fullsized":"2024-03-26T08:33:06.346277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:41:47.512904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"94'10","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.344271+0000","last_change":"2024-03-26T08:31:22.398868+0000","last_active":"2024-03-26T08:33:06.344271+0000","last_peered":"2024-03-26T08:33:06.344271+0000","last_clean":"2024-03-26T08:33:06.344271+0000","last_became_active":"2024-03-26T08:31:22.398536+0000","last_became_peered":"2024-03-26T08:31:22.398536+0000","last_unstale":"2024-03-26T08:33:06.344271+0000","last_undegraded":"2024-03-26T08:33:06.344271+0000","last_fullsized":"2024-03-26T08:33:06.344271+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:59:07.931120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"94'7","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.328913+0000","last_change":"2024-03-26T08:31:22.389805+0000","last_active":"2024-03-26T08:33:06.328913+0000","last_peered":"2024-03-26T08:33:06.328913+0000","last_clean":"2024-03-26T08:33:06.328913+0000","last_became_active":"2024-03-26T08:31:22.389410+0000","last_became_peered":"2024-03-26T08:31:22.389410+0000","last_unstale":"2024-03-26T08:33:06.328913+0000","last_undegraded":"2024-03-26T08:33:06.328913+0000","last_fullsized":"2024-03-26T08:33:06.328913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:36:29.934029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.310334+0000","last_change":"2024-03-26T08:31:26.437909+0000","last_active":"2024-03-26T08:32:38.310334+0000","last_peered":"2024-03-26T08:32:38.310334+0000","last_clean":"2024-03-26T08:32:38.310334+0000","last_became_active":"2024-03-26T08:31:26.437337+0000","last_became_peered":"2024-03-26T08:31:26.437337+0000","last_unstale":"2024-03-26T08:32:38.310334+0000","last_undegraded":"2024-03-26T08:32:38.310334+0000","last_fullsized":"2024-03-26T08:32:38.310334+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:01:15.856194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.273628+0000","last_change":"2024-03-26T08:31:21.367133+0000","last_active":"2024-03-26T08:32:38.273628+0000","last_peered":"2024-03-26T08:32:38.273628+0000","last_clean":"2024-03-26T08:32:38.273628+0000","last_became_active":"2024-03-26T08:31:21.366206+0000","last_became_peered":"2024-03-26T08:31:21.366206+0000","last_unstale":"2024-03-26T08:32:38.273628+0000","last_undegraded":"2024-03-26T08:32:38.273628+0000","last_fullsized":"2024-03-26T08:32:38.273628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:11:16.614464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"94'5","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.329519+0000","last_change":"2024-03-26T08:31:22.385984+0000","last_active":"2024-03-26T08:33:06.329519+0000","last_peered":"2024-03-26T08:33:06.329519+0000","last_clean":"2024-03-26T08:33:06.329519+0000","last_became_active":"2024-03-26T08:31:22.385512+0000","last_became_peered":"2024-03-26T08:31:22.385512+0000","last_unstale":"2024-03-26T08:33:06.329519+0000","last_undegraded":"2024-03-26T08:33:06.329519+0000","last_fullsized":"2024-03-26T08:33:06.329519+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:46:24.087975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.267156+0000","last_change":"2024-03-26T08:31:26.454001+0000","last_active":"2024-03-26T08:32:38.267156+0000","last_peered":"2024-03-26T08:32:38.267156+0000","last_clean":"2024-03-26T08:32:38.267156+0000","last_became_active":"2024-03-26T08:31:26.453474+0000","last_became_peered":"2024-03-26T08:31:26.453474+0000","last_unstale":"2024-03-26T08:32:38.267156+0000","last_undegraded":"2024-03-26T08:32:38.267156+0000","last_fullsized":"2024-03-26T08:32:38.267156+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:09:42.197500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.267437+0000","last_change":"2024-03-26T08:31:21.365012+0000","last_active":"2024-03-26T08:32:38.267437+0000","last_peered":"2024-03-26T08:32:38.267437+0000","last_clean":"2024-03-26T08:32:38.267437+0000","last_became_active":"2024-03-26T08:31:21.363981+0000","last_became_peered":"2024-03-26T08:31:21.363981+0000","last_unstale":"2024-03-26T08:32:38.267437+0000","last_undegraded":"2024-03-26T08:32:38.267437+0000","last_fullsized":"2024-03-26T08:32:38.267437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:12:36.581648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.346350+0000","last_change":"2024-03-26T08:31:22.387960+0000","last_active":"2024-03-26T08:33:06.346350+0000","last_peered":"2024-03-26T08:33:06.346350+0000","last_clean":"2024-03-26T08:33:06.346350+0000","last_became_active":"2024-03-26T08:31:22.387627+0000","last_became_peered":"2024-03-26T08:31:22.387627+0000","last_unstale":"2024-03-26T08:33:06.346350+0000","last_undegraded":"2024-03-26T08:33:06.346350+0000","last_fullsized":"2024-03-26T08:33:06.346350+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:20:09.265075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.269421+0000","last_change":"2024-03-26T08:31:26.450387+0000","last_active":"2024-03-26T08:32:38.269421+0000","last_peered":"2024-03-26T08:32:38.269421+0000","last_clean":"2024-03-26T08:32:38.269421+0000","last_became_active":"2024-03-26T08:31:26.449418+0000","last_became_peered":"2024-03-26T08:31:26.449418+0000","last_unstale":"2024-03-26T08:32:38.269421+0000","last_undegraded":"2024-03-26T08:32:38.269421+0000","last_fullsized":"2024-03-26T08:32:38.269421+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:51:59.022851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.268640+0000","last_change":"2024-03-26T08:31:21.373422+0000","last_active":"2024-03-26T08:32:38.268640+0000","last_peered":"2024-03-26T08:32:38.268640+0000","last_clean":"2024-03-26T08:32:38.268640+0000","last_became_active":"2024-03-26T08:31:21.372916+0000","last_became_peered":"2024-03-26T08:31:21.372916+0000","last_unstale":"2024-03-26T08:32:38.268640+0000","last_undegraded":"2024-03-26T08:32:38.268640+0000","last_fullsized":"2024-03-26T08:32:38.268640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:09:08.313365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"94'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.338926+0000","last_change":"2024-03-26T08:31:22.403385+0000","last_active":"2024-03-26T08:33:06.338926+0000","last_peered":"2024-03-26T08:33:06.338926+0000","last_clean":"2024-03-26T08:33:06.338926+0000","last_became_active":"2024-03-26T08:31:22.398021+0000","last_became_peered":"2024-03-26T08:31:22.398021+0000","last_unstale":"2024-03-26T08:33:06.338926+0000","last_undegraded":"2024-03-26T08:33:06.338926+0000","last_fullsized":"2024-03-26T08:33:06.338926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:56:06.691167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.267106+0000","last_change":"2024-03-26T08:31:26.449049+0000","last_active":"2024-03-26T08:32:38.267106+0000","last_peered":"2024-03-26T08:32:38.267106+0000","last_clean":"2024-03-26T08:32:38.267106+0000","last_became_active":"2024-03-26T08:31:26.448510+0000","last_became_peered":"2024-03-26T08:31:26.448510+0000","last_unstale":"2024-03-26T08:32:38.267106+0000","last_undegraded":"2024-03-26T08:32:38.267106+0000","last_fullsized":"2024-03-26T08:32:38.267106+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:24:58.699699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.267778+0000","last_change":"2024-03-26T08:31:21.367289+0000","last_active":"2024-03-26T08:32:38.267778+0000","last_peered":"2024-03-26T08:32:38.267778+0000","last_clean":"2024-03-26T08:32:38.267778+0000","last_became_active":"2024-03-26T08:31:21.366960+0000","last_became_peered":"2024-03-26T08:31:21.366960+0000","last_unstale":"2024-03-26T08:32:38.267778+0000","last_undegraded":"2024-03-26T08:32:38.267778+0000","last_fullsized":"2024-03-26T08:32:38.267778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:22:25.479383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"94'15","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.346079+0000","last_change":"2024-03-26T08:31:22.385815+0000","last_active":"2024-03-26T08:33:06.346079+0000","last_peered":"2024-03-26T08:33:06.346079+0000","last_clean":"2024-03-26T08:33:06.346079+0000","last_became_active":"2024-03-26T08:31:22.385503+0000","last_became_peered":"2024-03-26T08:31:22.385503+0000","last_unstale":"2024-03-26T08:33:06.346079+0000","last_undegraded":"2024-03-26T08:33:06.346079+0000","last_fullsized":"2024-03-26T08:33:06.346079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:33:35.768902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.311320+0000","last_change":"2024-03-26T08:31:26.449656+0000","last_active":"2024-03-26T08:32:38.311320+0000","last_peered":"2024-03-26T08:32:38.311320+0000","last_clean":"2024-03-26T08:32:38.311320+0000","last_became_active":"2024-03-26T08:31:26.448843+0000","last_became_peered":"2024-03-26T08:31:26.448843+0000","last_unstale":"2024-03-26T08:32:38.311320+0000","last_undegraded":"2024-03-26T08:32:38.311320+0000","last_fullsized":"2024-03-26T08:32:38.311320+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:11:28.135551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.456071+0000","last_change":"2024-03-26T08:31:21.373384+0000","last_active":"2024-03-26T08:32:38.456071+0000","last_peered":"2024-03-26T08:32:38.456071+0000","last_clean":"2024-03-26T08:32:38.456071+0000","last_became_active":"2024-03-26T08:31:21.371708+0000","last_became_peered":"2024-03-26T08:31:21.371708+0000","last_unstale":"2024-03-26T08:32:38.456071+0000","last_undegraded":"2024-03-26T08:32:38.456071+0000","last_fullsized":"2024-03-26T08:32:38.456071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:46:09.745738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.342417+0000","last_change":"2024-03-26T08:31:22.383990+0000","last_active":"2024-03-26T08:33:06.342417+0000","last_peered":"2024-03-26T08:33:06.342417+0000","last_clean":"2024-03-26T08:33:06.342417+0000","last_became_active":"2024-03-26T08:31:22.383580+0000","last_became_peered":"2024-03-26T08:31:22.383580+0000","last_unstale":"2024-03-26T08:33:06.342417+0000","last_undegraded":"2024-03-26T08:33:06.342417+0000","last_fullsized":"2024-03-26T08:33:06.342417+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:32:36.037407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.311026+0000","last_change":"2024-03-26T08:31:26.445710+0000","last_active":"2024-03-26T08:32:38.311026+0000","last_peered":"2024-03-26T08:32:38.311026+0000","last_clean":"2024-03-26T08:32:38.311026+0000","last_became_active":"2024-03-26T08:31:26.445189+0000","last_became_peered":"2024-03-26T08:31:26.445189+0000","last_unstale":"2024-03-26T08:32:38.311026+0000","last_undegraded":"2024-03-26T08:32:38.311026+0000","last_fullsized":"2024-03-26T08:32:38.311026+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:31:23.033192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.272979+0000","last_change":"2024-03-26T08:31:21.362175+0000","last_active":"2024-03-26T08:32:38.272979+0000","last_peered":"2024-03-26T08:32:38.272979+0000","last_clean":"2024-03-26T08:32:38.272979+0000","last_became_active":"2024-03-26T08:31:21.361805+0000","last_became_peered":"2024-03-26T08:31:21.361805+0000","last_unstale":"2024-03-26T08:32:38.272979+0000","last_undegraded":"2024-03-26T08:32:38.272979+0000","last_fullsized":"2024-03-26T08:32:38.272979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:38:26.537955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"94'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.338999+0000","last_change":"2024-03-26T08:31:22.380489+0000","last_active":"2024-03-26T08:33:06.338999+0000","last_peered":"2024-03-26T08:33:06.338999+0000","last_clean":"2024-03-26T08:33:06.338999+0000","last_became_active":"2024-03-26T08:31:22.380139+0000","last_became_peered":"2024-03-26T08:31:22.380139+0000","last_unstale":"2024-03-26T08:33:06.338999+0000","last_undegraded":"2024-03-26T08:33:06.338999+0000","last_fullsized":"2024-03-26T08:33:06.338999+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:09:18.550790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.572421+0000","last_change":"2024-03-26T08:31:26.454535+0000","last_active":"2024-03-26T08:32:38.572421+0000","last_peered":"2024-03-26T08:32:38.572421+0000","last_clean":"2024-03-26T08:32:38.572421+0000","last_became_active":"2024-03-26T08:31:26.454106+0000","last_became_peered":"2024-03-26T08:31:26.454106+0000","last_unstale":"2024-03-26T08:32:38.572421+0000","last_undegraded":"2024-03-26T08:32:38.572421+0000","last_fullsized":"2024-03-26T08:32:38.572421+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:57:28.870653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.571042+0000","last_change":"2024-03-26T08:31:21.347236+0000","last_active":"2024-03-26T08:32:38.571042+0000","last_peered":"2024-03-26T08:32:38.571042+0000","last_clean":"2024-03-26T08:32:38.571042+0000","last_became_active":"2024-03-26T08:31:21.346770+0000","last_became_peered":"2024-03-26T08:31:21.346770+0000","last_unstale":"2024-03-26T08:32:38.571042+0000","last_undegraded":"2024-03-26T08:32:38.571042+0000","last_fullsized":"2024-03-26T08:32:38.571042+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:17:22.927636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"94'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.336185+0000","last_change":"2024-03-26T08:31:22.390623+0000","last_active":"2024-03-26T08:33:06.336185+0000","last_peered":"2024-03-26T08:33:06.336185+0000","last_clean":"2024-03-26T08:33:06.336185+0000","last_became_active":"2024-03-26T08:31:22.389512+0000","last_became_peered":"2024-03-26T08:31:22.389512+0000","last_unstale":"2024-03-26T08:33:06.336185+0000","last_undegraded":"2024-03-26T08:33:06.336185+0000","last_fullsized":"2024-03-26T08:33:06.336185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:04:31.317386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.572594+0000","last_change":"2024-03-26T08:31:26.445277+0000","last_active":"2024-03-26T08:32:38.572594+0000","last_peered":"2024-03-26T08:32:38.572594+0000","last_clean":"2024-03-26T08:32:38.572594+0000","last_became_active":"2024-03-26T08:31:26.443558+0000","last_became_peered":"2024-03-26T08:31:26.443558+0000","last_unstale":"2024-03-26T08:32:38.572594+0000","last_undegraded":"2024-03-26T08:32:38.572594+0000","last_fullsized":"2024-03-26T08:32:38.572594+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:20:16.688699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.454859+0000","last_change":"2024-03-26T08:31:21.371523+0000","last_active":"2024-03-26T08:32:38.454859+0000","last_peered":"2024-03-26T08:32:38.454859+0000","last_clean":"2024-03-26T08:32:38.454859+0000","last_became_active":"2024-03-26T08:31:21.371136+0000","last_became_peered":"2024-03-26T08:31:21.371136+0000","last_unstale":"2024-03-26T08:32:38.454859+0000","last_undegraded":"2024-03-26T08:32:38.454859+0000","last_fullsized":"2024-03-26T08:32:38.454859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:06:55.083579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"94'11","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.347390+0000","last_change":"2024-03-26T08:31:22.386237+0000","last_active":"2024-03-26T08:33:06.347390+0000","last_peered":"2024-03-26T08:33:06.347390+0000","last_clean":"2024-03-26T08:33:06.347390+0000","last_became_active":"2024-03-26T08:31:22.385874+0000","last_became_peered":"2024-03-26T08:31:22.385874+0000","last_unstale":"2024-03-26T08:33:06.347390+0000","last_undegraded":"2024-03-26T08:33:06.347390+0000","last_fullsized":"2024-03-26T08:33:06.347390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:14:58.278611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.265815+0000","last_change":"2024-03-26T08:31:26.452464+0000","last_active":"2024-03-26T08:32:38.265815+0000","last_peered":"2024-03-26T08:32:38.265815+0000","last_clean":"2024-03-26T08:32:38.265815+0000","last_became_active":"2024-03-26T08:31:26.451862+0000","last_became_peered":"2024-03-26T08:31:26.451862+0000","last_unstale":"2024-03-26T08:32:38.265815+0000","last_undegraded":"2024-03-26T08:32:38.265815+0000","last_fullsized":"2024-03-26T08:32:38.265815+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:48:29.616247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.274651+0000","last_change":"2024-03-26T08:31:21.365283+0000","last_active":"2024-03-26T08:32:38.274651+0000","last_peered":"2024-03-26T08:32:38.274651+0000","last_clean":"2024-03-26T08:32:38.274651+0000","last_became_active":"2024-03-26T08:31:21.364902+0000","last_became_peered":"2024-03-26T08:31:21.364902+0000","last_unstale":"2024-03-26T08:32:38.274651+0000","last_undegraded":"2024-03-26T08:32:38.274651+0000","last_fullsized":"2024-03-26T08:32:38.274651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:55:39.394355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"94'10","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.344341+0000","last_change":"2024-03-26T08:31:22.391645+0000","last_active":"2024-03-26T08:33:06.344341+0000","last_peered":"2024-03-26T08:33:06.344341+0000","last_clean":"2024-03-26T08:33:06.344341+0000","last_became_active":"2024-03-26T08:31:22.391280+0000","last_became_peered":"2024-03-26T08:31:22.391280+0000","last_unstale":"2024-03-26T08:33:06.344341+0000","last_undegraded":"2024-03-26T08:33:06.344341+0000","last_fullsized":"2024-03-26T08:33:06.344341+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:31:48.963287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.272411+0000","last_change":"2024-03-26T08:31:26.446215+0000","last_active":"2024-03-26T08:32:38.272411+0000","last_peered":"2024-03-26T08:32:38.272411+0000","last_clean":"2024-03-26T08:32:38.272411+0000","last_became_active":"2024-03-26T08:31:26.445765+0000","last_became_peered":"2024-03-26T08:31:26.445765+0000","last_unstale":"2024-03-26T08:32:38.272411+0000","last_undegraded":"2024-03-26T08:32:38.272411+0000","last_fullsized":"2024-03-26T08:32:38.272411+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:57:14.809610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.572547+0000","last_change":"2024-03-26T08:31:21.373345+0000","last_active":"2024-03-26T08:32:38.572547+0000","last_peered":"2024-03-26T08:32:38.572547+0000","last_clean":"2024-03-26T08:32:38.572547+0000","last_became_active":"2024-03-26T08:31:21.372906+0000","last_became_peered":"2024-03-26T08:31:21.372906+0000","last_unstale":"2024-03-26T08:32:38.572547+0000","last_undegraded":"2024-03-26T08:32:38.572547+0000","last_fullsized":"2024-03-26T08:32:38.572547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:20:00.518815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.339863+0000","last_change":"2024-03-26T08:31:22.393433+0000","last_active":"2024-03-26T08:33:06.339863+0000","last_peered":"2024-03-26T08:33:06.339863+0000","last_clean":"2024-03-26T08:33:06.339863+0000","last_became_active":"2024-03-26T08:31:22.393100+0000","last_became_peered":"2024-03-26T08:31:22.393100+0000","last_unstale":"2024-03-26T08:33:06.339863+0000","last_undegraded":"2024-03-26T08:33:06.339863+0000","last_fullsized":"2024-03-26T08:33:06.339863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:33:30.418492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.268506+0000","last_change":"2024-03-26T08:31:26.449975+0000","last_active":"2024-03-26T08:32:38.268506+0000","last_peered":"2024-03-26T08:32:38.268506+0000","last_clean":"2024-03-26T08:32:38.268506+0000","last_became_active":"2024-03-26T08:31:26.448345+0000","last_became_peered":"2024-03-26T08:31:26.448345+0000","last_unstale":"2024-03-26T08:32:38.268506+0000","last_undegraded":"2024-03-26T08:32:38.268506+0000","last_fullsized":"2024-03-26T08:32:38.268506+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:01:24.765939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.457882+0000","last_change":"2024-03-26T08:31:21.364195+0000","last_active":"2024-03-26T08:32:38.457882+0000","last_peered":"2024-03-26T08:32:38.457882+0000","last_clean":"2024-03-26T08:32:38.457882+0000","last_became_active":"2024-03-26T08:31:21.363887+0000","last_became_peered":"2024-03-26T08:31:21.363887+0000","last_unstale":"2024-03-26T08:32:38.457882+0000","last_undegraded":"2024-03-26T08:32:38.457882+0000","last_fullsized":"2024-03-26T08:32:38.457882+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:03:03.015581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"94'11","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.339076+0000","last_change":"2024-03-26T08:31:37.493245+0000","last_active":"2024-03-26T08:33:06.339076+0000","last_peered":"2024-03-26T08:33:06.339076+0000","last_clean":"2024-03-26T08:33:06.339076+0000","last_became_active":"2024-03-26T08:31:37.492509+0000","last_became_peered":"2024-03-26T08:31:37.492509+0000","last_unstale":"2024-03-26T08:33:06.339076+0000","last_undegraded":"2024-03-26T08:33:06.339076+0000","last_fullsized":"2024-03-26T08:33:06.339076+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:28:56.355010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.272450+0000","last_change":"2024-03-26T08:31:26.449405+0000","last_active":"2024-03-26T08:32:38.272450+0000","last_peered":"2024-03-26T08:32:38.272450+0000","last_clean":"2024-03-26T08:32:38.272450+0000","last_became_active":"2024-03-26T08:31:26.448934+0000","last_became_peered":"2024-03-26T08:31:26.448934+0000","last_unstale":"2024-03-26T08:32:38.272450+0000","last_undegraded":"2024-03-26T08:32:38.272450+0000","last_fullsized":"2024-03-26T08:32:38.272450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:12:22.797816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'31","reported_seq":245,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:14.853112+0000","last_change":"2024-03-26T08:31:21.372300+0000","last_active":"2024-03-26T08:33:14.853112+0000","last_peered":"2024-03-26T08:33:14.853112+0000","last_clean":"2024-03-26T08:33:14.853112+0000","last_became_active":"2024-03-26T08:31:21.370103+0000","last_became_peered":"2024-03-26T08:31:21.370103+0000","last_unstale":"2024-03-26T08:33:14.853112+0000","last_undegraded":"2024-03-26T08:33:14.853112+0000","last_fullsized":"2024-03-26T08:33:14.853112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:54:52.657768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62321,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":184,"num_read_kb":154,"num_write":31,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.348143+0000","last_change":"2024-03-26T08:31:22.373102+0000","last_active":"2024-03-26T08:33:06.348143+0000","last_peered":"2024-03-26T08:33:06.348143+0000","last_clean":"2024-03-26T08:33:06.348143+0000","last_became_active":"2024-03-26T08:31:22.372792+0000","last_became_peered":"2024-03-26T08:31:22.372792+0000","last_unstale":"2024-03-26T08:33:06.348143+0000","last_undegraded":"2024-03-26T08:33:06.348143+0000","last_fullsized":"2024-03-26T08:33:06.348143+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:58:58.274941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.454102+0000","last_change":"2024-03-26T08:31:26.448172+0000","last_active":"2024-03-26T08:32:38.454102+0000","last_peered":"2024-03-26T08:32:38.454102+0000","last_clean":"2024-03-26T08:32:38.454102+0000","last_became_active":"2024-03-26T08:31:26.447550+0000","last_became_peered":"2024-03-26T08:31:26.447550+0000","last_unstale":"2024-03-26T08:32:38.454102+0000","last_undegraded":"2024-03-26T08:32:38.454102+0000","last_fullsized":"2024-03-26T08:32:38.454102+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:56:19.806076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.310789+0000","last_change":"2024-03-26T08:31:21.363946+0000","last_active":"2024-03-26T08:32:38.310789+0000","last_peered":"2024-03-26T08:32:38.310789+0000","last_clean":"2024-03-26T08:32:38.310789+0000","last_became_active":"2024-03-26T08:31:21.363566+0000","last_became_peered":"2024-03-26T08:31:21.363566+0000","last_unstale":"2024-03-26T08:32:38.310789+0000","last_undegraded":"2024-03-26T08:32:38.310789+0000","last_fullsized":"2024-03-26T08:32:38.310789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:32:51.714454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"94'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.349700+0000","last_change":"2024-03-26T08:31:22.375813+0000","last_active":"2024-03-26T08:33:06.349700+0000","last_peered":"2024-03-26T08:33:06.349700+0000","last_clean":"2024-03-26T08:33:06.349700+0000","last_became_active":"2024-03-26T08:31:22.375437+0000","last_became_peered":"2024-03-26T08:31:22.375437+0000","last_unstale":"2024-03-26T08:33:06.349700+0000","last_undegraded":"2024-03-26T08:33:06.349700+0000","last_fullsized":"2024-03-26T08:33:06.349700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:48:25.167143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.571501+0000","last_change":"2024-03-26T08:31:26.452079+0000","last_active":"2024-03-26T08:32:38.571501+0000","last_peered":"2024-03-26T08:32:38.571501+0000","last_clean":"2024-03-26T08:32:38.571501+0000","last_became_active":"2024-03-26T08:31:26.451146+0000","last_became_peered":"2024-03-26T08:31:26.451146+0000","last_unstale":"2024-03-26T08:32:38.571501+0000","last_undegraded":"2024-03-26T08:32:38.571501+0000","last_fullsized":"2024-03-26T08:32:38.571501+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:08:57.803180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.456006+0000","last_change":"2024-03-26T08:31:21.371222+0000","last_active":"2024-03-26T08:32:38.456006+0000","last_peered":"2024-03-26T08:32:38.456006+0000","last_clean":"2024-03-26T08:32:38.456006+0000","last_became_active":"2024-03-26T08:31:21.370843+0000","last_became_peered":"2024-03-26T08:31:21.370843+0000","last_unstale":"2024-03-26T08:32:38.456006+0000","last_undegraded":"2024-03-26T08:32:38.456006+0000","last_fullsized":"2024-03-26T08:32:38.456006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:32:00.518713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"94'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.350801+0000","last_change":"2024-03-26T08:31:22.379781+0000","last_active":"2024-03-26T08:33:06.350801+0000","last_peered":"2024-03-26T08:33:06.350801+0000","last_clean":"2024-03-26T08:33:06.350801+0000","last_became_active":"2024-03-26T08:31:22.379357+0000","last_became_peered":"2024-03-26T08:31:22.379357+0000","last_unstale":"2024-03-26T08:33:06.350801+0000","last_undegraded":"2024-03-26T08:33:06.350801+0000","last_fullsized":"2024-03-26T08:33:06.350801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:21:52.306343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.267928+0000","last_change":"2024-03-26T08:31:37.498490+0000","last_active":"2024-03-26T08:32:38.267928+0000","last_peered":"2024-03-26T08:32:38.267928+0000","last_clean":"2024-03-26T08:32:38.267928+0000","last_became_active":"2024-03-26T08:31:37.496608+0000","last_became_peered":"2024-03-26T08:31:37.496608+0000","last_unstale":"2024-03-26T08:32:38.267928+0000","last_undegraded":"2024-03-26T08:32:38.267928+0000","last_fullsized":"2024-03-26T08:32:38.267928+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:32:11.653087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.267117+0000","last_change":"2024-03-26T08:31:21.366945+0000","last_active":"2024-03-26T08:32:38.267117+0000","last_peered":"2024-03-26T08:32:38.267117+0000","last_clean":"2024-03-26T08:32:38.267117+0000","last_became_active":"2024-03-26T08:31:21.366548+0000","last_became_peered":"2024-03-26T08:31:21.366548+0000","last_unstale":"2024-03-26T08:32:38.267117+0000","last_undegraded":"2024-03-26T08:32:38.267117+0000","last_fullsized":"2024-03-26T08:32:38.267117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:13:52.037163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.347687+0000","last_change":"2024-03-26T08:31:22.387575+0000","last_active":"2024-03-26T08:33:06.347687+0000","last_peered":"2024-03-26T08:33:06.347687+0000","last_clean":"2024-03-26T08:33:06.347687+0000","last_became_active":"2024-03-26T08:31:22.383815+0000","last_became_peered":"2024-03-26T08:31:22.383815+0000","last_unstale":"2024-03-26T08:33:06.347687+0000","last_undegraded":"2024-03-26T08:33:06.347687+0000","last_fullsized":"2024-03-26T08:33:06.347687+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:10:20.318597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.570150+0000","last_change":"2024-03-26T08:31:26.440745+0000","last_active":"2024-03-26T08:32:38.570150+0000","last_peered":"2024-03-26T08:32:38.570150+0000","last_clean":"2024-03-26T08:32:38.570150+0000","last_became_active":"2024-03-26T08:31:26.440199+0000","last_became_peered":"2024-03-26T08:31:26.440199+0000","last_unstale":"2024-03-26T08:32:38.570150+0000","last_undegraded":"2024-03-26T08:32:38.570150+0000","last_fullsized":"2024-03-26T08:32:38.570150+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:25:11.407274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.266699+0000","last_change":"2024-03-26T08:31:21.369227+0000","last_active":"2024-03-26T08:32:38.266699+0000","last_peered":"2024-03-26T08:32:38.266699+0000","last_clean":"2024-03-26T08:32:38.266699+0000","last_became_active":"2024-03-26T08:31:21.368709+0000","last_became_peered":"2024-03-26T08:31:21.368709+0000","last_unstale":"2024-03-26T08:32:38.266699+0000","last_undegraded":"2024-03-26T08:32:38.266699+0000","last_fullsized":"2024-03-26T08:32:38.266699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:30:04.714715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"94'6","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.330464+0000","last_change":"2024-03-26T08:31:22.384199+0000","last_active":"2024-03-26T08:33:06.330464+0000","last_peered":"2024-03-26T08:33:06.330464+0000","last_clean":"2024-03-26T08:33:06.330464+0000","last_became_active":"2024-03-26T08:31:22.383854+0000","last_became_peered":"2024-03-26T08:31:22.383854+0000","last_unstale":"2024-03-26T08:33:06.330464+0000","last_undegraded":"2024-03-26T08:33:06.330464+0000","last_fullsized":"2024-03-26T08:33:06.330464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:33:24.240505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.453908+0000","last_change":"2024-03-26T08:31:26.447657+0000","last_active":"2024-03-26T08:32:38.453908+0000","last_peered":"2024-03-26T08:32:38.453908+0000","last_clean":"2024-03-26T08:32:38.453908+0000","last_became_active":"2024-03-26T08:31:26.444812+0000","last_became_peered":"2024-03-26T08:31:26.444812+0000","last_unstale":"2024-03-26T08:32:38.453908+0000","last_undegraded":"2024-03-26T08:32:38.453908+0000","last_fullsized":"2024-03-26T08:32:38.453908+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:58:21.422318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.574367+0000","last_change":"2024-03-26T08:31:37.510665+0000","last_active":"2024-03-26T08:32:38.574367+0000","last_peered":"2024-03-26T08:32:38.574367+0000","last_clean":"2024-03-26T08:32:38.574367+0000","last_became_active":"2024-03-26T08:31:37.504109+0000","last_became_peered":"2024-03-26T08:31:37.504109+0000","last_unstale":"2024-03-26T08:32:38.574367+0000","last_undegraded":"2024-03-26T08:32:38.574367+0000","last_fullsized":"2024-03-26T08:32:38.574367+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:38:21.446881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"94'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.331161+0000","last_change":"2024-03-26T08:31:37.502088+0000","last_active":"2024-03-26T08:33:06.331161+0000","last_peered":"2024-03-26T08:33:06.331161+0000","last_clean":"2024-03-26T08:33:06.331161+0000","last_became_active":"2024-03-26T08:31:37.499767+0000","last_became_peered":"2024-03-26T08:31:37.499767+0000","last_unstale":"2024-03-26T08:33:06.331161+0000","last_undegraded":"2024-03-26T08:33:06.331161+0000","last_fullsized":"2024-03-26T08:33:06.331161+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:55:32.652231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.267066+0000","last_change":"2024-03-26T08:31:26.479654+0000","last_active":"2024-03-26T08:32:38.267066+0000","last_peered":"2024-03-26T08:32:38.267066+0000","last_clean":"2024-03-26T08:32:38.267066+0000","last_became_active":"2024-03-26T08:31:26.478948+0000","last_became_peered":"2024-03-26T08:31:26.478948+0000","last_unstale":"2024-03-26T08:32:38.267066+0000","last_undegraded":"2024-03-26T08:32:38.267066+0000","last_fullsized":"2024-03-26T08:32:38.267066+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:33:58.445531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.272835+0000","last_change":"2024-03-26T08:31:21.365203+0000","last_active":"2024-03-26T08:32:38.272835+0000","last_peered":"2024-03-26T08:32:38.272835+0000","last_clean":"2024-03-26T08:32:38.272835+0000","last_became_active":"2024-03-26T08:31:21.364816+0000","last_became_peered":"2024-03-26T08:31:21.364816+0000","last_unstale":"2024-03-26T08:32:38.272835+0000","last_undegraded":"2024-03-26T08:32:38.272835+0000","last_fullsized":"2024-03-26T08:32:38.272835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:59:50.157011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"94'4","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.341384+0000","last_change":"2024-03-26T08:31:22.392014+0000","last_active":"2024-03-26T08:33:06.341384+0000","last_peered":"2024-03-26T08:33:06.341384+0000","last_clean":"2024-03-26T08:33:06.341384+0000","last_became_active":"2024-03-26T08:31:22.391627+0000","last_became_peered":"2024-03-26T08:31:22.391627+0000","last_unstale":"2024-03-26T08:33:06.341384+0000","last_undegraded":"2024-03-26T08:33:06.341384+0000","last_fullsized":"2024-03-26T08:33:06.341384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:59:12.544248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.570150+0000","last_change":"2024-03-26T08:31:26.444754+0000","last_active":"2024-03-26T08:32:38.570150+0000","last_peered":"2024-03-26T08:32:38.570150+0000","last_clean":"2024-03-26T08:32:38.570150+0000","last_became_active":"2024-03-26T08:31:26.444331+0000","last_became_peered":"2024-03-26T08:31:26.444331+0000","last_unstale":"2024-03-26T08:32:38.570150+0000","last_undegraded":"2024-03-26T08:32:38.570150+0000","last_fullsized":"2024-03-26T08:32:38.570150+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:27:58.430695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.572968+0000","last_change":"2024-03-26T08:31:21.371028+0000","last_active":"2024-03-26T08:32:38.572968+0000","last_peered":"2024-03-26T08:32:38.572968+0000","last_clean":"2024-03-26T08:32:38.572968+0000","last_became_active":"2024-03-26T08:31:21.370631+0000","last_became_peered":"2024-03-26T08:31:21.370631+0000","last_unstale":"2024-03-26T08:32:38.572968+0000","last_undegraded":"2024-03-26T08:32:38.572968+0000","last_fullsized":"2024-03-26T08:32:38.572968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:33:25.546255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"94'7","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.335399+0000","last_change":"2024-03-26T08:31:22.388791+0000","last_active":"2024-03-26T08:33:06.335399+0000","last_peered":"2024-03-26T08:33:06.335399+0000","last_clean":"2024-03-26T08:33:06.335399+0000","last_became_active":"2024-03-26T08:31:22.388342+0000","last_became_peered":"2024-03-26T08:31:22.388342+0000","last_unstale":"2024-03-26T08:33:06.335399+0000","last_undegraded":"2024-03-26T08:33:06.335399+0000","last_fullsized":"2024-03-26T08:33:06.335399+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:26:30.537880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.570444+0000","last_change":"2024-03-26T08:31:26.478919+0000","last_active":"2024-03-26T08:32:38.570444+0000","last_peered":"2024-03-26T08:32:38.570444+0000","last_clean":"2024-03-26T08:32:38.570444+0000","last_became_active":"2024-03-26T08:31:26.478330+0000","last_became_peered":"2024-03-26T08:31:26.478330+0000","last_unstale":"2024-03-26T08:32:38.570444+0000","last_undegraded":"2024-03-26T08:32:38.570444+0000","last_fullsized":"2024-03-26T08:32:38.570444+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:37:36.421982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"94'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.335085+0000","last_change":"2024-03-26T08:31:22.384299+0000","last_active":"2024-03-26T08:33:06.335085+0000","last_peered":"2024-03-26T08:33:06.335085+0000","last_clean":"2024-03-26T08:33:06.335085+0000","last_became_active":"2024-03-26T08:31:22.383955+0000","last_became_peered":"2024-03-26T08:31:22.383955+0000","last_unstale":"2024-03-26T08:33:06.335085+0000","last_undegraded":"2024-03-26T08:33:06.335085+0000","last_fullsized":"2024-03-26T08:33:06.335085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:08:53.065524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.570916+0000","last_change":"2024-03-26T08:31:21.358464+0000","last_active":"2024-03-26T08:32:38.570916+0000","last_peered":"2024-03-26T08:32:38.570916+0000","last_clean":"2024-03-26T08:32:38.570916+0000","last_became_active":"2024-03-26T08:31:21.356853+0000","last_became_peered":"2024-03-26T08:31:21.356853+0000","last_unstale":"2024-03-26T08:32:38.570916+0000","last_undegraded":"2024-03-26T08:32:38.570916+0000","last_fullsized":"2024-03-26T08:32:38.570916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:08:21.395306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"92'1","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.572677+0000","last_change":"2024-03-26T08:31:37.502144+0000","last_active":"2024-03-26T08:32:38.572677+0000","last_peered":"2024-03-26T08:32:38.572677+0000","last_clean":"2024-03-26T08:32:38.572677+0000","last_became_active":"2024-03-26T08:31:37.501465+0000","last_became_peered":"2024-03-26T08:31:37.501465+0000","last_unstale":"2024-03-26T08:32:38.572677+0000","last_undegraded":"2024-03-26T08:32:38.572677+0000","last_fullsized":"2024-03-26T08:32:38.572677+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:30:18.771898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"94'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.332952+0000","last_change":"2024-03-26T08:31:22.403380+0000","last_active":"2024-03-26T08:33:06.332952+0000","last_peered":"2024-03-26T08:33:06.332952+0000","last_clean":"2024-03-26T08:33:06.332952+0000","last_became_active":"2024-03-26T08:31:22.400991+0000","last_became_peered":"2024-03-26T08:31:22.400991+0000","last_unstale":"2024-03-26T08:33:06.332952+0000","last_undegraded":"2024-03-26T08:33:06.332952+0000","last_fullsized":"2024-03-26T08:33:06.332952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:04:38.842919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.268183+0000","last_change":"2024-03-26T08:31:21.358570+0000","last_active":"2024-03-26T08:32:38.268183+0000","last_peered":"2024-03-26T08:32:38.268183+0000","last_clean":"2024-03-26T08:32:38.268183+0000","last_became_active":"2024-03-26T08:31:21.356603+0000","last_became_peered":"2024-03-26T08:31:21.356603+0000","last_unstale":"2024-03-26T08:32:38.268183+0000","last_undegraded":"2024-03-26T08:32:38.268183+0000","last_fullsized":"2024-03-26T08:32:38.268183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:10:28.829223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":356,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.573161+0000","last_change":"2024-03-26T08:29:39.048593+0000","last_active":"2024-03-26T08:32:38.573161+0000","last_peered":"2024-03-26T08:32:38.573161+0000","last_clean":"2024-03-26T08:32:38.573161+0000","last_became_active":"2024-03-26T08:29:39.047909+0000","last_became_peered":"2024-03-26T08:29:39.047909+0000","last_unstale":"2024-03-26T08:32:38.573161+0000","last_undegraded":"2024-03-26T08:32:38.573161+0000","last_fullsized":"2024-03-26T08:32:38.573161+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:24:55.011726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:24:55.011726+0000","last_clean_scrub_stamp":"2024-03-26T08:24:55.011726+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:46:55.510857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.458439+0000","last_change":"2024-03-26T08:31:26.478740+0000","last_active":"2024-03-26T08:32:38.458439+0000","last_peered":"2024-03-26T08:32:38.458439+0000","last_clean":"2024-03-26T08:32:38.458439+0000","last_became_active":"2024-03-26T08:31:26.478218+0000","last_became_peered":"2024-03-26T08:31:26.478218+0000","last_unstale":"2024-03-26T08:32:38.458439+0000","last_undegraded":"2024-03-26T08:32:38.458439+0000","last_fullsized":"2024-03-26T08:32:38.458439+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:02:54.506805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"94'5","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.331680+0000","last_change":"2024-03-26T08:31:22.381931+0000","last_active":"2024-03-26T08:33:06.331680+0000","last_peered":"2024-03-26T08:33:06.331680+0000","last_clean":"2024-03-26T08:33:06.331680+0000","last_became_active":"2024-03-26T08:31:22.379021+0000","last_became_peered":"2024-03-26T08:31:22.379021+0000","last_unstale":"2024-03-26T08:33:06.331680+0000","last_undegraded":"2024-03-26T08:33:06.331680+0000","last_fullsized":"2024-03-26T08:33:06.331680+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:07:31.555154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.572111+0000","last_change":"2024-03-26T08:31:21.375753+0000","last_active":"2024-03-26T08:32:38.572111+0000","last_peered":"2024-03-26T08:32:38.572111+0000","last_clean":"2024-03-26T08:32:38.572111+0000","last_became_active":"2024-03-26T08:31:21.375340+0000","last_became_peered":"2024-03-26T08:31:21.375340+0000","last_unstale":"2024-03-26T08:32:38.572111+0000","last_undegraded":"2024-03-26T08:32:38.572111+0000","last_fullsized":"2024-03-26T08:32:38.572111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:21:14.241774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.571348+0000","last_change":"2024-03-26T08:31:26.454026+0000","last_active":"2024-03-26T08:32:38.571348+0000","last_peered":"2024-03-26T08:32:38.571348+0000","last_clean":"2024-03-26T08:32:38.571348+0000","last_became_active":"2024-03-26T08:31:26.453494+0000","last_became_peered":"2024-03-26T08:31:26.453494+0000","last_unstale":"2024-03-26T08:32:38.571348+0000","last_undegraded":"2024-03-26T08:32:38.571348+0000","last_fullsized":"2024-03-26T08:32:38.571348+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:30:10.388280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.268131+0000","last_change":"2024-03-26T08:31:21.358930+0000","last_active":"2024-03-26T08:32:38.268131+0000","last_peered":"2024-03-26T08:32:38.268131+0000","last_clean":"2024-03-26T08:32:38.268131+0000","last_became_active":"2024-03-26T08:31:21.357573+0000","last_became_peered":"2024-03-26T08:31:21.357573+0000","last_unstale":"2024-03-26T08:32:38.268131+0000","last_undegraded":"2024-03-26T08:32:38.268131+0000","last_fullsized":"2024-03-26T08:32:38.268131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:36:41.572993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"94'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.343400+0000","last_change":"2024-03-26T08:31:22.378520+0000","last_active":"2024-03-26T08:33:06.343400+0000","last_peered":"2024-03-26T08:33:06.343400+0000","last_clean":"2024-03-26T08:33:06.343400+0000","last_became_active":"2024-03-26T08:31:22.378081+0000","last_became_peered":"2024-03-26T08:31:22.378081+0000","last_unstale":"2024-03-26T08:33:06.343400+0000","last_undegraded":"2024-03-26T08:33:06.343400+0000","last_fullsized":"2024-03-26T08:33:06.343400+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:09:34.748182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.265421+0000","last_change":"2024-03-26T08:31:26.479545+0000","last_active":"2024-03-26T08:32:38.265421+0000","last_peered":"2024-03-26T08:32:38.265421+0000","last_clean":"2024-03-26T08:32:38.265421+0000","last_became_active":"2024-03-26T08:31:26.479029+0000","last_became_peered":"2024-03-26T08:31:26.479029+0000","last_unstale":"2024-03-26T08:32:38.265421+0000","last_undegraded":"2024-03-26T08:32:38.265421+0000","last_fullsized":"2024-03-26T08:32:38.265421+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:49:03.795752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.273412+0000","last_change":"2024-03-26T08:31:21.360510+0000","last_active":"2024-03-26T08:32:38.273412+0000","last_peered":"2024-03-26T08:32:38.273412+0000","last_clean":"2024-03-26T08:32:38.273412+0000","last_became_active":"2024-03-26T08:31:21.359593+0000","last_became_peered":"2024-03-26T08:31:21.359593+0000","last_unstale":"2024-03-26T08:32:38.273412+0000","last_undegraded":"2024-03-26T08:32:38.273412+0000","last_fullsized":"2024-03-26T08:32:38.273412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:37:33.752689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"94'5","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.348139+0000","last_change":"2024-03-26T08:31:22.390181+0000","last_active":"2024-03-26T08:33:06.348139+0000","last_peered":"2024-03-26T08:33:06.348139+0000","last_clean":"2024-03-26T08:33:06.348139+0000","last_became_active":"2024-03-26T08:31:22.389686+0000","last_became_peered":"2024-03-26T08:31:22.389686+0000","last_unstale":"2024-03-26T08:33:06.348139+0000","last_undegraded":"2024-03-26T08:33:06.348139+0000","last_fullsized":"2024-03-26T08:33:06.348139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:48:09.913134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.309800+0000","last_change":"2024-03-26T08:31:26.449160+0000","last_active":"2024-03-26T08:32:38.309800+0000","last_peered":"2024-03-26T08:32:38.309800+0000","last_clean":"2024-03-26T08:32:38.309800+0000","last_became_active":"2024-03-26T08:31:26.448570+0000","last_became_peered":"2024-03-26T08:31:26.448570+0000","last_unstale":"2024-03-26T08:32:38.309800+0000","last_undegraded":"2024-03-26T08:32:38.309800+0000","last_fullsized":"2024-03-26T08:32:38.309800+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:05:21.258982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.268339+0000","last_change":"2024-03-26T08:31:21.365723+0000","last_active":"2024-03-26T08:32:38.268339+0000","last_peered":"2024-03-26T08:32:38.268339+0000","last_clean":"2024-03-26T08:32:38.268339+0000","last_became_active":"2024-03-26T08:31:21.364893+0000","last_became_peered":"2024-03-26T08:31:21.364893+0000","last_unstale":"2024-03-26T08:32:38.268339+0000","last_undegraded":"2024-03-26T08:32:38.268339+0000","last_fullsized":"2024-03-26T08:32:38.268339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:24:17.210093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.342502+0000","last_change":"2024-03-26T08:31:22.396341+0000","last_active":"2024-03-26T08:33:06.342502+0000","last_peered":"2024-03-26T08:33:06.342502+0000","last_clean":"2024-03-26T08:33:06.342502+0000","last_became_active":"2024-03-26T08:31:22.395956+0000","last_became_peered":"2024-03-26T08:31:22.395956+0000","last_unstale":"2024-03-26T08:33:06.342502+0000","last_undegraded":"2024-03-26T08:33:06.342502+0000","last_fullsized":"2024-03-26T08:33:06.342502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:08:51.194370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.267538+0000","last_change":"2024-03-26T08:31:37.497971+0000","last_active":"2024-03-26T08:32:38.267538+0000","last_peered":"2024-03-26T08:32:38.267538+0000","last_clean":"2024-03-26T08:32:38.267538+0000","last_became_active":"2024-03-26T08:31:37.497379+0000","last_became_peered":"2024-03-26T08:31:37.497379+0000","last_unstale":"2024-03-26T08:32:38.267538+0000","last_undegraded":"2024-03-26T08:32:38.267538+0000","last_fullsized":"2024-03-26T08:32:38.267538+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:35:55.266756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.534861+0000","last_change":"2024-03-26T08:31:37.493616+0000","last_active":"2024-03-26T08:32:38.534861+0000","last_peered":"2024-03-26T08:32:38.534861+0000","last_clean":"2024-03-26T08:32:38.534861+0000","last_became_active":"2024-03-26T08:31:37.492288+0000","last_became_peered":"2024-03-26T08:31:37.492288+0000","last_unstale":"2024-03-26T08:32:38.534861+0000","last_undegraded":"2024-03-26T08:32:38.534861+0000","last_fullsized":"2024-03-26T08:32:38.534861+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:49:10.413751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"94'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.275996+0000","last_change":"2024-03-26T08:31:22.384347+0000","last_active":"2024-03-26T08:33:06.275996+0000","last_peered":"2024-03-26T08:33:06.275996+0000","last_clean":"2024-03-26T08:33:06.275996+0000","last_became_active":"2024-03-26T08:31:22.380639+0000","last_became_peered":"2024-03-26T08:31:22.380639+0000","last_unstale":"2024-03-26T08:33:06.275996+0000","last_undegraded":"2024-03-26T08:33:06.275996+0000","last_fullsized":"2024-03-26T08:33:06.275996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:39:07.778005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.273214+0000","last_change":"2024-03-26T08:31:26.455706+0000","last_active":"2024-03-26T08:32:38.273214+0000","last_peered":"2024-03-26T08:32:38.273214+0000","last_clean":"2024-03-26T08:32:38.273214+0000","last_became_active":"2024-03-26T08:31:26.454591+0000","last_became_peered":"2024-03-26T08:31:26.454591+0000","last_unstale":"2024-03-26T08:32:38.273214+0000","last_undegraded":"2024-03-26T08:32:38.273214+0000","last_fullsized":"2024-03-26T08:32:38.273214+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:07:30.612220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.266005+0000","last_change":"2024-03-26T08:31:21.357734+0000","last_active":"2024-03-26T08:32:38.266005+0000","last_peered":"2024-03-26T08:32:38.266005+0000","last_clean":"2024-03-26T08:32:38.266005+0000","last_became_active":"2024-03-26T08:31:21.357361+0000","last_became_peered":"2024-03-26T08:31:21.357361+0000","last_unstale":"2024-03-26T08:32:38.266005+0000","last_undegraded":"2024-03-26T08:32:38.266005+0000","last_fullsized":"2024-03-26T08:32:38.266005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:26:57.454577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"94'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.338137+0000","last_change":"2024-03-26T08:31:22.383118+0000","last_active":"2024-03-26T08:33:06.338137+0000","last_peered":"2024-03-26T08:33:06.338137+0000","last_clean":"2024-03-26T08:33:06.338137+0000","last_became_active":"2024-03-26T08:31:22.382773+0000","last_became_peered":"2024-03-26T08:31:22.382773+0000","last_unstale":"2024-03-26T08:33:06.338137+0000","last_undegraded":"2024-03-26T08:33:06.338137+0000","last_fullsized":"2024-03-26T08:33:06.338137+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:50:34.889914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"92'1","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.268151+0000","last_change":"2024-03-26T08:31:26.451467+0000","last_active":"2024-03-26T08:32:38.268151+0000","last_peered":"2024-03-26T08:32:38.268151+0000","last_clean":"2024-03-26T08:32:38.268151+0000","last_became_active":"2024-03-26T08:31:26.451022+0000","last_became_peered":"2024-03-26T08:31:26.451022+0000","last_unstale":"2024-03-26T08:32:38.268151+0000","last_undegraded":"2024-03-26T08:32:38.268151+0000","last_fullsized":"2024-03-26T08:32:38.268151+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:19:50.442779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.310275+0000","last_change":"2024-03-26T08:31:21.361596+0000","last_active":"2024-03-26T08:32:38.310275+0000","last_peered":"2024-03-26T08:32:38.310275+0000","last_clean":"2024-03-26T08:32:38.310275+0000","last_became_active":"2024-03-26T08:31:21.357502+0000","last_became_peered":"2024-03-26T08:31:21.357502+0000","last_unstale":"2024-03-26T08:32:38.310275+0000","last_undegraded":"2024-03-26T08:32:38.310275+0000","last_fullsized":"2024-03-26T08:32:38.310275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:05:33.951346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"94'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.339334+0000","last_change":"2024-03-26T08:31:22.397344+0000","last_active":"2024-03-26T08:33:06.339334+0000","last_peered":"2024-03-26T08:33:06.339334+0000","last_clean":"2024-03-26T08:33:06.339334+0000","last_became_active":"2024-03-26T08:31:22.396936+0000","last_became_peered":"2024-03-26T08:31:22.396936+0000","last_unstale":"2024-03-26T08:33:06.339334+0000","last_undegraded":"2024-03-26T08:33:06.339334+0000","last_fullsized":"2024-03-26T08:33:06.339334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:07:49.412098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.267394+0000","last_change":"2024-03-26T08:31:26.479060+0000","last_active":"2024-03-26T08:32:38.267394+0000","last_peered":"2024-03-26T08:32:38.267394+0000","last_clean":"2024-03-26T08:32:38.267394+0000","last_became_active":"2024-03-26T08:31:26.478519+0000","last_became_peered":"2024-03-26T08:31:26.478519+0000","last_unstale":"2024-03-26T08:32:38.267394+0000","last_undegraded":"2024-03-26T08:32:38.267394+0000","last_fullsized":"2024-03-26T08:32:38.267394+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:37:20.528845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.267142+0000","last_change":"2024-03-26T08:31:21.374128+0000","last_active":"2024-03-26T08:32:38.267142+0000","last_peered":"2024-03-26T08:32:38.267142+0000","last_clean":"2024-03-26T08:32:38.267142+0000","last_became_active":"2024-03-26T08:31:21.372934+0000","last_became_peered":"2024-03-26T08:31:21.372934+0000","last_unstale":"2024-03-26T08:32:38.267142+0000","last_undegraded":"2024-03-26T08:32:38.267142+0000","last_fullsized":"2024-03-26T08:32:38.267142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:25:15.919855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"94'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.349410+0000","last_change":"2024-03-26T08:31:22.381847+0000","last_active":"2024-03-26T08:33:06.349410+0000","last_peered":"2024-03-26T08:33:06.349410+0000","last_clean":"2024-03-26T08:33:06.349410+0000","last_became_active":"2024-03-26T08:31:22.379956+0000","last_became_peered":"2024-03-26T08:31:22.379956+0000","last_unstale":"2024-03-26T08:33:06.349410+0000","last_undegraded":"2024-03-26T08:33:06.349410+0000","last_fullsized":"2024-03-26T08:33:06.349410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:16:38.048495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.534107+0000","last_change":"2024-03-26T08:31:26.455503+0000","last_active":"2024-03-26T08:32:38.534107+0000","last_peered":"2024-03-26T08:32:38.534107+0000","last_clean":"2024-03-26T08:32:38.534107+0000","last_became_active":"2024-03-26T08:31:26.455023+0000","last_became_peered":"2024-03-26T08:31:26.455023+0000","last_unstale":"2024-03-26T08:32:38.534107+0000","last_undegraded":"2024-03-26T08:32:38.534107+0000","last_fullsized":"2024-03-26T08:32:38.534107+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:06:47.565407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.311173+0000","last_change":"2024-03-26T08:31:21.362832+0000","last_active":"2024-03-26T08:32:38.311173+0000","last_peered":"2024-03-26T08:32:38.311173+0000","last_clean":"2024-03-26T08:32:38.311173+0000","last_became_active":"2024-03-26T08:31:21.362411+0000","last_became_peered":"2024-03-26T08:31:21.362411+0000","last_unstale":"2024-03-26T08:32:38.311173+0000","last_undegraded":"2024-03-26T08:32:38.311173+0000","last_fullsized":"2024-03-26T08:32:38.311173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:34:07.646011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.349836+0000","last_change":"2024-03-26T08:31:22.403106+0000","last_active":"2024-03-26T08:33:06.349836+0000","last_peered":"2024-03-26T08:33:06.349836+0000","last_clean":"2024-03-26T08:33:06.349836+0000","last_became_active":"2024-03-26T08:31:22.401172+0000","last_became_peered":"2024-03-26T08:31:22.401172+0000","last_unstale":"2024-03-26T08:33:06.349836+0000","last_undegraded":"2024-03-26T08:33:06.349836+0000","last_fullsized":"2024-03-26T08:33:06.349836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:31:42.654817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.572016+0000","last_change":"2024-03-26T08:31:37.495759+0000","last_active":"2024-03-26T08:32:38.572016+0000","last_peered":"2024-03-26T08:32:38.572016+0000","last_clean":"2024-03-26T08:32:38.572016+0000","last_became_active":"2024-03-26T08:31:37.493610+0000","last_became_peered":"2024-03-26T08:31:37.493610+0000","last_unstale":"2024-03-26T08:32:38.572016+0000","last_undegraded":"2024-03-26T08:32:38.572016+0000","last_fullsized":"2024-03-26T08:32:38.572016+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:17:05.345033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.458336+0000","last_change":"2024-03-26T08:31:21.369396+0000","last_active":"2024-03-26T08:32:38.458336+0000","last_peered":"2024-03-26T08:32:38.458336+0000","last_clean":"2024-03-26T08:32:38.458336+0000","last_became_active":"2024-03-26T08:31:21.364011+0000","last_became_peered":"2024-03-26T08:31:21.364011+0000","last_unstale":"2024-03-26T08:32:38.458336+0000","last_undegraded":"2024-03-26T08:32:38.458336+0000","last_fullsized":"2024-03-26T08:32:38.458336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:23:47.347158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.350016+0000","last_change":"2024-03-26T08:31:22.389586+0000","last_active":"2024-03-26T08:33:06.350016+0000","last_peered":"2024-03-26T08:33:06.350016+0000","last_clean":"2024-03-26T08:33:06.350016+0000","last_became_active":"2024-03-26T08:31:22.389200+0000","last_became_peered":"2024-03-26T08:31:22.389200+0000","last_unstale":"2024-03-26T08:33:06.350016+0000","last_undegraded":"2024-03-26T08:33:06.350016+0000","last_fullsized":"2024-03-26T08:33:06.350016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:54:30.076744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.454375+0000","last_change":"2024-03-26T08:31:26.452274+0000","last_active":"2024-03-26T08:32:38.454375+0000","last_peered":"2024-03-26T08:32:38.454375+0000","last_clean":"2024-03-26T08:32:38.454375+0000","last_became_active":"2024-03-26T08:31:26.451670+0000","last_became_peered":"2024-03-26T08:31:26.451670+0000","last_unstale":"2024-03-26T08:32:38.454375+0000","last_undegraded":"2024-03-26T08:32:38.454375+0000","last_fullsized":"2024-03-26T08:32:38.454375+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:25.394383+0000","last_clean_scrub_stamp":"2024-03-26T08:31:25.394383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:48:36.028810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:32:38.573718+0000","last_change":"2024-03-26T08:31:21.356259+0000","last_active":"2024-03-26T08:32:38.573718+0000","last_peered":"2024-03-26T08:32:38.573718+0000","last_clean":"2024-03-26T08:32:38.573718+0000","last_became_active":"2024-03-26T08:31:21.355724+0000","last_became_peered":"2024-03-26T08:31:21.355724+0000","last_unstale":"2024-03-26T08:32:38.573718+0000","last_undegraded":"2024-03-26T08:32:38.573718+0000","last_fullsized":"2024-03-26T08:32:38.573718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:20.300330+0000","last_clean_scrub_stamp":"2024-03-26T08:31:20.300330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:30:05.621417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.340153+0000","last_change":"2024-03-26T08:31:22.370869+0000","last_active":"2024-03-26T08:33:06.340153+0000","last_peered":"2024-03-26T08:33:06.340153+0000","last_clean":"2024-03-26T08:33:06.340153+0000","last_became_active":"2024-03-26T08:31:22.369797+0000","last_became_peered":"2024-03-26T08:31:22.369797+0000","last_unstale":"2024-03-26T08:33:06.340153+0000","last_undegraded":"2024-03-26T08:33:06.340153+0000","last_fullsized":"2024-03-26T08:33:06.340153+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:59:33.064619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"94'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.336607+0000","last_change":"2024-03-26T08:31:22.377927+0000","last_active":"2024-03-26T08:33:06.336607+0000","last_peered":"2024-03-26T08:33:06.336607+0000","last_clean":"2024-03-26T08:33:06.336607+0000","last_became_active":"2024-03-26T08:31:22.377559+0000","last_became_peered":"2024-03-26T08:31:22.377559+0000","last_unstale":"2024-03-26T08:33:06.336607+0000","last_undegraded":"2024-03-26T08:33:06.336607+0000","last_fullsized":"2024-03-26T08:33:06.336607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:06:05.544179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"94'12","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.340397+0000","last_change":"2024-03-26T08:31:22.380287+0000","last_active":"2024-03-26T08:33:06.340397+0000","last_peered":"2024-03-26T08:33:06.340397+0000","last_clean":"2024-03-26T08:33:06.340397+0000","last_became_active":"2024-03-26T08:31:22.379948+0000","last_became_peered":"2024-03-26T08:31:22.379948+0000","last_unstale":"2024-03-26T08:33:06.340397+0000","last_undegraded":"2024-03-26T08:33:06.340397+0000","last_fullsized":"2024-03-26T08:33:06.340397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:57:12.636151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"94'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.333723+0000","last_change":"2024-03-26T08:31:22.390109+0000","last_active":"2024-03-26T08:33:06.333723+0000","last_peered":"2024-03-26T08:33:06.333723+0000","last_clean":"2024-03-26T08:33:06.333723+0000","last_became_active":"2024-03-26T08:31:22.389766+0000","last_became_peered":"2024-03-26T08:31:22.389766+0000","last_unstale":"2024-03-26T08:33:06.333723+0000","last_undegraded":"2024-03-26T08:33:06.333723+0000","last_fullsized":"2024-03-26T08:33:06.333723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:24:27.454598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.341634+0000","last_change":"2024-03-26T08:31:22.391882+0000","last_active":"2024-03-26T08:33:06.341634+0000","last_peered":"2024-03-26T08:33:06.341634+0000","last_clean":"2024-03-26T08:33:06.341634+0000","last_became_active":"2024-03-26T08:31:22.391395+0000","last_became_peered":"2024-03-26T08:31:22.391395+0000","last_unstale":"2024-03-26T08:33:06.341634+0000","last_undegraded":"2024-03-26T08:33:06.341634+0000","last_fullsized":"2024-03-26T08:33:06.341634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:45:06.685287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"94'4","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.333414+0000","last_change":"2024-03-26T08:31:22.385993+0000","last_active":"2024-03-26T08:33:06.333414+0000","last_peered":"2024-03-26T08:33:06.333414+0000","last_clean":"2024-03-26T08:33:06.333414+0000","last_became_active":"2024-03-26T08:31:22.385625+0000","last_became_peered":"2024-03-26T08:31:22.385625+0000","last_unstale":"2024-03-26T08:33:06.333414+0000","last_undegraded":"2024-03-26T08:33:06.333414+0000","last_fullsized":"2024-03-26T08:33:06.333414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:16:20.731718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"94'6","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.334408+0000","last_change":"2024-03-26T08:31:22.379588+0000","last_active":"2024-03-26T08:33:06.334408+0000","last_peered":"2024-03-26T08:33:06.334408+0000","last_clean":"2024-03-26T08:33:06.334408+0000","last_became_active":"2024-03-26T08:31:22.379215+0000","last_became_peered":"2024-03-26T08:31:22.379215+0000","last_unstale":"2024-03-26T08:33:06.334408+0000","last_undegraded":"2024-03-26T08:33:06.334408+0000","last_fullsized":"2024-03-26T08:33:06.334408+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:23:20.339737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"94'15","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.345962+0000","last_change":"2024-03-26T08:31:22.404354+0000","last_active":"2024-03-26T08:33:06.345962+0000","last_peered":"2024-03-26T08:33:06.345962+0000","last_clean":"2024-03-26T08:33:06.345962+0000","last_became_active":"2024-03-26T08:31:22.403973+0000","last_became_peered":"2024-03-26T08:31:22.403973+0000","last_unstale":"2024-03-26T08:33:06.345962+0000","last_undegraded":"2024-03-26T08:33:06.345962+0000","last_fullsized":"2024-03-26T08:33:06.345962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:36:54.360147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"94'2","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.318506+0000","last_change":"2024-03-26T08:31:22.379642+0000","last_active":"2024-03-26T08:33:06.318506+0000","last_peered":"2024-03-26T08:33:06.318506+0000","last_clean":"2024-03-26T08:33:06.318506+0000","last_became_active":"2024-03-26T08:31:22.379241+0000","last_became_peered":"2024-03-26T08:31:22.379241+0000","last_unstale":"2024-03-26T08:33:06.318506+0000","last_undegraded":"2024-03-26T08:33:06.318506+0000","last_fullsized":"2024-03-26T08:33:06.318506+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:21:51.117616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"94'6","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.344111+0000","last_change":"2024-03-26T08:31:22.378809+0000","last_active":"2024-03-26T08:33:06.344111+0000","last_peered":"2024-03-26T08:33:06.344111+0000","last_clean":"2024-03-26T08:33:06.344111+0000","last_became_active":"2024-03-26T08:31:22.378388+0000","last_became_peered":"2024-03-26T08:31:22.378388+0000","last_unstale":"2024-03-26T08:33:06.344111+0000","last_undegraded":"2024-03-26T08:33:06.344111+0000","last_fullsized":"2024-03-26T08:33:06.344111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:33:21.583423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"94'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.340907+0000","last_change":"2024-03-26T08:31:22.389230+0000","last_active":"2024-03-26T08:33:06.340907+0000","last_peered":"2024-03-26T08:33:06.340907+0000","last_clean":"2024-03-26T08:33:06.340907+0000","last_became_active":"2024-03-26T08:31:22.388870+0000","last_became_peered":"2024-03-26T08:31:22.388870+0000","last_unstale":"2024-03-26T08:33:06.340907+0000","last_undegraded":"2024-03-26T08:33:06.340907+0000","last_fullsized":"2024-03-26T08:33:06.340907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:40:20.377122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"94'10","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.351060+0000","last_change":"2024-03-26T08:31:22.396232+0000","last_active":"2024-03-26T08:33:06.351060+0000","last_peered":"2024-03-26T08:33:06.351060+0000","last_clean":"2024-03-26T08:33:06.351060+0000","last_became_active":"2024-03-26T08:31:22.392940+0000","last_became_peered":"2024-03-26T08:31:22.392940+0000","last_unstale":"2024-03-26T08:33:06.351060+0000","last_undegraded":"2024-03-26T08:33:06.351060+0000","last_fullsized":"2024-03-26T08:33:06.351060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:01:15.460526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"94'11","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.348113+0000","last_change":"2024-03-26T08:31:22.370880+0000","last_active":"2024-03-26T08:33:06.348113+0000","last_peered":"2024-03-26T08:33:06.348113+0000","last_clean":"2024-03-26T08:33:06.348113+0000","last_became_active":"2024-03-26T08:31:22.370546+0000","last_became_peered":"2024-03-26T08:31:22.370546+0000","last_unstale":"2024-03-26T08:33:06.348113+0000","last_undegraded":"2024-03-26T08:33:06.348113+0000","last_fullsized":"2024-03-26T08:33:06.348113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:46:11.657763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"94'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T08:33:06.341245+0000","last_change":"2024-03-26T08:31:22.387573+0000","last_active":"2024-03-26T08:33:06.341245+0000","last_peered":"2024-03-26T08:33:06.341245+0000","last_clean":"2024-03-26T08:33:06.341245+0000","last_became_active":"2024-03-26T08:31:22.387142+0000","last_became_peered":"2024-03-26T08:31:22.387142+0000","last_unstale":"2024-03-26T08:33:06.341245+0000","last_undegraded":"2024-03-26T08:33:06.341245+0000","last_fullsized":"2024-03-26T08:33:06.341245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:31:21.324106+0000","last_clean_scrub_stamp":"2024-03-26T08:31:21.324106+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:13:32.307195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281284,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63119,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":190,"num_read_kb":160,"num_write":86,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":189357,"data_compressed":5637,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351016,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":521983,"data_compressed":26706,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612655,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32304,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712848,"statfs":{"total":95995035648,"available":95961956352,"internally_reserved":0,"allocated":1155072,"data_stored":1174844,"data_compressed":64362,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776183,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":522543,"data_compressed":26706,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037822,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":522543,"data_compressed":26706,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299460,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":1117007,"data_compressed":62483,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462987,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":517865,"data_compressed":26706,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":580186,"data_compressed":28585,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738484,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":517955,"data_compressed":26706,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411433,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1093632,"data_stored":580220,"data_compressed":28585,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149793,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":517969,"data_compressed":26706,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888153,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":521983,"data_compressed":26706,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724626,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":1112511,"data_compressed":62483,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34133,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34133,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34133,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35777,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35777,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35777,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35777,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35777,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62321,"data_compressed":1879,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62355,"data_compressed":1879,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62515,"data_compressed":1879,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T08:33:21.193 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-26T08:33:21.193 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-26T08:33:21.193 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-26T08:33:21.193 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph health --format=json 2024-03-26T08:33:21.702 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:33:22.107 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:21 smithi043 ceph-mon[31929]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:33:22.108 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:21 smithi043 ceph-mon[31929]: pgmap v436: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T08:33:22.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:21 smithi183 ceph-mon[35818]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:33:22.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:21 smithi183 ceph-mon[35818]: pgmap v436: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T08:33:22.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:21 smithi052 ceph-mon[35071]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:33:22.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:21 smithi052 ceph-mon[35071]: pgmap v436: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T08:33:24.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.089+0000 7fcbd8400700 1 -- 172.21.15.43:0/3384867328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbd00743c0 msgr2=0x7fcbd00747a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:24.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.089+0000 7fcbd8400700 1 --2- 172.21.15.43:0/3384867328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbd00743c0 0x7fcbd00747a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40099c0 tx=0x7fcbc4009cd0 comp rx=0 tx=0).stop 2024-03-26T08:33:24.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.089+0000 7fcbd8400700 1 -- 172.21.15.43:0/3384867328 shutdown_connections 2024-03-26T08:33:24.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.089+0000 7fcbd8400700 1 --2- 172.21.15.43:0/3384867328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbd0113360 0x7fcbd0115750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:24.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.089+0000 7fcbd8400700 1 --2- 172.21.15.43:0/3384867328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbd00743c0 0x7fcbd00747a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:24.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.089+0000 7fcbd8400700 1 --2- 172.21.15.43:0/3384867328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbd0074ce0 0x7fcbd0112e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:24.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.089+0000 7fcbd8400700 1 -- 172.21.15.43:0/3384867328 >> 172.21.15.43:0/3384867328 conn(0x7fcbd00792f0 msgr2=0x7fcbd0079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:24.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.089+0000 7fcbd8400700 1 -- 172.21.15.43:0/3384867328 shutdown_connections 2024-03-26T08:33:24.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.089+0000 7fcbd8400700 1 -- 172.21.15.43:0/3384867328 wait complete. 2024-03-26T08:33:24.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.090+0000 7fcbd8400700 1 Processor -- start 2024-03-26T08:33:24.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.090+0000 7fcbd8400700 1 -- start start 2024-03-26T08:33:24.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.091+0000 7fcbd8400700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbd00743c0 0x7fcbd019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:24.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.091+0000 7fcbd8400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbd0074ce0 0x7fcbd019f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:24.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.091+0000 7fcbd8400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbd0113360 0x7fcbd01a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:24.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.091+0000 7fcbd8400700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd0118a40 con 0x7fcbd0113360 2024-03-26T08:33:24.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.091+0000 7fcbd8400700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd01188c0 con 0x7fcbd00743c0 2024-03-26T08:33:24.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.091+0000 7fcbd8400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd0118bc0 con 0x7fcbd0074ce0 2024-03-26T08:33:24.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.091+0000 7fcbd6400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbd0113360 0x7fcbd01a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:24.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.091+0000 7fcbd6400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbd0113360 0x7fcbd01a3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51948/0 (socket says 172.21.15.43:51948) 2024-03-26T08:33:24.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.091+0000 7fcbd6400700 1 -- 172.21.15.43:0/37792479 learned_addr learned my addr 172.21.15.43:0/37792479 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.091+0000 7fcbd5a00700 1 --2- 172.21.15.43:0/37792479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbd00743c0 0x7fcbd019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.091+0000 7fcbd5000700 1 --2- 172.21.15.43:0/37792479 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbd0074ce0 0x7fcbd019f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.091+0000 7fcbd6400700 1 -- 172.21.15.43:0/37792479 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbd0074ce0 msgr2=0x7fcbd019f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.091+0000 7fcbd6400700 1 --2- 172.21.15.43:0/37792479 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbd0074ce0 0x7fcbd019f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.091+0000 7fcbd6400700 1 -- 172.21.15.43:0/37792479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbd00743c0 msgr2=0x7fcbd019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.091+0000 7fcbd6400700 1 --2- 172.21.15.43:0/37792479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbd00743c0 0x7fcbd019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.091+0000 7fcbd6400700 1 -- 172.21.15.43:0/37792479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd01a3f80 con 0x7fcbd0113360 2024-03-26T08:33:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.092+0000 7fcbd6400700 1 --2- 172.21.15.43:0/37792479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbd0113360 0x7fcbd01a3840 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcbc800ab50 tx=0x7fcbc800af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.092+0000 7fcbce000700 1 -- 172.21.15.43:0/37792479 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8017020 con 0x7fcbd0113360 2024-03-26T08:33:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.092+0000 7fcbce000700 1 -- 172.21.15.43:0/37792479 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcbc8004bb0 con 0x7fcbd0113360 2024-03-26T08:33:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.092+0000 7fcbd8400700 1 -- 172.21.15.43:0/37792479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd01a4270 con 0x7fcbd0113360 2024-03-26T08:33:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.092+0000 7fcbce000700 1 -- 172.21.15.43:0/37792479 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8005350 con 0x7fcbd0113360 2024-03-26T08:33:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.092+0000 7fcbd8400700 1 -- 172.21.15.43:0/37792479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd01abaf0 con 0x7fcbd0113360 2024-03-26T08:33:24.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.093+0000 7fcbd8400700 1 -- 172.21.15.43:0/37792479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd0068e50 con 0x7fcbd0113360 2024-03-26T08:33:24.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.095+0000 7fcbce000700 1 -- 172.21.15.43:0/37792479 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fcbc800c6d0 con 0x7fcbd0113360 2024-03-26T08:33:24.088 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:23 smithi043 ceph-mon[31929]: pgmap v437: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:24.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.097+0000 7fcbce000700 1 --2- 172.21.15.43:0/37792479 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fcbb80a2890 0x7fcbb80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:24.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.097+0000 7fcbce000700 1 -- 172.21.15.43:0/37792479 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7fcbc80ce490 con 0x7fcbd0113360 2024-03-26T08:33:24.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.097+0000 7fcbd5a00700 1 --2- 172.21.15.43:0/37792479 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fcbb80a2890 0x7fcbb80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:24.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.098+0000 7fcbd5a00700 1 --2- 172.21.15.43:0/37792479 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fcbb80a2890 0x7fcbb80a4d50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4000c00 tx=0x7fcbc4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.103+0000 7fcbce000700 1 -- 172.21.15.43:0/37792479 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcbc8021020 con 0x7fcbd0113360 2024-03-26T08:33:24.100 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:23 smithi183 ceph-mon[35818]: pgmap v437: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:24.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:23 smithi052 ceph-mon[35071]: pgmap v437: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:24.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.382+0000 7fcbd8400700 1 -- 172.21.15.43:0/37792479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fcbd01abcc0 con 0x7fcbd0113360 2024-03-26T08:33:24.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.383+0000 7fcbce000700 1 -- 172.21.15.43:0/37792479 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fcbc809b1e0 con 0x7fcbd0113360 2024-03-26T08:33:24.376 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:33:24.377 INFO:teuthology.orchestra.run.smithi043.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-26T08:33:24.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.386+0000 7fcbd8400700 1 -- 172.21.15.43:0/37792479 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fcbb80a2890 msgr2=0x7fcbb80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:24.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.386+0000 7fcbd8400700 1 --2- 172.21.15.43:0/37792479 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fcbb80a2890 0x7fcbb80a4d50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4000c00 tx=0x7fcbc4005fb0 comp rx=0 tx=0).stop 2024-03-26T08:33:24.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.386+0000 7fcbd8400700 1 -- 172.21.15.43:0/37792479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbd0113360 msgr2=0x7fcbd01a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:24.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.386+0000 7fcbd8400700 1 --2- 172.21.15.43:0/37792479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbd0113360 0x7fcbd01a3840 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcbc800ab50 tx=0x7fcbc800af10 comp rx=0 tx=0).stop 2024-03-26T08:33:24.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.387+0000 7fcbd8400700 1 -- 172.21.15.43:0/37792479 shutdown_connections 2024-03-26T08:33:24.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.387+0000 7fcbd8400700 1 --2- 172.21.15.43:0/37792479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbd0113360 0x7fcbd01a3840 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:24.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.387+0000 7fcbd8400700 1 --2- 172.21.15.43:0/37792479 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fcbb80a2890 0x7fcbb80a4d50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:24.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.387+0000 7fcbd8400700 1 --2- 172.21.15.43:0/37792479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbd00743c0 0x7fcbd019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:24.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.387+0000 7fcbd8400700 1 --2- 172.21.15.43:0/37792479 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbd0074ce0 0x7fcbd019f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:24.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.387+0000 7fcbd8400700 1 -- 172.21.15.43:0/37792479 >> 172.21.15.43:0/37792479 conn(0x7fcbd00792f0 msgr2=0x7fcbd01121a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:24.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.387+0000 7fcbd8400700 1 -- 172.21.15.43:0/37792479 shutdown_connections 2024-03-26T08:33:24.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:24.387+0000 7fcbd8400700 1 -- 172.21.15.43:0/37792479 wait complete. 2024-03-26T08:33:25.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:24 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/37792479' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-26T08:33:25.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:24 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/37792479' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-26T08:33:25.317 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-26T08:33:25.318 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:33:25.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:24 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/37792479' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-26T08:33:25.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.559+0000 7f8e8fe00640 1 -- 172.21.15.43:0/3304645541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e88105490 msgr2=0x7f8e8810bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:25.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.559+0000 7f8e8fe00640 1 --2- 172.21.15.43:0/3304645541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e88105490 0x7f8e8810bd20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009ca0 tx=0x7f8e8002c100 comp rx=0 tx=0).stop 2024-03-26T08:33:25.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.559+0000 7f8e8fe00640 1 -- 172.21.15.43:0/3304645541 shutdown_connections 2024-03-26T08:33:25.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.559+0000 7f8e8fe00640 1 --2- 172.21.15.43:0/3304645541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e88105490 0x7f8e8810bd20 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:25.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.559+0000 7f8e8fe00640 1 --2- 172.21.15.43:0/3304645541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e88104ad0 0x7f8e88104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:25.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.559+0000 7f8e8fe00640 1 --2- 172.21.15.43:0/3304645541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e881038e0 0x7f8e88103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:25.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.559+0000 7f8e8fe00640 1 -- 172.21.15.43:0/3304645541 >> 172.21.15.43:0/3304645541 conn(0x7f8e880fee90 msgr2=0x7f8e881012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:25.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.560+0000 7f8e8fe00640 1 -- 172.21.15.43:0/3304645541 shutdown_connections 2024-03-26T08:33:25.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.560+0000 7f8e8fe00640 1 -- 172.21.15.43:0/3304645541 wait complete. 2024-03-26T08:33:25.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.561+0000 7f8e8fe00640 1 Processor -- start 2024-03-26T08:33:25.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.561+0000 7f8e8fe00640 1 -- start start 2024-03-26T08:33:25.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.561+0000 7f8e8fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e881038e0 0x7f8e88199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:25.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.561+0000 7f8e8fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e88104ad0 0x7f8e88199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:25.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.561+0000 7f8e8fe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e88105490 0x7f8e881a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:25.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.561+0000 7f8e8fe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8810f410 con 0x7f8e881038e0 2024-03-26T08:33:25.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.561+0000 7f8e8fe00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8810f290 con 0x7f8e88105490 2024-03-26T08:33:25.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.561+0000 7f8e8fe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8810f590 con 0x7f8e88104ad0 2024-03-26T08:33:25.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.561+0000 7f8e8d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e881038e0 0x7f8e88199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:25.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.561+0000 7f8e8d400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e881038e0 0x7f8e88199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51972/0 (socket says 172.21.15.43:51972) 2024-03-26T08:33:25.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.561+0000 7f8e8d400640 1 -- 172.21.15.43:0/1032658144 learned_addr learned my addr 172.21.15.43:0/1032658144 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:25.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.562+0000 7f8e8de00640 1 --2- 172.21.15.43:0/1032658144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e88105490 0x7f8e881a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:25.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.562+0000 7f8e8d400640 1 -- 172.21.15.43:0/1032658144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e88104ad0 msgr2=0x7f8e88199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:25.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.562+0000 7f8e8d400640 1 --2- 172.21.15.43:0/1032658144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e88104ad0 0x7f8e88199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:25.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.562+0000 7f8e8d400640 1 -- 172.21.15.43:0/1032658144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e88105490 msgr2=0x7f8e881a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:25.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.562+0000 7f8e8d400640 1 --2- 172.21.15.43:0/1032658144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e88105490 0x7f8e881a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:25.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.562+0000 7f8e8d400640 1 -- 172.21.15.43:0/1032658144 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e881a16b0 con 0x7f8e881038e0 2024-03-26T08:33:25.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.562+0000 7f8e8d400640 1 --2- 172.21.15.43:0/1032658144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e881038e0 0x7f8e88199a90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00d950 tx=0x7f8e7c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:25.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.562+0000 7f8e86000640 1 -- 172.21.15.43:0/1032658144 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c017070 con 0x7f8e881038e0 2024-03-26T08:33:25.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.563+0000 7f8e86000640 1 -- 172.21.15.43:0/1032658144 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8e7c009c30 con 0x7f8e881038e0 2024-03-26T08:33:25.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.563+0000 7f8e8fe00640 1 -- 172.21.15.43:0/1032658144 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e881a19a0 con 0x7f8e881038e0 2024-03-26T08:33:25.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.563+0000 7f8e86000640 1 -- 172.21.15.43:0/1032658144 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c00f620 con 0x7f8e881038e0 2024-03-26T08:33:25.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.563+0000 7f8e8fe00640 1 -- 172.21.15.43:0/1032658144 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e881a1ef0 con 0x7f8e881038e0 2024-03-26T08:33:25.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.564+0000 7f8e8fe00640 1 -- 172.21.15.43:0/1032658144 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e8810acd0 con 0x7f8e881038e0 2024-03-26T08:33:25.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.566+0000 7f8e86000640 1 -- 172.21.15.43:0/1032658144 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8e7c01a070 con 0x7f8e881038e0 2024-03-26T08:33:25.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.568+0000 7f8e86000640 1 --2- 172.21.15.43:0/1032658144 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8e700a2880 0x7f8e700a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:25.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.568+0000 7f8e86000640 1 -- 172.21.15.43:0/1032658144 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f8e7c0ce8e0 con 0x7f8e881038e0 2024-03-26T08:33:25.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.568+0000 7f8e8ca00640 1 --2- 172.21.15.43:0/1032658144 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8e700a2880 0x7f8e700a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:25.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.569+0000 7f8e8ca00640 1 --2- 172.21.15.43:0/1032658144 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8e700a2880 0x7f8e700a4d40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8e78005fd0 tx=0x7f8e78005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:25.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.573+0000 7f8e86000640 1 -- 172.21.15.43:0/1032658144 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8e7c009de0 con 0x7f8e881038e0 2024-03-26T08:33:25.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.776+0000 7f8e8fe00640 1 -- 172.21.15.43:0/1032658144 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e88062340 con 0x7f8e881038e0 2024-03-26T08:33:25.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.778+0000 7f8e86000640 1 -- 172.21.15.43:0/1032658144 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24670 (secure 0 0 0) 0x7f8e7c09b5a0 con 0x7f8e881038e0 2024-03-26T08:33:25.771 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:33:25.771 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":94,"fsid":"88759420-eb49-11ee-95d0-87774f69a715","created":"2024-03-26T08:20:27.817107+0000","modified":"2024-03-26T08:32:38.256877+0000","last_up_change":"2024-03-26T08:30:10.520976+0000","last_in_change":"2024-03-26T08:29:46.394360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:24:54.852427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:31:19.924902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:31:21.323881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T08:31:24.951015+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"28670673-9193-4fa3-9cc0-c1a5a6b27284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6803","nonce":936601146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6805","nonce":936601146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6809","nonce":936601146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6807","nonce":936601146}]},"public_addr":"172.21.15.43:6803/936601146","cluster_addr":"172.21.15.43:6805/936601146","heartbeat_back_addr":"172.21.15.43:6809/936601146","heartbeat_front_addr":"172.21.15.43:6807/936601146","state":["exists","up"]},{"osd":1,"uuid":"fa8ad6b6-277a-42b8-b0f8-b36fc219de2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6801","nonce":3268412328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6803","nonce":3268412328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6807","nonce":3268412328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6805","nonce":3268412328}]},"public_addr":"172.21.15.52:6801/3268412328","cluster_addr":"172.21.15.52:6803/3268412328","heartbeat_back_addr":"172.21.15.52:6807/3268412328","heartbeat_front_addr":"172.21.15.52:6805/3268412328","state":["exists","up"]},{"osd":2,"uuid":"ec0039c3-f9bd-419a-81de-fa4418359808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6801","nonce":2103180908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6803","nonce":2103180908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6807","nonce":2103180908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6805","nonce":2103180908}]},"public_addr":"172.21.15.183:6801/2103180908","cluster_addr":"172.21.15.183:6803/2103180908","heartbeat_back_addr":"172.21.15.183:6807/2103180908","heartbeat_front_addr":"172.21.15.183:6805/2103180908","state":["exists","up"]},{"osd":3,"uuid":"e7dcbf00-6586-4664-a69e-3cb0df8dfd30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6811","nonce":3722846033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6813","nonce":3722846033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6817","nonce":3722846033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6815","nonce":3722846033}]},"public_addr":"172.21.15.43:6811/3722846033","cluster_addr":"172.21.15.43:6813/3722846033","heartbeat_back_addr":"172.21.15.43:6817/3722846033","heartbeat_front_addr":"172.21.15.43:6815/3722846033","state":["exists","up"]},{"osd":4,"uuid":"db089a49-5e98-432d-9687-cdc044114c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6809","nonce":4028436523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6811","nonce":4028436523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6815","nonce":4028436523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6813","nonce":4028436523}]},"public_addr":"172.21.15.52:6809/4028436523","cluster_addr":"172.21.15.52:6811/4028436523","heartbeat_back_addr":"172.21.15.52:6815/4028436523","heartbeat_front_addr":"172.21.15.52:6813/4028436523","state":["exists","up"]},{"osd":5,"uuid":"556ac804-02ea-423b-a599-55baf5be4ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6809","nonce":787132252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6811","nonce":787132252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6815","nonce":787132252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6813","nonce":787132252}]},"public_addr":"172.21.15.183:6809/787132252","cluster_addr":"172.21.15.183:6811/787132252","heartbeat_back_addr":"172.21.15.183:6815/787132252","heartbeat_front_addr":"172.21.15.183:6813/787132252","state":["exists","up"]},{"osd":6,"uuid":"3de2a49d-6cef-4580-9d8a-f90cdfcecd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6819","nonce":4206861158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6821","nonce":4206861158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6825","nonce":4206861158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6823","nonce":4206861158}]},"public_addr":"172.21.15.43:6819/4206861158","cluster_addr":"172.21.15.43:6821/4206861158","heartbeat_back_addr":"172.21.15.43:6825/4206861158","heartbeat_front_addr":"172.21.15.43:6823/4206861158","state":["exists","up"]},{"osd":7,"uuid":"a3b832f7-7d7d-4a62-b679-db6da98275d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6817","nonce":3452064146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6819","nonce":3452064146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6823","nonce":3452064146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6821","nonce":3452064146}]},"public_addr":"172.21.15.52:6817/3452064146","cluster_addr":"172.21.15.52:6819/3452064146","heartbeat_back_addr":"172.21.15.52:6823/3452064146","heartbeat_front_addr":"172.21.15.52:6821/3452064146","state":["exists","up"]},{"osd":8,"uuid":"ddc6b07f-d703-414d-a59a-02316376b13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6817","nonce":1023131725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6819","nonce":1023131725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6823","nonce":1023131725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6821","nonce":1023131725}]},"public_addr":"172.21.15.183:6817/1023131725","cluster_addr":"172.21.15.183:6819/1023131725","heartbeat_back_addr":"172.21.15.183:6823/1023131725","heartbeat_front_addr":"172.21.15.183:6821/1023131725","state":["exists","up"]},{"osd":9,"uuid":"ac16bb0e-6232-45c8-b16a-4d878874c588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6827","nonce":3935195657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6829","nonce":3935195657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6833","nonce":3935195657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6831","nonce":3935195657}]},"public_addr":"172.21.15.43:6827/3935195657","cluster_addr":"172.21.15.43:6829/3935195657","heartbeat_back_addr":"172.21.15.43:6833/3935195657","heartbeat_front_addr":"172.21.15.43:6831/3935195657","state":["exists","up"]},{"osd":10,"uuid":"fd839188-f398-4cff-aa7b-36522ff1a8a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6825","nonce":377628320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6827","nonce":377628320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6831","nonce":377628320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6829","nonce":377628320}]},"public_addr":"172.21.15.52:6825/377628320","cluster_addr":"172.21.15.52:6827/377628320","heartbeat_back_addr":"172.21.15.52:6831/377628320","heartbeat_front_addr":"172.21.15.52:6829/377628320","state":["exists","up"]},{"osd":11,"uuid":"a40282a0-0c64-49ee-b64e-6f3f8119e827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6825","nonce":1373086433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6827","nonce":1373086433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6831","nonce":1373086433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6829","nonce":1373086433}]},"public_addr":"172.21.15.183:6825/1373086433","cluster_addr":"172.21.15.183:6827/1373086433","heartbeat_back_addr":"172.21.15.183:6831/1373086433","heartbeat_front_addr":"172.21.15.183:6829/1373086433","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:23:43.478905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:17.573532+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:50.286610+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:25:27.407663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:03.081294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:36.057416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:12.062013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:47.032271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:21.023201+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:57.364739+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:29:34.039995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:30:08.827577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6801/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/4240775295":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/510203140":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/4251957687":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:6801/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1885244665":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/298320867":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1301092625":"2024-03-27T08:21:36.149092+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:33:25.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.781+0000 7f8e8fe00640 1 -- 172.21.15.43:0/1032658144 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8e700a2880 msgr2=0x7f8e700a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:25.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.781+0000 7f8e8fe00640 1 --2- 172.21.15.43:0/1032658144 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8e700a2880 0x7f8e700a4d40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8e78005fd0 tx=0x7f8e78005850 comp rx=0 tx=0).stop 2024-03-26T08:33:25.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.781+0000 7f8e8fe00640 1 -- 172.21.15.43:0/1032658144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e881038e0 msgr2=0x7f8e88199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:25.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.781+0000 7f8e8fe00640 1 --2- 172.21.15.43:0/1032658144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e881038e0 0x7f8e88199a90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00d950 tx=0x7f8e7c00de10 comp rx=0 tx=0).stop 2024-03-26T08:33:25.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.781+0000 7f8e8fe00640 1 -- 172.21.15.43:0/1032658144 shutdown_connections 2024-03-26T08:33:25.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.781+0000 7f8e8fe00640 1 --2- 172.21.15.43:0/1032658144 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8e700a2880 0x7f8e700a4d40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:25.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.781+0000 7f8e8fe00640 1 --2- 172.21.15.43:0/1032658144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e88105490 0x7f8e881a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:25.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.781+0000 7f8e8fe00640 1 --2- 172.21.15.43:0/1032658144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e88104ad0 0x7f8e88199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:25.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.781+0000 7f8e8fe00640 1 --2- 172.21.15.43:0/1032658144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e881038e0 0x7f8e88199a90 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:25.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.781+0000 7f8e8fe00640 1 -- 172.21.15.43:0/1032658144 >> 172.21.15.43:0/1032658144 conn(0x7f8e880fee90 msgr2=0x7f8e88107ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:25.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.781+0000 7f8e8fe00640 1 -- 172.21.15.43:0/1032658144 shutdown_connections 2024-03-26T08:33:25.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:25.781+0000 7f8e8fe00640 1 -- 172.21.15.43:0/1032658144 wait complete. 2024-03-26T08:33:25.794 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T08:24:54.852427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T08:31:19.924902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T08:31:21.323881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T08:31:24.951015+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T08:33:25.794 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T08:33:26.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.020+0000 7f3dbbe00640 1 -- 172.21.15.43:0/3836540541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dbc109770 msgr2=0x7f3dbc110000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:26.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.020+0000 7f3dbbe00640 1 --2- 172.21.15.43:0/3836540541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dbc109770 0x7f3dbc110000 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3db4009ca0 tx=0x7f3db402c0f0 comp rx=0 tx=0).stop 2024-03-26T08:33:26.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.020+0000 7f3dbbe00640 1 -- 172.21.15.43:0/3836540541 shutdown_connections 2024-03-26T08:33:26.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.020+0000 7f3dbbe00640 1 --2- 172.21.15.43:0/3836540541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dbc109770 0x7f3dbc110000 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:26.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.020+0000 7f3dbbe00640 1 --2- 172.21.15.43:0/3836540541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3dbc108db0 0x7f3dbc109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:26.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.020+0000 7f3dbbe00640 1 --2- 172.21.15.43:0/3836540541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3dbc107bc0 0x7f3dbc107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:26.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.020+0000 7f3dbbe00640 1 -- 172.21.15.43:0/3836540541 >> 172.21.15.43:0/3836540541 conn(0x7f3dbc076b70 msgr2=0x7f3dbc078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:26.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.020+0000 7f3dbbe00640 1 -- 172.21.15.43:0/3836540541 shutdown_connections 2024-03-26T08:33:26.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.020+0000 7f3dbbe00640 1 -- 172.21.15.43:0/3836540541 wait complete. 2024-03-26T08:33:26.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.021+0000 7f3dbbe00640 1 Processor -- start 2024-03-26T08:33:26.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.021+0000 7f3dbbe00640 1 -- start start 2024-03-26T08:33:26.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.022+0000 7f3dbbe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dbc107bc0 0x7f3dbc19ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:26.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.022+0000 7f3dbbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3dbc108db0 0x7f3dbc19e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:26.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.022+0000 7f3dbbe00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3dbc109770 0x7f3dbc1a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:26.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.022+0000 7f3dbbe00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dbc1136f0 con 0x7f3dbc107bc0 2024-03-26T08:33:26.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.022+0000 7f3dbbe00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dbc113570 con 0x7f3dbc109770 2024-03-26T08:33:26.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.022+0000 7f3dbbe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dbc113870 con 0x7f3dbc108db0 2024-03-26T08:33:26.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.022+0000 7f3dbaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dbc107bc0 0x7f3dbc19ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:26.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.022+0000 7f3dbaa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dbc107bc0 0x7f3dbc19ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51976/0 (socket says 172.21.15.43:51976) 2024-03-26T08:33:26.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.022+0000 7f3dbaa00640 1 -- 172.21.15.43:0/1184916008 learned_addr learned my addr 172.21.15.43:0/1184916008 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:26.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.022+0000 7f3dbb400640 1 --2- 172.21.15.43:0/1184916008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3dbc109770 0x7f3dbc1a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:26.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.022+0000 7f3dba000640 1 --2- 172.21.15.43:0/1184916008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3dbc108db0 0x7f3dbc19e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:26.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.023+0000 7f3dbaa00640 1 -- 172.21.15.43:0/1184916008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3dbc108db0 msgr2=0x7f3dbc19e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:26.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.023+0000 7f3dbaa00640 1 --2- 172.21.15.43:0/1184916008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3dbc108db0 0x7f3dbc19e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:26.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.023+0000 7f3dbaa00640 1 -- 172.21.15.43:0/1184916008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3dbc109770 msgr2=0x7f3dbc1a5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:26.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.023+0000 7f3dbaa00640 1 --2- 172.21.15.43:0/1184916008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3dbc109770 0x7f3dbc1a5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:26.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.023+0000 7f3dbaa00640 1 -- 172.21.15.43:0/1184916008 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dbc1a5990 con 0x7f3dbc107bc0 2024-03-26T08:33:26.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.023+0000 7f3dba000640 1 --2- 172.21.15.43:0/1184916008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3dbc108db0 0x7f3dbc19e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:33:26.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.023+0000 7f3dbaa00640 1 --2- 172.21.15.43:0/1184916008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dbc107bc0 0x7f3dbc19ddc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3dac00e9c0 tx=0x7f3dac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:26.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.023+0000 7f3db3400640 1 -- 172.21.15.43:0/1184916008 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac00cd80 con 0x7f3dbc107bc0 2024-03-26T08:33:26.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.023+0000 7f3db3400640 1 -- 172.21.15.43:0/1184916008 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3dac0035f0 con 0x7f3dbc107bc0 2024-03-26T08:33:26.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.023+0000 7f3db3400640 1 -- 172.21.15.43:0/1184916008 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac010690 con 0x7f3dbc107bc0 2024-03-26T08:33:26.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.023+0000 7f3dbbe00640 1 -- 172.21.15.43:0/1184916008 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dbc1a5c80 con 0x7f3dbc107bc0 2024-03-26T08:33:26.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.024+0000 7f3dbbe00640 1 -- 172.21.15.43:0/1184916008 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dbc1a61d0 con 0x7f3dbc107bc0 2024-03-26T08:33:26.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.025+0000 7f3dbbe00640 1 -- 172.21.15.43:0/1184916008 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dbc10eee0 con 0x7f3dbc107bc0 2024-03-26T08:33:26.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.027+0000 7f3db3400640 1 -- 172.21.15.43:0/1184916008 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f3dac0037a0 con 0x7f3dbc107bc0 2024-03-26T08:33:26.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.028+0000 7f3db3400640 1 --2- 172.21.15.43:0/1184916008 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3da00a2800 0x7f3da00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:26.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.028+0000 7f3db3400640 1 -- 172.21.15.43:0/1184916008 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f3dac0cf3b0 con 0x7f3dbc107bc0 2024-03-26T08:33:26.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.029+0000 7f3dba000640 1 --2- 172.21.15.43:0/1184916008 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3da00a2800 0x7f3da00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:26.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.029+0000 7f3dba000640 1 --2- 172.21.15.43:0/1184916008 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3da00a2800 0x7f3da00a4cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc19f280 tx=0x7f3da8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:26.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.031+0000 7f3db3400640 1 -- 172.21.15.43:0/1184916008 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3dac09c0d0 con 0x7f3dbc107bc0 2024-03-26T08:33:26.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:25 smithi043 ceph-mon[31929]: pgmap v438: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:26.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:25 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1032658144' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:33:26.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.235+0000 7f3dbbe00640 1 -- 172.21.15.43:0/1184916008 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3dbc02d2d0 con 0x7f3dbc107bc0 2024-03-26T08:33:26.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.236+0000 7f3db3400640 1 -- 172.21.15.43:0/1184916008 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3dac09b9d0 con 0x7f3dbc107bc0 2024-03-26T08:33:26.229 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-26T08:33:26.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.238+0000 7f3dbbe00640 1 -- 172.21.15.43:0/1184916008 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3da00a2800 msgr2=0x7f3da00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:26.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.238+0000 7f3dbbe00640 1 --2- 172.21.15.43:0/1184916008 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3da00a2800 0x7f3da00a4cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc19f280 tx=0x7f3da8005f90 comp rx=0 tx=0).stop 2024-03-26T08:33:26.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.238+0000 7f3dbbe00640 1 -- 172.21.15.43:0/1184916008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dbc107bc0 msgr2=0x7f3dbc19ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:26.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.238+0000 7f3dbbe00640 1 --2- 172.21.15.43:0/1184916008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dbc107bc0 0x7f3dbc19ddc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3dac00e9c0 tx=0x7f3dac00ee80 comp rx=0 tx=0).stop 2024-03-26T08:33:26.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.239+0000 7f3dbbe00640 1 -- 172.21.15.43:0/1184916008 shutdown_connections 2024-03-26T08:33:26.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.239+0000 7f3dbbe00640 1 --2- 172.21.15.43:0/1184916008 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3da00a2800 0x7f3da00a4cc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:26.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.239+0000 7f3dbbe00640 1 --2- 172.21.15.43:0/1184916008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3dbc109770 0x7f3dbc1a5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:26.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.239+0000 7f3dbbe00640 1 --2- 172.21.15.43:0/1184916008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3dbc108db0 0x7f3dbc19e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:26.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.239+0000 7f3dbbe00640 1 --2- 172.21.15.43:0/1184916008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dbc107bc0 0x7f3dbc19ddc0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:26.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.239+0000 7f3dbbe00640 1 -- 172.21.15.43:0/1184916008 >> 172.21.15.43:0/1184916008 conn(0x7f3dbc076b70 msgr2=0x7f3dbc10c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:26.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.239+0000 7f3dbbe00640 1 -- 172.21.15.43:0/1184916008 shutdown_connections 2024-03-26T08:33:26.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.239+0000 7f3dbbe00640 1 -- 172.21.15.43:0/1184916008 wait complete. 2024-03-26T08:33:26.251 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T08:33:26.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:25 smithi183 ceph-mon[35818]: pgmap v438: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:26.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:25 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1032658144' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:33:26.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:25 smithi052 ceph-mon[35071]: pgmap v438: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:26.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:25 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1032658144' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:33:26.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.493+0000 7f4887200640 1 -- 172.21.15.43:0/4239980282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4888101150 msgr2=0x7f48881015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:26.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.493+0000 7f4887200640 1 --2- 172.21.15.43:0/4239980282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4888101150 0x7f48881015d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4880009ca0 tx=0x7f488002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:33:26.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.493+0000 7f4887200640 1 -- 172.21.15.43:0/4239980282 shutdown_connections 2024-03-26T08:33:26.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.493+0000 7f4887200640 1 --2- 172.21.15.43:0/4239980282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4888101150 0x7f48881015d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:26.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.493+0000 7f4887200640 1 --2- 172.21.15.43:0/4239980282 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48881007b0 0x7f4888100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:26.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.493+0000 7f4887200640 1 --2- 172.21.15.43:0/4239980282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48880ffe50 0x7f4888100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:26.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.493+0000 7f4887200640 1 -- 172.21.15.43:0/4239980282 >> 172.21.15.43:0/4239980282 conn(0x7f48880fba10 msgr2=0x7f48880fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:26.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.493+0000 7f4887200640 1 -- 172.21.15.43:0/4239980282 shutdown_connections 2024-03-26T08:33:26.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.493+0000 7f4887200640 1 -- 172.21.15.43:0/4239980282 wait complete. 2024-03-26T08:33:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.494+0000 7f4887200640 1 Processor -- start 2024-03-26T08:33:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.495+0000 7f4887200640 1 -- start start 2024-03-26T08:33:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.495+0000 7f4887200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48880ffe50 0x7f4888199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.495+0000 7f4887200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48881007b0 0x7f4888199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:26.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.495+0000 7f4887200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4888101150 0x7f48881a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:26.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.495+0000 7f4887200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488810f2a0 con 0x7f48881007b0 2024-03-26T08:33:26.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.495+0000 7f4887200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488810f120 con 0x7f48880ffe50 2024-03-26T08:33:26.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.495+0000 7f4887200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488810f420 con 0x7f4888101150 2024-03-26T08:33:26.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.495+0000 7f4885400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48881007b0 0x7f4888199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:26.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.495+0000 7f4885400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48881007b0 0x7f4888199e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52002/0 (socket says 172.21.15.43:52002) 2024-03-26T08:33:26.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.495+0000 7f4885400640 1 -- 172.21.15.43:0/2129257282 learned_addr learned my addr 172.21.15.43:0/2129257282 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:26.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.496+0000 7f4885e00640 1 --2- 172.21.15.43:0/2129257282 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48880ffe50 0x7f4888199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:26.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.496+0000 7f4885400640 1 -- 172.21.15.43:0/2129257282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4888101150 msgr2=0x7f48881a0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:26.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.496+0000 7f4885400640 1 --2- 172.21.15.43:0/2129257282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4888101150 0x7f48881a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:26.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.496+0000 7f4885400640 1 -- 172.21.15.43:0/2129257282 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48880ffe50 msgr2=0x7f4888199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:26.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.496+0000 7f4885400640 1 --2- 172.21.15.43:0/2129257282 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48880ffe50 0x7f4888199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:26.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.496+0000 7f4885400640 1 -- 172.21.15.43:0/2129257282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48881a1530 con 0x7f48881007b0 2024-03-26T08:33:26.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.496+0000 7f4885400640 1 --2- 172.21.15.43:0/2129257282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48881007b0 0x7f4888199e50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f487400d950 tx=0x7f487400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:26.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.496+0000 7f487ea00640 1 -- 172.21.15.43:0/2129257282 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4874017070 con 0x7f48881007b0 2024-03-26T08:33:26.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.496+0000 7f487ea00640 1 -- 172.21.15.43:0/2129257282 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4874009c30 con 0x7f48881007b0 2024-03-26T08:33:26.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.496+0000 7f487ea00640 1 -- 172.21.15.43:0/2129257282 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487400f620 con 0x7f48881007b0 2024-03-26T08:33:26.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.497+0000 7f4887200640 1 -- 172.21.15.43:0/2129257282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48881a1820 con 0x7f48881007b0 2024-03-26T08:33:26.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.497+0000 7f4887200640 1 -- 172.21.15.43:0/2129257282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48881a1d20 con 0x7f48881007b0 2024-03-26T08:33:26.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.498+0000 7f4887200640 1 -- 172.21.15.43:0/2129257282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f488810ad30 con 0x7f48881007b0 2024-03-26T08:33:26.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.501+0000 7f487ea00640 1 -- 172.21.15.43:0/2129257282 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f48740109a0 con 0x7f48881007b0 2024-03-26T08:33:26.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.505+0000 7f487ea00640 1 --2- 172.21.15.43:0/2129257282 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f486c0a2800 0x7f486c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:26.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.505+0000 7f487ea00640 1 -- 172.21.15.43:0/2129257282 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f48740ceb40 con 0x7f48881007b0 2024-03-26T08:33:26.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.505+0000 7f4885e00640 1 --2- 172.21.15.43:0/2129257282 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f486c0a2800 0x7f486c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:26.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.506+0000 7f4885e00640 1 --2- 172.21.15.43:0/2129257282 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f486c0a2800 0x7f486c0a4cc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4878005fd0 tx=0x7f4878005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:26.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.509+0000 7f487ea00640 1 -- 172.21.15.43:0/2129257282 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4874009de0 con 0x7f48881007b0 2024-03-26T08:33:26.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.724+0000 7f4887200640 1 -- 172.21.15.43:0/2129257282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f488802d290 con 0x7f48881007b0 2024-03-26T08:33:26.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.725+0000 7f487ea00640 1 -- 172.21.15.43:0/2129257282 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f487409b800 con 0x7f48881007b0 2024-03-26T08:33:26.718 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-26T08:33:26.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.728+0000 7f4887200640 1 -- 172.21.15.43:0/2129257282 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f486c0a2800 msgr2=0x7f486c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:26.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.728+0000 7f4887200640 1 --2- 172.21.15.43:0/2129257282 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f486c0a2800 0x7f486c0a4cc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4878005fd0 tx=0x7f4878005850 comp rx=0 tx=0).stop 2024-03-26T08:33:26.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.728+0000 7f4887200640 1 -- 172.21.15.43:0/2129257282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48881007b0 msgr2=0x7f4888199e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:26.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.728+0000 7f4887200640 1 --2- 172.21.15.43:0/2129257282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48881007b0 0x7f4888199e50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f487400d950 tx=0x7f487400de10 comp rx=0 tx=0).stop 2024-03-26T08:33:26.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.728+0000 7f4887200640 1 -- 172.21.15.43:0/2129257282 shutdown_connections 2024-03-26T08:33:26.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.728+0000 7f4887200640 1 --2- 172.21.15.43:0/2129257282 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f486c0a2800 0x7f486c0a4cc0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:26.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.728+0000 7f4887200640 1 --2- 172.21.15.43:0/2129257282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4888101150 0x7f48881a0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:26.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.728+0000 7f4887200640 1 --2- 172.21.15.43:0/2129257282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48881007b0 0x7f4888199e50 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:26.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.728+0000 7f4887200640 1 --2- 172.21.15.43:0/2129257282 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48880ffe50 0x7f4888199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:26.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.728+0000 7f4887200640 1 -- 172.21.15.43:0/2129257282 >> 172.21.15.43:0/2129257282 conn(0x7f48880fba10 msgr2=0x7f48880fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:26.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.728+0000 7f4887200640 1 -- 172.21.15.43:0/2129257282 shutdown_connections 2024-03-26T08:33:26.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.728+0000 7f4887200640 1 -- 172.21.15.43:0/2129257282 wait complete. 2024-03-26T08:33:26.741 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T08:33:26.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.976+0000 7f26e7e00640 1 -- 172.21.15.43:0/1498744293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e8108db0 msgr2=0x7f26e8109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:26.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.976+0000 7f26e7e00640 1 --2- 172.21.15.43:0/1498744293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e8108db0 0x7f26e8109230 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009900 tx=0x7f26d402bc30 comp rx=0 tx=0).stop 2024-03-26T08:33:26.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.976+0000 7f26e7e00640 1 -- 172.21.15.43:0/1498744293 shutdown_connections 2024-03-26T08:33:26.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.976+0000 7f26e7e00640 1 --2- 172.21.15.43:0/1498744293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26e8109770 0x7f26e8110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:26.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.976+0000 7f26e7e00640 1 --2- 172.21.15.43:0/1498744293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e8108db0 0x7f26e8109230 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:26.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.976+0000 7f26e7e00640 1 --2- 172.21.15.43:0/1498744293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26e8107bc0 0x7f26e8107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:26.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.976+0000 7f26e7e00640 1 -- 172.21.15.43:0/1498744293 >> 172.21.15.43:0/1498744293 conn(0x7f26e8076b70 msgr2=0x7f26e8078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:26.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.976+0000 7f26e7e00640 1 -- 172.21.15.43:0/1498744293 shutdown_connections 2024-03-26T08:33:26.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.976+0000 7f26e7e00640 1 -- 172.21.15.43:0/1498744293 wait complete. 2024-03-26T08:33:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.977+0000 7f26e7e00640 1 Processor -- start 2024-03-26T08:33:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.977+0000 7f26e7e00640 1 -- start start 2024-03-26T08:33:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.977+0000 7f26e7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e8107bc0 0x7f26e819de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.977+0000 7f26e7e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26e8108db0 0x7f26e819e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.977+0000 7f26e7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26e8109770 0x7f26e81a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.977+0000 7f26e7e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e8113720 con 0x7f26e8107bc0 2024-03-26T08:33:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.977+0000 7f26e7e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e81135a0 con 0x7f26e8108db0 2024-03-26T08:33:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.977+0000 7f26e7e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e81138a0 con 0x7f26e8109770 2024-03-26T08:33:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.978+0000 7f26e6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e8107bc0 0x7f26e819de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.978+0000 7f26e6000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26e8108db0 0x7f26e819e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.978+0000 7f26e6a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e8107bc0 0x7f26e819de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52026/0 (socket says 172.21.15.43:52026) 2024-03-26T08:33:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.978+0000 7f26e6a00640 1 -- 172.21.15.43:0/2665440247 learned_addr learned my addr 172.21.15.43:0/2665440247 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.978+0000 7f26e7400640 1 --2- 172.21.15.43:0/2665440247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26e8109770 0x7f26e81a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:26.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.979+0000 7f26e6a00640 1 -- 172.21.15.43:0/2665440247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26e8109770 msgr2=0x7f26e81a53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:26.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.979+0000 7f26e6a00640 1 --2- 172.21.15.43:0/2665440247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26e8109770 0x7f26e81a53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.979+0000 7f26e6a00640 1 -- 172.21.15.43:0/2665440247 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26e8108db0 msgr2=0x7f26e819e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.979+0000 7f26e6a00640 1 --2- 172.21.15.43:0/2665440247 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26e8108db0 0x7f26e819e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.979+0000 7f26e6a00640 1 -- 172.21.15.43:0/2665440247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e81a5a20 con 0x7f26e8107bc0 2024-03-26T08:33:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.979+0000 7f26e6a00640 1 --2- 172.21.15.43:0/2665440247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e8107bc0 0x7f26e819de00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f26d800ea80 tx=0x7f26d800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.979+0000 7f26df400640 1 -- 172.21.15.43:0/2665440247 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d800ce40 con 0x7f26e8107bc0 2024-03-26T08:33:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.979+0000 7f26df400640 1 -- 172.21.15.43:0/2665440247 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f26d8011920 con 0x7f26e8107bc0 2024-03-26T08:33:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.979+0000 7f26e7e00640 1 -- 172.21.15.43:0/2665440247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e81a5d10 con 0x7f26e8107bc0 2024-03-26T08:33:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.979+0000 7f26df400640 1 -- 172.21.15.43:0/2665440247 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d801aaa0 con 0x7f26e8107bc0 2024-03-26T08:33:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.980+0000 7f26e7e00640 1 -- 172.21.15.43:0/2665440247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e81a6180 con 0x7f26e8107bc0 2024-03-26T08:33:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.981+0000 7f26e7e00640 1 -- 172.21.15.43:0/2665440247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e810ef00 con 0x7f26e8107bc0 2024-03-26T08:33:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.983+0000 7f26df400640 1 -- 172.21.15.43:0/2665440247 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f26d8011ad0 con 0x7f26e8107bc0 2024-03-26T08:33:26.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.984+0000 7f26df400640 1 --2- 172.21.15.43:0/2665440247 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f26cc0a2800 0x7f26cc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:26.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.984+0000 7f26df400640 1 -- 172.21.15.43:0/2665440247 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f26d80cf2e0 con 0x7f26e8107bc0 2024-03-26T08:33:26.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.984+0000 7f26e6000640 1 --2- 172.21.15.43:0/2665440247 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f26cc0a2800 0x7f26cc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:26.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.985+0000 7f26e6000640 1 --2- 172.21.15.43:0/2665440247 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f26cc0a2800 0x7f26cc0a4cc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f26d4000c00 tx=0x7f26d402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:26.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:26.989+0000 7f26df400640 1 -- 172.21.15.43:0/2665440247 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f26d8016030 con 0x7f26e8107bc0 2024-03-26T08:33:27.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:26 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1184916008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:33:27.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:26 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2129257282' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:33:27.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.191+0000 7f26e7e00640 1 -- 172.21.15.43:0/2665440247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f26e802d430 con 0x7f26e8107bc0 2024-03-26T08:33:27.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.191+0000 7f26df400640 1 -- 172.21.15.43:0/2665440247 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f26d809c000 con 0x7f26e8107bc0 2024-03-26T08:33:27.184 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-26T08:33:27.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.194+0000 7f26e7e00640 1 -- 172.21.15.43:0/2665440247 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f26cc0a2800 msgr2=0x7f26cc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:27.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.194+0000 7f26e7e00640 1 --2- 172.21.15.43:0/2665440247 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f26cc0a2800 0x7f26cc0a4cc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f26d4000c00 tx=0x7f26d402c7e0 comp rx=0 tx=0).stop 2024-03-26T08:33:27.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.194+0000 7f26e7e00640 1 -- 172.21.15.43:0/2665440247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e8107bc0 msgr2=0x7f26e819de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:27.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.194+0000 7f26e7e00640 1 --2- 172.21.15.43:0/2665440247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e8107bc0 0x7f26e819de00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f26d800ea80 tx=0x7f26d800ef40 comp rx=0 tx=0).stop 2024-03-26T08:33:27.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.195+0000 7f26e7e00640 1 -- 172.21.15.43:0/2665440247 shutdown_connections 2024-03-26T08:33:27.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.195+0000 7f26e7e00640 1 --2- 172.21.15.43:0/2665440247 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f26cc0a2800 0x7f26cc0a4cc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:27.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.195+0000 7f26e7e00640 1 --2- 172.21.15.43:0/2665440247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26e8109770 0x7f26e81a53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:27.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.195+0000 7f26e7e00640 1 --2- 172.21.15.43:0/2665440247 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26e8108db0 0x7f26e819e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:27.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.195+0000 7f26e7e00640 1 --2- 172.21.15.43:0/2665440247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e8107bc0 0x7f26e819de00 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:27.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.195+0000 7f26e7e00640 1 -- 172.21.15.43:0/2665440247 >> 172.21.15.43:0/2665440247 conn(0x7f26e8076b70 msgr2=0x7f26e810c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:27.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.195+0000 7f26e7e00640 1 -- 172.21.15.43:0/2665440247 shutdown_connections 2024-03-26T08:33:27.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.195+0000 7f26e7e00640 1 -- 172.21.15.43:0/2665440247 wait complete. 2024-03-26T08:33:27.209 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T08:33:27.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:26 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1184916008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:33:27.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:26 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2129257282' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:33:27.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:26 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1184916008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:33:27.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:26 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2129257282' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:33:27.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.446+0000 7f0255600640 1 -- 172.21.15.43:0/283764867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0250107bc0 msgr2=0x7f0250107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.446+0000 7f0255600640 1 --2- 172.21.15.43:0/283764867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0250107bc0 0x7f0250107fe0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0240009ca0 tx=0x7f024002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:33:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.446+0000 7f0255600640 1 -- 172.21.15.43:0/283764867 shutdown_connections 2024-03-26T08:33:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.446+0000 7f0255600640 1 --2- 172.21.15.43:0/283764867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0250109770 0x7f0250110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.446+0000 7f0255600640 1 --2- 172.21.15.43:0/283764867 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0250108db0 0x7f0250109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.446+0000 7f0255600640 1 --2- 172.21.15.43:0/283764867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0250107bc0 0x7f0250107fe0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.446+0000 7f0255600640 1 -- 172.21.15.43:0/283764867 >> 172.21.15.43:0/283764867 conn(0x7f0250076b70 msgr2=0x7f0250078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.446+0000 7f0255600640 1 -- 172.21.15.43:0/283764867 shutdown_connections 2024-03-26T08:33:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.446+0000 7f0255600640 1 -- 172.21.15.43:0/283764867 wait complete. 2024-03-26T08:33:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.447+0000 7f0255600640 1 Processor -- start 2024-03-26T08:33:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.447+0000 7f0255600640 1 -- start start 2024-03-26T08:33:27.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.447+0000 7f0255600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0250107bc0 0x7f025019ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:27.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.447+0000 7f0255600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0250108db0 0x7f025019e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:27.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.447+0000 7f0255600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0250109770 0x7f02501a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:27.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.447+0000 7f0255600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02501136f0 con 0x7f0250107bc0 2024-03-26T08:33:27.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.447+0000 7f0255600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0250113570 con 0x7f0250108db0 2024-03-26T08:33:27.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.447+0000 7f0255600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0250113870 con 0x7f0250109770 2024-03-26T08:33:27.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.448+0000 7f024fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0250107bc0 0x7f025019ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:27.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.448+0000 7f024fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0250107bc0 0x7f025019ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52052/0 (socket says 172.21.15.43:52052) 2024-03-26T08:33:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.448+0000 7f024fe00640 1 -- 172.21.15.43:0/2579369530 learned_addr learned my addr 172.21.15.43:0/2579369530 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.448+0000 7f0247400640 1 --2- 172.21.15.43:0/2579369530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0250108db0 0x7f025019e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.448+0000 7f0254c00640 1 --2- 172.21.15.43:0/2579369530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0250109770 0x7f02501a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.448+0000 7f024fe00640 1 -- 172.21.15.43:0/2579369530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0250109770 msgr2=0x7f02501a53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.448+0000 7f024fe00640 1 --2- 172.21.15.43:0/2579369530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0250109770 0x7f02501a53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.448+0000 7f024fe00640 1 -- 172.21.15.43:0/2579369530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0250108db0 msgr2=0x7f025019e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.448+0000 7f024fe00640 1 --2- 172.21.15.43:0/2579369530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0250108db0 0x7f025019e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.448+0000 7f024fe00640 1 -- 172.21.15.43:0/2579369530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02501a5a10 con 0x7f0250107bc0 2024-03-26T08:33:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.448+0000 7f0247400640 1 --2- 172.21.15.43:0/2579369530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0250108db0 0x7f025019e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:33:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.448+0000 7f0254c00640 1 --2- 172.21.15.43:0/2579369530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0250109770 0x7f02501a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:33:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.449+0000 7f024fe00640 1 --2- 172.21.15.43:0/2579369530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0250107bc0 0x7f025019ddf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f024002c700 tx=0x7f02400328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.449+0000 7f024d600640 1 -- 172.21.15.43:0/2579369530 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0240041020 con 0x7f0250107bc0 2024-03-26T08:33:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.449+0000 7f024d600640 1 -- 172.21.15.43:0/2579369530 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0240003e20 con 0x7f0250107bc0 2024-03-26T08:33:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.449+0000 7f024d600640 1 -- 172.21.15.43:0/2579369530 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0240039040 con 0x7f0250107bc0 2024-03-26T08:33:27.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.449+0000 7f0255600640 1 -- 172.21.15.43:0/2579369530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02501a5ca0 con 0x7f0250107bc0 2024-03-26T08:33:27.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.449+0000 7f0255600640 1 -- 172.21.15.43:0/2579369530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02501a6140 con 0x7f0250107bc0 2024-03-26T08:33:27.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.450+0000 7f0255600640 1 -- 172.21.15.43:0/2579369530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025010ef50 con 0x7f0250107bc0 2024-03-26T08:33:27.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.452+0000 7f024d600640 1 -- 172.21.15.43:0/2579369530 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0240044020 con 0x7f0250107bc0 2024-03-26T08:33:27.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.454+0000 7f024d600640 1 --2- 172.21.15.43:0/2579369530 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f02340a2800 0x7f02340a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.454+0000 7f024d600640 1 -- 172.21.15.43:0/2579369530 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f02400f1240 con 0x7f0250107bc0 2024-03-26T08:33:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.454+0000 7f0247400640 1 --2- 172.21.15.43:0/2579369530 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f02340a2800 0x7f02340a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.455+0000 7f0247400640 1 --2- 172.21.15.43:0/2579369530 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f02340a2800 0x7f02340a4cc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f025019f300 tx=0x7f0238005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:27.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.458+0000 7f024d600640 1 -- 172.21.15.43:0/2579369530 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f024004b020 con 0x7f0250107bc0 2024-03-26T08:33:27.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.663+0000 7f0255600640 1 -- 172.21.15.43:0/2579369530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f02500622a0 con 0x7f0250107bc0 2024-03-26T08:33:27.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.664+0000 7f024d600640 1 -- 172.21.15.43:0/2579369530 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7f024003b2f0 con 0x7f0250107bc0 2024-03-26T08:33:27.657 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-26T08:33:27.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.666+0000 7f0255600640 1 -- 172.21.15.43:0/2579369530 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f02340a2800 msgr2=0x7f02340a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:27.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.666+0000 7f0255600640 1 --2- 172.21.15.43:0/2579369530 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f02340a2800 0x7f02340a4cc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f025019f300 tx=0x7f0238005e50 comp rx=0 tx=0).stop 2024-03-26T08:33:27.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.666+0000 7f0255600640 1 -- 172.21.15.43:0/2579369530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0250107bc0 msgr2=0x7f025019ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:27.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.666+0000 7f0255600640 1 --2- 172.21.15.43:0/2579369530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0250107bc0 0x7f025019ddf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f024002c700 tx=0x7f02400328a0 comp rx=0 tx=0).stop 2024-03-26T08:33:27.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.666+0000 7f0255600640 1 -- 172.21.15.43:0/2579369530 shutdown_connections 2024-03-26T08:33:27.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.666+0000 7f0255600640 1 --2- 172.21.15.43:0/2579369530 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f02340a2800 0x7f02340a4cc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:27.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.666+0000 7f0255600640 1 --2- 172.21.15.43:0/2579369530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0250109770 0x7f02501a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:27.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.666+0000 7f0255600640 1 --2- 172.21.15.43:0/2579369530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0250108db0 0x7f025019e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:27.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.666+0000 7f0255600640 1 --2- 172.21.15.43:0/2579369530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0250107bc0 0x7f025019ddf0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:27.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.667+0000 7f0255600640 1 -- 172.21.15.43:0/2579369530 >> 172.21.15.43:0/2579369530 conn(0x7f0250076b70 msgr2=0x7f025010c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:27.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.667+0000 7f0255600640 1 -- 172.21.15.43:0/2579369530 shutdown_connections 2024-03-26T08:33:27.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.667+0000 7f0255600640 1 -- 172.21.15.43:0/2579369530 wait complete. 2024-03-26T08:33:27.681 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:33:27.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.904+0000 7f57adc00640 1 -- 172.21.15.43:0/2149703123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57a81052a0 msgr2=0x7f57a810bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:27.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.904+0000 7f57adc00640 1 --2- 172.21.15.43:0/2149703123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57a81052a0 0x7f57a810bb30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f57a0009ca0 tx=0x7f57a002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:33:27.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.904+0000 7f57adc00640 1 -- 172.21.15.43:0/2149703123 shutdown_connections 2024-03-26T08:33:27.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.904+0000 7f57adc00640 1 --2- 172.21.15.43:0/2149703123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57a81052a0 0x7f57a810bb30 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:27.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.904+0000 7f57adc00640 1 --2- 172.21.15.43:0/2149703123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57a81048e0 0x7f57a8104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:27.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.904+0000 7f57adc00640 1 --2- 172.21.15.43:0/2149703123 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57a81036f0 0x7f57a8103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:27.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.904+0000 7f57adc00640 1 -- 172.21.15.43:0/2149703123 >> 172.21.15.43:0/2149703123 conn(0x7f57a80fec80 msgr2=0x7f57a81010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:27.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.904+0000 7f57adc00640 1 -- 172.21.15.43:0/2149703123 shutdown_connections 2024-03-26T08:33:27.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.904+0000 7f57adc00640 1 -- 172.21.15.43:0/2149703123 wait complete. 2024-03-26T08:33:27.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.905+0000 7f57adc00640 1 Processor -- start 2024-03-26T08:33:27.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.905+0000 7f57adc00640 1 -- start start 2024-03-26T08:33:27.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.906+0000 7f57adc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57a81036f0 0x7f57a8199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:27.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.906+0000 7f57adc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57a81048e0 0x7f57a8199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:27.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.906+0000 7f57adc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57a81052a0 0x7f57a81a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:27.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.906+0000 7f57adc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a810f200 con 0x7f57a81052a0 2024-03-26T08:33:27.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.906+0000 7f57adc00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a810f080 con 0x7f57a81036f0 2024-03-26T08:33:27.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.906+0000 7f57adc00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a810f380 con 0x7f57a81048e0 2024-03-26T08:33:27.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.906+0000 7f57ad200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57a81052a0 0x7f57a81a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:27.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.906+0000 7f57ad200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57a81052a0 0x7f57a81a0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52060/0 (socket says 172.21.15.43:52060) 2024-03-26T08:33:27.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.906+0000 7f57ad200640 1 -- 172.21.15.43:0/1168240367 learned_addr learned my addr 172.21.15.43:0/1168240367 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:27.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.906+0000 7f57a7e00640 1 --2- 172.21.15.43:0/1168240367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57a81036f0 0x7f57a8199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:27.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.906+0000 7f57ad200640 1 -- 172.21.15.43:0/1168240367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57a81048e0 msgr2=0x7f57a8199e40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-26T08:33:27.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.906+0000 7f57ad200640 1 --2- 172.21.15.43:0/1168240367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57a81048e0 0x7f57a8199e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:27.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.906+0000 7f57ad200640 1 -- 172.21.15.43:0/1168240367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57a81036f0 msgr2=0x7f57a8199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:27.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.906+0000 7f57ad200640 1 --2- 172.21.15.43:0/1168240367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57a81036f0 0x7f57a8199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:27.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.906+0000 7f57ad200640 1 -- 172.21.15.43:0/1168240367 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a81a14d0 con 0x7f57a81052a0 2024-03-26T08:33:27.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.907+0000 7f57ad200640 1 --2- 172.21.15.43:0/1168240367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57a81052a0 0x7f57a81a0e60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f57a0032380 tx=0x7f57a0032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:27.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.907+0000 7f57a4c00640 1 -- 172.21.15.43:0/1168240367 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a0041020 con 0x7f57a81052a0 2024-03-26T08:33:27.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.907+0000 7f57a4c00640 1 -- 172.21.15.43:0/1168240367 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f57a0003e20 con 0x7f57a81052a0 2024-03-26T08:33:27.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.907+0000 7f57a4c00640 1 -- 172.21.15.43:0/1168240367 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a0039040 con 0x7f57a81052a0 2024-03-26T08:33:27.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.907+0000 7f57adc00640 1 -- 172.21.15.43:0/1168240367 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a81a1760 con 0x7f57a81052a0 2024-03-26T08:33:27.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.907+0000 7f57adc00640 1 -- 172.21.15.43:0/1168240367 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a81a1ca0 con 0x7f57a81052a0 2024-03-26T08:33:27.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.909+0000 7f57adc00640 1 -- 172.21.15.43:0/1168240367 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a810aa30 con 0x7f57a81052a0 2024-03-26T08:33:27.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.910+0000 7f57a4c00640 1 -- 172.21.15.43:0/1168240367 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f57a0044020 con 0x7f57a81052a0 2024-03-26T08:33:27.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.913+0000 7f57a4c00640 1 --2- 172.21.15.43:0/1168240367 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f578c0a2880 0x7f578c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:27.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.913+0000 7f57a4c00640 1 -- 172.21.15.43:0/1168240367 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f57a00f1120 con 0x7f57a81052a0 2024-03-26T08:33:27.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.913+0000 7f57a7e00640 1 --2- 172.21.15.43:0/1168240367 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f578c0a2880 0x7f578c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:27.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.915+0000 7f57a7e00640 1 --2- 172.21.15.43:0/1168240367 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f578c0a2880 0x7f578c0a4d40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5798005fd0 tx=0x7f5798005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:27.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:27.917+0000 7f57a4c00640 1 -- 172.21.15.43:0/1168240367 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f57a004b020 con 0x7f57a81052a0 2024-03-26T08:33:28.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:27 smithi043 ceph-mon[31929]: pgmap v439: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:33:28.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:27 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2665440247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:33:28.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:27 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2579369530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T08:33:28.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.160+0000 7f57adc00640 1 -- 172.21.15.43:0/1168240367 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57a802d070 con 0x7f57a81052a0 2024-03-26T08:33:28.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.161+0000 7f57a4c00640 1 -- 172.21.15.43:0/1168240367 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11331 (secure 0 0 0) 0x7f57a003b2f0 con 0x7f57a81052a0 2024-03-26T08:33:28.154 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:33:28.154 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1452206807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1452206807},{"type":"v1","addr":"172.21.15.43:6835","nonce":1452206807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3151775467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3151775467},{"type":"v1","addr":"172.21.15.43:6837","nonce":3151775467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/1284655682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":1284655682},{"type":"v1","addr":"172.21.15.43:6839","nonce":1284655682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3081085787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3081085787},{"type":"v1","addr":"172.21.15.43:6841","nonce":3081085787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1842295306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1842295306},{"type":"v1","addr":"172.21.15.183:6835","nonce":1842295306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2267781666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2267781666},{"type":"v1","addr":"172.21.15.183:6837","nonce":2267781666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3572429948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3572429948},{"type":"v1","addr":"172.21.15.183:6839","nonce":3572429948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34448,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/1108327179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108327179},{"type":"v1","addr":"172.21.15.52:6833","nonce":1108327179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34487,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/742057410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":742057410},{"type":"v1","addr":"172.21.15.52:6835","nonce":742057410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:31:21.819689+0000","modified":"2024-03-26T08:31:48.080233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34517":{"gid":34517,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/748239918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":748239918},{"type":"v1","addr":"172.21.15.52:6837","nonce":748239918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34559":{"gid":34559,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.52:6839/2551965101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2551965101},{"type":"v1","addr":"172.21.15.52:6839","nonce":2551965101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34559,"qdb_cluster":[34559]},"id":1}]} 2024-03-26T08:33:28.154 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 21 2024-03-26T08:33:28.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.163+0000 7f57adc00640 1 -- 172.21.15.43:0/1168240367 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f578c0a2880 msgr2=0x7f578c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:28.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.163+0000 7f57adc00640 1 --2- 172.21.15.43:0/1168240367 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f578c0a2880 0x7f578c0a4d40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5798005fd0 tx=0x7f5798005850 comp rx=0 tx=0).stop 2024-03-26T08:33:28.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.163+0000 7f57adc00640 1 -- 172.21.15.43:0/1168240367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57a81052a0 msgr2=0x7f57a81a0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:28.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.163+0000 7f57adc00640 1 --2- 172.21.15.43:0/1168240367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57a81052a0 0x7f57a81a0e60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f57a0032380 tx=0x7f57a0032820 comp rx=0 tx=0).stop 2024-03-26T08:33:28.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.163+0000 7f57adc00640 1 -- 172.21.15.43:0/1168240367 shutdown_connections 2024-03-26T08:33:28.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.163+0000 7f57adc00640 1 --2- 172.21.15.43:0/1168240367 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f578c0a2880 0x7f578c0a4d40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:28.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.163+0000 7f57adc00640 1 --2- 172.21.15.43:0/1168240367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57a81052a0 0x7f57a81a0e60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:28.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.163+0000 7f57adc00640 1 --2- 172.21.15.43:0/1168240367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57a81048e0 0x7f57a8199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:28.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.163+0000 7f57adc00640 1 --2- 172.21.15.43:0/1168240367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57a81036f0 0x7f57a8199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:28.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.163+0000 7f57adc00640 1 -- 172.21.15.43:0/1168240367 >> 172.21.15.43:0/1168240367 conn(0x7f57a80fec80 msgr2=0x7f57a8107c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:28.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.163+0000 7f57adc00640 1 -- 172.21.15.43:0/1168240367 shutdown_connections 2024-03-26T08:33:28.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.164+0000 7f57adc00640 1 -- 172.21.15.43:0/1168240367 wait complete. 2024-03-26T08:33:28.176 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:33:28.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:27 smithi183 ceph-mon[35818]: pgmap v439: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:33:28.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:27 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2665440247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:33:28.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:27 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2579369530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T08:33:28.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:27 smithi052 ceph-mon[35071]: pgmap v439: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:33:28.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:27 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2665440247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:33:28.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:27 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2579369530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T08:33:28.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.404+0000 7fb8acc00640 1 -- 172.21.15.43:0/3448540305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a81026d0 msgr2=0x7fb8a8102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:28.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.404+0000 7fb8acc00640 1 --2- 172.21.15.43:0/3448540305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a81026d0 0x7fb8a8102b50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb890009900 tx=0x7fb89002bc30 comp rx=0 tx=0).stop 2024-03-26T08:33:28.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.404+0000 7fb8acc00640 1 -- 172.21.15.43:0/3448540305 shutdown_connections 2024-03-26T08:33:28.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.404+0000 7fb8acc00640 1 --2- 172.21.15.43:0/3448540305 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8a8103090 0x7fb8a8109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:28.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.404+0000 7fb8acc00640 1 --2- 172.21.15.43:0/3448540305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a81026d0 0x7fb8a8102b50 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:28.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.404+0000 7fb8acc00640 1 --2- 172.21.15.43:0/3448540305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8a81014e0 0x7fb8a8101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:28.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.404+0000 7fb8acc00640 1 -- 172.21.15.43:0/3448540305 >> 172.21.15.43:0/3448540305 conn(0x7fb8a80fca30 msgr2=0x7fb8a80fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:28.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.404+0000 7fb8acc00640 1 -- 172.21.15.43:0/3448540305 shutdown_connections 2024-03-26T08:33:28.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.404+0000 7fb8acc00640 1 -- 172.21.15.43:0/3448540305 wait complete. 2024-03-26T08:33:28.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.405+0000 7fb8acc00640 1 Processor -- start 2024-03-26T08:33:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.406+0000 7fb8acc00640 1 -- start start 2024-03-26T08:33:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.406+0000 7fb8acc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a81014e0 0x7fb8a8075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.406+0000 7fb8acc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8a81026d0 0x7fb8a8073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.406+0000 7fb8acc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8a8103090 0x7fb8a8073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.406+0000 7fb8acc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a810d070 con 0x7fb8a81014e0 2024-03-26T08:33:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.406+0000 7fb8acc00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a810cef0 con 0x7fb8a8103090 2024-03-26T08:33:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.406+0000 7fb8acc00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a810d1f0 con 0x7fb8a81026d0 2024-03-26T08:33:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.406+0000 7fb8a7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a81014e0 0x7fb8a8075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.406+0000 7fb8a7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a81014e0 0x7fb8a8075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52080/0 (socket says 172.21.15.43:52080) 2024-03-26T08:33:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.406+0000 7fb8a7400640 1 -- 172.21.15.43:0/2565854886 learned_addr learned my addr 172.21.15.43:0/2565854886 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.407+0000 7fb89fe00640 1 --2- 172.21.15.43:0/2565854886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8a81026d0 0x7fb8a8073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:28.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.407+0000 7fb8a7e00640 1 --2- 172.21.15.43:0/2565854886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8a8103090 0x7fb8a8073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:28.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.407+0000 7fb8a7400640 1 -- 172.21.15.43:0/2565854886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8a81026d0 msgr2=0x7fb8a8073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:28.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.407+0000 7fb8a7400640 1 --2- 172.21.15.43:0/2565854886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8a81026d0 0x7fb8a8073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:28.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.407+0000 7fb8a7400640 1 -- 172.21.15.43:0/2565854886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8a8103090 msgr2=0x7fb8a8073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:28.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.407+0000 7fb8a7400640 1 --2- 172.21.15.43:0/2565854886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8a8103090 0x7fb8a8073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:28.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.407+0000 7fb8a7400640 1 -- 172.21.15.43:0/2565854886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a8074860 con 0x7fb8a81014e0 2024-03-26T08:33:28.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.407+0000 7fb8a7400640 1 --2- 172.21.15.43:0/2565854886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a81014e0 0x7fb8a8075450 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb89800d770 tx=0x7fb89800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:28.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.407+0000 7fb8a4c00640 1 -- 172.21.15.43:0/2565854886 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb898017070 con 0x7fb8a81014e0 2024-03-26T08:33:28.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.408+0000 7fb8acc00640 1 -- 172.21.15.43:0/2565854886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a81a1850 con 0x7fb8a81014e0 2024-03-26T08:33:28.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.408+0000 7fb8a4c00640 1 -- 172.21.15.43:0/2565854886 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb898009c30 con 0x7fb8a81014e0 2024-03-26T08:33:28.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.408+0000 7fb8a4c00640 1 -- 172.21.15.43:0/2565854886 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb898003d00 con 0x7fb8a81014e0 2024-03-26T08:33:28.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.408+0000 7fb8acc00640 1 -- 172.21.15.43:0/2565854886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a81a1dc0 con 0x7fb8a81014e0 2024-03-26T08:33:28.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.409+0000 7fb8acc00640 1 -- 172.21.15.43:0/2565854886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a8108880 con 0x7fb8a81014e0 2024-03-26T08:33:28.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.411+0000 7fb8a4c00640 1 -- 172.21.15.43:0/2565854886 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb898003ea0 con 0x7fb8a81014e0 2024-03-26T08:33:28.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.412+0000 7fb8a4c00640 1 --2- 172.21.15.43:0/2565854886 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb88c0a2800 0x7fb88c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:28.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.412+0000 7fb8a4c00640 1 -- 172.21.15.43:0/2565854886 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7fb8980cec10 con 0x7fb8a81014e0 2024-03-26T08:33:28.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.413+0000 7fb89fe00640 1 --2- 172.21.15.43:0/2565854886 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb88c0a2800 0x7fb88c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:28.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.413+0000 7fb89fe00640 1 --2- 172.21.15.43:0/2565854886 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb88c0a2800 0x7fb88c0a4cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb890006010 tx=0x7fb890000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:28.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.415+0000 7fb8a4c00640 1 -- 172.21.15.43:0/2565854886 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb89809b930 con 0x7fb8a81014e0 2024-03-26T08:33:28.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.619+0000 7fb8acc00640 1 -- 172.21.15.43:0/2565854886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8a802d300 con 0x7fb8a81014e0 2024-03-26T08:33:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.623+0000 7fb8a4c00640 1 -- 172.21.15.43:0/2565854886 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24670 (secure 0 0 0) 0x7fb898009de0 con 0x7fb8a81014e0 2024-03-26T08:33:28.617 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:33:28.617 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":94,"fsid":"88759420-eb49-11ee-95d0-87774f69a715","created":"2024-03-26T08:20:27.817107+0000","modified":"2024-03-26T08:32:38.256877+0000","last_up_change":"2024-03-26T08:30:10.520976+0000","last_in_change":"2024-03-26T08:29:46.394360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:24:54.852427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:31:19.924902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:31:21.323881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T08:31:24.951015+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"28670673-9193-4fa3-9cc0-c1a5a6b27284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6803","nonce":936601146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6805","nonce":936601146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6809","nonce":936601146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6807","nonce":936601146}]},"public_addr":"172.21.15.43:6803/936601146","cluster_addr":"172.21.15.43:6805/936601146","heartbeat_back_addr":"172.21.15.43:6809/936601146","heartbeat_front_addr":"172.21.15.43:6807/936601146","state":["exists","up"]},{"osd":1,"uuid":"fa8ad6b6-277a-42b8-b0f8-b36fc219de2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6801","nonce":3268412328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6803","nonce":3268412328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6807","nonce":3268412328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6805","nonce":3268412328}]},"public_addr":"172.21.15.52:6801/3268412328","cluster_addr":"172.21.15.52:6803/3268412328","heartbeat_back_addr":"172.21.15.52:6807/3268412328","heartbeat_front_addr":"172.21.15.52:6805/3268412328","state":["exists","up"]},{"osd":2,"uuid":"ec0039c3-f9bd-419a-81de-fa4418359808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6801","nonce":2103180908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6803","nonce":2103180908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6807","nonce":2103180908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6805","nonce":2103180908}]},"public_addr":"172.21.15.183:6801/2103180908","cluster_addr":"172.21.15.183:6803/2103180908","heartbeat_back_addr":"172.21.15.183:6807/2103180908","heartbeat_front_addr":"172.21.15.183:6805/2103180908","state":["exists","up"]},{"osd":3,"uuid":"e7dcbf00-6586-4664-a69e-3cb0df8dfd30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6811","nonce":3722846033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6813","nonce":3722846033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6817","nonce":3722846033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6815","nonce":3722846033}]},"public_addr":"172.21.15.43:6811/3722846033","cluster_addr":"172.21.15.43:6813/3722846033","heartbeat_back_addr":"172.21.15.43:6817/3722846033","heartbeat_front_addr":"172.21.15.43:6815/3722846033","state":["exists","up"]},{"osd":4,"uuid":"db089a49-5e98-432d-9687-cdc044114c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6809","nonce":4028436523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6811","nonce":4028436523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6815","nonce":4028436523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6813","nonce":4028436523}]},"public_addr":"172.21.15.52:6809/4028436523","cluster_addr":"172.21.15.52:6811/4028436523","heartbeat_back_addr":"172.21.15.52:6815/4028436523","heartbeat_front_addr":"172.21.15.52:6813/4028436523","state":["exists","up"]},{"osd":5,"uuid":"556ac804-02ea-423b-a599-55baf5be4ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6809","nonce":787132252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6811","nonce":787132252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6815","nonce":787132252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6813","nonce":787132252}]},"public_addr":"172.21.15.183:6809/787132252","cluster_addr":"172.21.15.183:6811/787132252","heartbeat_back_addr":"172.21.15.183:6815/787132252","heartbeat_front_addr":"172.21.15.183:6813/787132252","state":["exists","up"]},{"osd":6,"uuid":"3de2a49d-6cef-4580-9d8a-f90cdfcecd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6819","nonce":4206861158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6821","nonce":4206861158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6825","nonce":4206861158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6823","nonce":4206861158}]},"public_addr":"172.21.15.43:6819/4206861158","cluster_addr":"172.21.15.43:6821/4206861158","heartbeat_back_addr":"172.21.15.43:6825/4206861158","heartbeat_front_addr":"172.21.15.43:6823/4206861158","state":["exists","up"]},{"osd":7,"uuid":"a3b832f7-7d7d-4a62-b679-db6da98275d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6817","nonce":3452064146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6819","nonce":3452064146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6823","nonce":3452064146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6821","nonce":3452064146}]},"public_addr":"172.21.15.52:6817/3452064146","cluster_addr":"172.21.15.52:6819/3452064146","heartbeat_back_addr":"172.21.15.52:6823/3452064146","heartbeat_front_addr":"172.21.15.52:6821/3452064146","state":["exists","up"]},{"osd":8,"uuid":"ddc6b07f-d703-414d-a59a-02316376b13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6817","nonce":1023131725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6819","nonce":1023131725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6823","nonce":1023131725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6821","nonce":1023131725}]},"public_addr":"172.21.15.183:6817/1023131725","cluster_addr":"172.21.15.183:6819/1023131725","heartbeat_back_addr":"172.21.15.183:6823/1023131725","heartbeat_front_addr":"172.21.15.183:6821/1023131725","state":["exists","up"]},{"osd":9,"uuid":"ac16bb0e-6232-45c8-b16a-4d878874c588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6827","nonce":3935195657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6829","nonce":3935195657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6833","nonce":3935195657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6831","nonce":3935195657}]},"public_addr":"172.21.15.43:6827/3935195657","cluster_addr":"172.21.15.43:6829/3935195657","heartbeat_back_addr":"172.21.15.43:6833/3935195657","heartbeat_front_addr":"172.21.15.43:6831/3935195657","state":["exists","up"]},{"osd":10,"uuid":"fd839188-f398-4cff-aa7b-36522ff1a8a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6825","nonce":377628320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6827","nonce":377628320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6831","nonce":377628320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6829","nonce":377628320}]},"public_addr":"172.21.15.52:6825/377628320","cluster_addr":"172.21.15.52:6827/377628320","heartbeat_back_addr":"172.21.15.52:6831/377628320","heartbeat_front_addr":"172.21.15.52:6829/377628320","state":["exists","up"]},{"osd":11,"uuid":"a40282a0-0c64-49ee-b64e-6f3f8119e827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6825","nonce":1373086433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6827","nonce":1373086433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6831","nonce":1373086433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6829","nonce":1373086433}]},"public_addr":"172.21.15.183:6825/1373086433","cluster_addr":"172.21.15.183:6827/1373086433","heartbeat_back_addr":"172.21.15.183:6831/1373086433","heartbeat_front_addr":"172.21.15.183:6829/1373086433","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:23:43.478905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:17.573532+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:50.286610+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:25:27.407663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:03.081294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:36.057416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:12.062013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:47.032271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:21.023201+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:57.364739+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:29:34.039995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:30:08.827577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6801/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/4240775295":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/510203140":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/4251957687":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:6801/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1885244665":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/298320867":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1301092625":"2024-03-27T08:21:36.149092+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:33:28.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.626+0000 7fb8acc00640 1 -- 172.21.15.43:0/2565854886 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb88c0a2800 msgr2=0x7fb88c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:28.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.626+0000 7fb8acc00640 1 --2- 172.21.15.43:0/2565854886 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb88c0a2800 0x7fb88c0a4cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb890006010 tx=0x7fb890000bc0 comp rx=0 tx=0).stop 2024-03-26T08:33:28.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.626+0000 7fb8acc00640 1 -- 172.21.15.43:0/2565854886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a81014e0 msgr2=0x7fb8a8075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:28.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.626+0000 7fb8acc00640 1 --2- 172.21.15.43:0/2565854886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a81014e0 0x7fb8a8075450 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb89800d770 tx=0x7fb89800dc30 comp rx=0 tx=0).stop 2024-03-26T08:33:28.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.626+0000 7fb8acc00640 1 -- 172.21.15.43:0/2565854886 shutdown_connections 2024-03-26T08:33:28.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.627+0000 7fb8acc00640 1 --2- 172.21.15.43:0/2565854886 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb88c0a2800 0x7fb88c0a4cc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:28.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.627+0000 7fb8acc00640 1 --2- 172.21.15.43:0/2565854886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb8a8103090 0x7fb8a8073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:28.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.627+0000 7fb8acc00640 1 --2- 172.21.15.43:0/2565854886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8a81026d0 0x7fb8a8073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:28.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.627+0000 7fb8acc00640 1 --2- 172.21.15.43:0/2565854886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a81014e0 0x7fb8a8075450 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:28.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.627+0000 7fb8acc00640 1 -- 172.21.15.43:0/2565854886 >> 172.21.15.43:0/2565854886 conn(0x7fb8a80fca30 msgr2=0x7fb8a8105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:28.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.627+0000 7fb8acc00640 1 -- 172.21.15.43:0/2565854886 shutdown_connections 2024-03-26T08:33:28.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.627+0000 7fb8acc00640 1 -- 172.21.15.43:0/2565854886 wait complete. 2024-03-26T08:33:28.640 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T08:24:54.852427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T08:31:19.924902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T08:31:21.323881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T08:31:24.951015+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T08:33:28.641 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T08:33:28.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.881+0000 7f0336800640 1 -- 172.21.15.43:0/1873862123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03301049b0 msgr2=0x7f0330104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:28.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.881+0000 7f0336800640 1 --2- 172.21.15.43:0/1873862123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03301049b0 0x7f0330104e30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f031c009900 tx=0x7f031c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:33:28.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.881+0000 7f0336800640 1 -- 172.21.15.43:0/1873862123 shutdown_connections 2024-03-26T08:33:28.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.881+0000 7f0336800640 1 --2- 172.21.15.43:0/1873862123 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0330105370 0x7f033010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:28.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.881+0000 7f0336800640 1 --2- 172.21.15.43:0/1873862123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03301049b0 0x7f0330104e30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:28.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.881+0000 7f0336800640 1 --2- 172.21.15.43:0/1873862123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03301037c0 0x7f0330103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:28.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.881+0000 7f0336800640 1 -- 172.21.15.43:0/1873862123 >> 172.21.15.43:0/1873862123 conn(0x7f03300fed50 msgr2=0x7f0330101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:28.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.881+0000 7f0336800640 1 -- 172.21.15.43:0/1873862123 shutdown_connections 2024-03-26T08:33:28.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.881+0000 7f0336800640 1 -- 172.21.15.43:0/1873862123 wait complete. 2024-03-26T08:33:28.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.882+0000 7f0336800640 1 Processor -- start 2024-03-26T08:33:28.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.882+0000 7f0336800640 1 -- start start 2024-03-26T08:33:28.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.883+0000 7f0336800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03301037c0 0x7f03301999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:28.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.883+0000 7f0336800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03301049b0 0x7f0330199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:28.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.883+0000 7f0336800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0330105370 0x7f03301a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:28.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.883+0000 7f0336800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033010f340 con 0x7f03301037c0 2024-03-26T08:33:28.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.883+0000 7f0336800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033010f1c0 con 0x7f0330105370 2024-03-26T08:33:28.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.883+0000 7f0336800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033010f4c0 con 0x7f03301049b0 2024-03-26T08:33:28.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.883+0000 7f0335400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03301037c0 0x7f03301999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:28.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.883+0000 7f0335400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03301037c0 0x7f03301999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52104/0 (socket says 172.21.15.43:52104) 2024-03-26T08:33:28.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.883+0000 7f0335400640 1 -- 172.21.15.43:0/4051926787 learned_addr learned my addr 172.21.15.43:0/4051926787 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:28.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.883+0000 7f0335e00640 1 --2- 172.21.15.43:0/4051926787 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0330105370 0x7f03301a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:28.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.883+0000 7f0335400640 1 -- 172.21.15.43:0/4051926787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03301049b0 msgr2=0x7f0330199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:28.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.883+0000 7f0335400640 1 --2- 172.21.15.43:0/4051926787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03301049b0 0x7f0330199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:28.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.883+0000 7f0335400640 1 -- 172.21.15.43:0/4051926787 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0330105370 msgr2=0x7f03301a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:28.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.883+0000 7f0335400640 1 --2- 172.21.15.43:0/4051926787 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0330105370 0x7f03301a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:28.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.883+0000 7f0335400640 1 -- 172.21.15.43:0/4051926787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03301a1610 con 0x7f03301037c0 2024-03-26T08:33:28.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.883+0000 7f0335e00640 1 --2- 172.21.15.43:0/4051926787 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0330105370 0x7f03301a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:33:28.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.884+0000 7f0335400640 1 --2- 172.21.15.43:0/4051926787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03301037c0 0x7f03301999f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f032400d950 tx=0x7f032400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:28.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.884+0000 7f032e000640 1 -- 172.21.15.43:0/4051926787 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0324017070 con 0x7f03301037c0 2024-03-26T08:33:28.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.884+0000 7f032e000640 1 -- 172.21.15.43:0/4051926787 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0324009c30 con 0x7f03301037c0 2024-03-26T08:33:28.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.884+0000 7f032e000640 1 -- 172.21.15.43:0/4051926787 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032400f620 con 0x7f03301037c0 2024-03-26T08:33:28.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.884+0000 7f0336800640 1 -- 172.21.15.43:0/4051926787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03301a1900 con 0x7f03301037c0 2024-03-26T08:33:28.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.884+0000 7f0336800640 1 -- 172.21.15.43:0/4051926787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03301a1e00 con 0x7f03301037c0 2024-03-26T08:33:28.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.886+0000 7f0336800640 1 -- 172.21.15.43:0/4051926787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f033010ab60 con 0x7f03301037c0 2024-03-26T08:33:28.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.887+0000 7f032e000640 1 -- 172.21.15.43:0/4051926787 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f032401a070 con 0x7f03301037c0 2024-03-26T08:33:28.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.889+0000 7f032e000640 1 --2- 172.21.15.43:0/4051926787 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f03180a2880 0x7f03180a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:28.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.889+0000 7f032e000640 1 -- 172.21.15.43:0/4051926787 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f03240ce8e0 con 0x7f03301037c0 2024-03-26T08:33:28.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.889+0000 7f0334a00640 1 --2- 172.21.15.43:0/4051926787 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f03180a2880 0x7f03180a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:28.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.890+0000 7f0334a00640 1 --2- 172.21.15.43:0/4051926787 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f03180a2880 0x7f03180a4d40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f031c005c20 tx=0x7f031c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:28.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:28.893+0000 7f032e000640 1 -- 172.21.15.43:0/4051926787 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0324009de0 con 0x7f03301037c0 2024-03-26T08:33:28.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:28 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1168240367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:33:28.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:28 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2565854886' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:33:29.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.095+0000 7f0336800640 1 -- 172.21.15.43:0/4051926787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f03300622d0 con 0x7f03301037c0 2024-03-26T08:33:29.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.096+0000 7f032e000640 1 -- 172.21.15.43:0/4051926787 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f032409b5a0 con 0x7f03301037c0 2024-03-26T08:33:29.089 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-26T08:33:29.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.099+0000 7f0336800640 1 -- 172.21.15.43:0/4051926787 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f03180a2880 msgr2=0x7f03180a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:29.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.099+0000 7f0336800640 1 --2- 172.21.15.43:0/4051926787 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f03180a2880 0x7f03180a4d40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f031c005c20 tx=0x7f031c000bc0 comp rx=0 tx=0).stop 2024-03-26T08:33:29.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.099+0000 7f0336800640 1 -- 172.21.15.43:0/4051926787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03301037c0 msgr2=0x7f03301999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:29.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.099+0000 7f0336800640 1 --2- 172.21.15.43:0/4051926787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03301037c0 0x7f03301999f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f032400d950 tx=0x7f032400de10 comp rx=0 tx=0).stop 2024-03-26T08:33:29.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.099+0000 7f0336800640 1 -- 172.21.15.43:0/4051926787 shutdown_connections 2024-03-26T08:33:29.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.099+0000 7f0336800640 1 --2- 172.21.15.43:0/4051926787 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f03180a2880 0x7f03180a4d40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:29.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.099+0000 7f0336800640 1 --2- 172.21.15.43:0/4051926787 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0330105370 0x7f03301a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:29.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.099+0000 7f0336800640 1 --2- 172.21.15.43:0/4051926787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03301049b0 0x7f0330199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:29.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.099+0000 7f0336800640 1 --2- 172.21.15.43:0/4051926787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03301037c0 0x7f03301999f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:29.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.099+0000 7f0336800640 1 -- 172.21.15.43:0/4051926787 >> 172.21.15.43:0/4051926787 conn(0x7f03300fed50 msgr2=0x7f0330107d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:29.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.099+0000 7f0336800640 1 -- 172.21.15.43:0/4051926787 shutdown_connections 2024-03-26T08:33:29.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.099+0000 7f0336800640 1 -- 172.21.15.43:0/4051926787 wait complete. 2024-03-26T08:33:29.109 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T08:33:29.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:28 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1168240367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:33:29.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:28 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2565854886' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:33:29.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:28 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1168240367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:33:29.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:28 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2565854886' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:33:29.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.347+0000 7f623ac00640 1 -- 172.21.15.43:0/595499638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6234102820 msgr2=0x7f6234102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:29.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.347+0000 7f623ac00640 1 --2- 172.21.15.43:0/595499638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6234102820 0x7f6234102ca0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6220009930 tx=0x7f622002bc20 comp rx=0 tx=0).stop 2024-03-26T08:33:29.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.347+0000 7f623ac00640 1 -- 172.21.15.43:0/595499638 shutdown_connections 2024-03-26T08:33:29.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.347+0000 7f623ac00640 1 --2- 172.21.15.43:0/595499638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62341031e0 0x7f6234109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:29.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.347+0000 7f623ac00640 1 --2- 172.21.15.43:0/595499638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6234102820 0x7f6234102ca0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:29.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.347+0000 7f623ac00640 1 --2- 172.21.15.43:0/595499638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62340ff850 0x7f62340ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:29.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.347+0000 7f623ac00640 1 -- 172.21.15.43:0/595499638 >> 172.21.15.43:0/595499638 conn(0x7f62340fb180 msgr2=0x7f62340fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:29.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.347+0000 7f623ac00640 1 -- 172.21.15.43:0/595499638 shutdown_connections 2024-03-26T08:33:29.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.347+0000 7f623ac00640 1 -- 172.21.15.43:0/595499638 wait complete. 2024-03-26T08:33:29.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.348+0000 7f623ac00640 1 Processor -- start 2024-03-26T08:33:29.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.348+0000 7f623ac00640 1 -- start start 2024-03-26T08:33:29.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.349+0000 7f623ac00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62340ff850 0x7f6234102190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:29.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.349+0000 7f623ac00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6234102820 0x7f62341007d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:29.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.349+0000 7f623ac00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62341031e0 0x7f6234100d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:29.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.349+0000 7f623ac00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623410d0d0 con 0x7f62341031e0 2024-03-26T08:33:29.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.349+0000 7f623ac00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623410cf50 con 0x7f62340ff850 2024-03-26T08:33:29.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.349+0000 7f623ac00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623410d250 con 0x7f6234102820 2024-03-26T08:33:29.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.349+0000 7f623a200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62341031e0 0x7f6234100d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:29.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.349+0000 7f623a200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62341031e0 0x7f6234100d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52128/0 (socket says 172.21.15.43:52128) 2024-03-26T08:33:29.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.349+0000 7f623a200640 1 -- 172.21.15.43:0/605509721 learned_addr learned my addr 172.21.15.43:0/605509721 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:29.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.349+0000 7f6238e00640 1 --2- 172.21.15.43:0/605509721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6234102820 0x7f62341007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:29.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.349+0000 7f6239800640 1 --2- 172.21.15.43:0/605509721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62340ff850 0x7f6234102190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:29.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.350+0000 7f623a200640 1 -- 172.21.15.43:0/605509721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6234102820 msgr2=0x7f62341007d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:29.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.350+0000 7f623a200640 1 --2- 172.21.15.43:0/605509721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6234102820 0x7f62341007d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:29.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.350+0000 7f623a200640 1 -- 172.21.15.43:0/605509721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62340ff850 msgr2=0x7f6234102190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:29.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.350+0000 7f623a200640 1 --2- 172.21.15.43:0/605509721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62340ff850 0x7f6234102190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:29.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.350+0000 7f623a200640 1 -- 172.21.15.43:0/605509721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62341015a0 con 0x7f62341031e0 2024-03-26T08:33:29.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.350+0000 7f623a200640 1 --2- 172.21.15.43:0/605509721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62341031e0 0x7f6234100d10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f622c00cce0 tx=0x7f622c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:29.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.350+0000 7f6232000640 1 -- 172.21.15.43:0/605509721 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c00f450 con 0x7f62341031e0 2024-03-26T08:33:29.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.350+0000 7f623ac00640 1 -- 172.21.15.43:0/605509721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62341a1890 con 0x7f62341031e0 2024-03-26T08:33:29.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.350+0000 7f6232000640 1 -- 172.21.15.43:0/605509721 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f622c00faa0 con 0x7f62341031e0 2024-03-26T08:33:29.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.350+0000 7f6232000640 1 -- 172.21.15.43:0/605509721 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c018b70 con 0x7f62341031e0 2024-03-26T08:33:29.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.350+0000 7f623ac00640 1 -- 172.21.15.43:0/605509721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62341a1d60 con 0x7f62341031e0 2024-03-26T08:33:29.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.352+0000 7f623ac00640 1 -- 172.21.15.43:0/605509721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6234068800 con 0x7f62341031e0 2024-03-26T08:33:29.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.354+0000 7f6232000640 1 -- 172.21.15.43:0/605509721 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f622c018d10 con 0x7f62341031e0 2024-03-26T08:33:29.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.355+0000 7f6232000640 1 --2- 172.21.15.43:0/605509721 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f621c0a2800 0x7f621c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:29.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.355+0000 7f6232000640 1 -- 172.21.15.43:0/605509721 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f622c0cf380 con 0x7f62341031e0 2024-03-26T08:33:29.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.355+0000 7f6239800640 1 --2- 172.21.15.43:0/605509721 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f621c0a2800 0x7f621c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:29.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.356+0000 7f6239800640 1 --2- 172.21.15.43:0/605509721 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f621c0a2800 0x7f621c0a4cc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6228005fd0 tx=0x7f6228005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:29.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.360+0000 7f6232000640 1 -- 172.21.15.43:0/605509721 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f622c014030 con 0x7f62341031e0 2024-03-26T08:33:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.566+0000 7f623ac00640 1 -- 172.21.15.43:0/605509721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f62340622a0 con 0x7f62341031e0 2024-03-26T08:33:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.567+0000 7f6232000640 1 -- 172.21.15.43:0/605509721 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f622c09c040 con 0x7f62341031e0 2024-03-26T08:33:29.560 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-26T08:33:29.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.570+0000 7f623ac00640 1 -- 172.21.15.43:0/605509721 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f621c0a2800 msgr2=0x7f621c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:29.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.570+0000 7f623ac00640 1 --2- 172.21.15.43:0/605509721 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f621c0a2800 0x7f621c0a4cc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6228005fd0 tx=0x7f6228005850 comp rx=0 tx=0).stop 2024-03-26T08:33:29.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.570+0000 7f623ac00640 1 -- 172.21.15.43:0/605509721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62341031e0 msgr2=0x7f6234100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:29.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.570+0000 7f623ac00640 1 --2- 172.21.15.43:0/605509721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62341031e0 0x7f6234100d10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f622c00cce0 tx=0x7f622c00a620 comp rx=0 tx=0).stop 2024-03-26T08:33:29.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.570+0000 7f623ac00640 1 -- 172.21.15.43:0/605509721 shutdown_connections 2024-03-26T08:33:29.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.570+0000 7f623ac00640 1 --2- 172.21.15.43:0/605509721 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f621c0a2800 0x7f621c0a4cc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.570+0000 7f623ac00640 1 --2- 172.21.15.43:0/605509721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62341031e0 0x7f6234100d10 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.570+0000 7f623ac00640 1 --2- 172.21.15.43:0/605509721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6234102820 0x7f62341007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.570+0000 7f623ac00640 1 --2- 172.21.15.43:0/605509721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62340ff850 0x7f6234102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.570+0000 7f623ac00640 1 -- 172.21.15.43:0/605509721 >> 172.21.15.43:0/605509721 conn(0x7f62340fb180 msgr2=0x7f6234105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.570+0000 7f623ac00640 1 -- 172.21.15.43:0/605509721 shutdown_connections 2024-03-26T08:33:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.570+0000 7f623ac00640 1 -- 172.21.15.43:0/605509721 wait complete. 2024-03-26T08:33:29.583 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T08:33:29.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.805+0000 7fbda2000640 1 -- 172.21.15.43:0/3085045713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd9c1037c0 msgr2=0x7fbd9c103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:29.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.805+0000 7fbda2000640 1 --2- 172.21.15.43:0/3085045713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd9c1037c0 0x7fbd9c103be0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbd90009900 tx=0x7fbd9002bc50 comp rx=0 tx=0).stop 2024-03-26T08:33:29.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.805+0000 7fbda2000640 1 -- 172.21.15.43:0/3085045713 shutdown_connections 2024-03-26T08:33:29.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.805+0000 7fbda2000640 1 --2- 172.21.15.43:0/3085045713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd9c105370 0x7fbd9c10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:29.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.805+0000 7fbda2000640 1 --2- 172.21.15.43:0/3085045713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd9c1049b0 0x7fbd9c104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:29.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.805+0000 7fbda2000640 1 --2- 172.21.15.43:0/3085045713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd9c1037c0 0x7fbd9c103be0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:29.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.805+0000 7fbda2000640 1 -- 172.21.15.43:0/3085045713 >> 172.21.15.43:0/3085045713 conn(0x7fbd9c0fed50 msgr2=0x7fbd9c101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:29.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.806+0000 7fbda2000640 1 -- 172.21.15.43:0/3085045713 shutdown_connections 2024-03-26T08:33:29.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.806+0000 7fbda2000640 1 -- 172.21.15.43:0/3085045713 wait complete. 2024-03-26T08:33:29.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.807+0000 7fbda2000640 1 Processor -- start 2024-03-26T08:33:29.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.807+0000 7fbda2000640 1 -- start start 2024-03-26T08:33:29.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.807+0000 7fbda2000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd9c1037c0 0x7fbd9c199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:29.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.807+0000 7fbda2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd9c1049b0 0x7fbd9c199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:29.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.807+0000 7fbda2000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd9c105370 0x7fbd9c1a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:29.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.807+0000 7fbda2000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c10f3b0 con 0x7fbd9c1049b0 2024-03-26T08:33:29.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.807+0000 7fbda2000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c10f230 con 0x7fbd9c105370 2024-03-26T08:33:29.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.807+0000 7fbda2000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c10f530 con 0x7fbd9c1037c0 2024-03-26T08:33:29.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.808+0000 7fbd9be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd9c1049b0 0x7fbd9c199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:29.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.808+0000 7fbd9be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd9c1049b0 0x7fbd9c199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52156/0 (socket says 172.21.15.43:52156) 2024-03-26T08:33:29.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.808+0000 7fbd9be00640 1 -- 172.21.15.43:0/2366064316 learned_addr learned my addr 172.21.15.43:0/2366064316 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:29.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.808+0000 7fbda0c00640 1 --2- 172.21.15.43:0/2366064316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd9c1037c0 0x7fbd9c199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:29.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.808+0000 7fbda1600640 1 --2- 172.21.15.43:0/2366064316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd9c105370 0x7fbd9c1a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:29.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.808+0000 7fbd9be00640 1 -- 172.21.15.43:0/2366064316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd9c1037c0 msgr2=0x7fbd9c199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:29.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.808+0000 7fbd9be00640 1 --2- 172.21.15.43:0/2366064316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd9c1037c0 0x7fbd9c199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:29.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.808+0000 7fbd9be00640 1 -- 172.21.15.43:0/2366064316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd9c105370 msgr2=0x7fbd9c1a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:29.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.808+0000 7fbd9be00640 1 --2- 172.21.15.43:0/2366064316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd9c105370 0x7fbd9c1a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:29.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.808+0000 7fbd9be00640 1 -- 172.21.15.43:0/2366064316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c1a16b0 con 0x7fbd9c1049b0 2024-03-26T08:33:29.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.808+0000 7fbda0c00640 1 --2- 172.21.15.43:0/2366064316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd9c1037c0 0x7fbd9c199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:33:29.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.808+0000 7fbd9be00640 1 --2- 172.21.15.43:0/2366064316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd9c1049b0 0x7fbd9c199fd0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800c890 tx=0x7fbd8800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:29.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.809+0000 7fbd99600640 1 -- 172.21.15.43:0/2366064316 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd88004220 con 0x7fbd9c1049b0 2024-03-26T08:33:29.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.809+0000 7fbd99600640 1 -- 172.21.15.43:0/2366064316 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbd880043c0 con 0x7fbd9c1049b0 2024-03-26T08:33:29.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.809+0000 7fbda2000640 1 -- 172.21.15.43:0/2366064316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd9c1a19a0 con 0x7fbd9c1049b0 2024-03-26T08:33:29.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.809+0000 7fbd99600640 1 -- 172.21.15.43:0/2366064316 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd88004a00 con 0x7fbd9c1049b0 2024-03-26T08:33:29.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.809+0000 7fbda2000640 1 -- 172.21.15.43:0/2366064316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd9c1a1f70 con 0x7fbd9c1049b0 2024-03-26T08:33:29.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.810+0000 7fbda2000640 1 -- 172.21.15.43:0/2366064316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd9c10ab60 con 0x7fbd9c1049b0 2024-03-26T08:33:29.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.813+0000 7fbd99600640 1 -- 172.21.15.43:0/2366064316 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fbd88004ba0 con 0x7fbd9c1049b0 2024-03-26T08:33:29.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.814+0000 7fbd99600640 1 --2- 172.21.15.43:0/2366064316 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fbd840a2800 0x7fbd840a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:29.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.815+0000 7fbd99600640 1 -- 172.21.15.43:0/2366064316 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7fbd880cebc0 con 0x7fbd9c1049b0 2024-03-26T08:33:29.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.815+0000 7fbda0c00640 1 --2- 172.21.15.43:0/2366064316 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fbd840a2800 0x7fbd840a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:29.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.816+0000 7fbda0c00640 1 --2- 172.21.15.43:0/2366064316 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fbd840a2800 0x7fbd840a4cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbd9002c260 tx=0x7fbd90005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:29.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:29.819+0000 7fbd99600640 1 -- 172.21.15.43:0/2366064316 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbd88014030 con 0x7fbd9c1049b0 2024-03-26T08:33:30.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.025+0000 7fbda2000640 1 -- 172.21.15.43:0/2366064316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbd9c0622a0 con 0x7fbd9c1049b0 2024-03-26T08:33:30.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.026+0000 7fbd99600640 1 -- 172.21.15.43:0/2366064316 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbd8809b880 con 0x7fbd9c1049b0 2024-03-26T08:33:30.019 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-26T08:33:30.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.029+0000 7fbda2000640 1 -- 172.21.15.43:0/2366064316 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fbd840a2800 msgr2=0x7fbd840a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:30.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.029+0000 7fbda2000640 1 --2- 172.21.15.43:0/2366064316 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fbd840a2800 0x7fbd840a4cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbd9002c260 tx=0x7fbd90005fd0 comp rx=0 tx=0).stop 2024-03-26T08:33:30.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.029+0000 7fbda2000640 1 -- 172.21.15.43:0/2366064316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd9c1049b0 msgr2=0x7fbd9c199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:30.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.029+0000 7fbda2000640 1 --2- 172.21.15.43:0/2366064316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd9c1049b0 0x7fbd9c199fd0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800c890 tx=0x7fbd8800cd50 comp rx=0 tx=0).stop 2024-03-26T08:33:30.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.029+0000 7fbda2000640 1 -- 172.21.15.43:0/2366064316 shutdown_connections 2024-03-26T08:33:30.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.029+0000 7fbda2000640 1 --2- 172.21.15.43:0/2366064316 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fbd840a2800 0x7fbd840a4cc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:30.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.029+0000 7fbda2000640 1 --2- 172.21.15.43:0/2366064316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd9c105370 0x7fbd9c1a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:30.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.029+0000 7fbda2000640 1 --2- 172.21.15.43:0/2366064316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd9c1049b0 0x7fbd9c199fd0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:30.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.029+0000 7fbda2000640 1 --2- 172.21.15.43:0/2366064316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd9c1037c0 0x7fbd9c199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:30.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.029+0000 7fbda2000640 1 -- 172.21.15.43:0/2366064316 >> 172.21.15.43:0/2366064316 conn(0x7fbd9c0fed50 msgr2=0x7fbd9c107d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:30.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.029+0000 7fbda2000640 1 -- 172.21.15.43:0/2366064316 shutdown_connections 2024-03-26T08:33:30.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.029+0000 7fbda2000640 1 -- 172.21.15.43:0/2366064316 wait complete. 2024-03-26T08:33:30.042 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T08:33:30.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:29 smithi043 ceph-mon[31929]: pgmap v440: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:30.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:29 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/4051926787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:33:30.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:29 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/605509721' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:33:30.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.282+0000 7f5a7ca00640 1 -- 172.21.15.43:0/4088136138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a78101670 msgr2=0x7f5a78101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:30.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.282+0000 7f5a7ca00640 1 --2- 172.21.15.43:0/4088136138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a78101670 0x7f5a78101af0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009900 tx=0x7f5a6402bc40 comp rx=0 tx=0).stop 2024-03-26T08:33:30.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.282+0000 7f5a7ca00640 1 -- 172.21.15.43:0/4088136138 shutdown_connections 2024-03-26T08:33:30.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.282+0000 7f5a7ca00640 1 --2- 172.21.15.43:0/4088136138 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a78102030 0x7f5a781088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:30.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.282+0000 7f5a7ca00640 1 --2- 172.21.15.43:0/4088136138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a78101670 0x7f5a78101af0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:30.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.282+0000 7f5a7ca00640 1 --2- 172.21.15.43:0/4088136138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a78100480 0x7f5a781008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:30.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.282+0000 7f5a7ca00640 1 -- 172.21.15.43:0/4088136138 >> 172.21.15.43:0/4088136138 conn(0x7f5a780fba10 msgr2=0x7f5a780fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:30.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.282+0000 7f5a7ca00640 1 -- 172.21.15.43:0/4088136138 shutdown_connections 2024-03-26T08:33:30.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.282+0000 7f5a7ca00640 1 -- 172.21.15.43:0/4088136138 wait complete. 2024-03-26T08:33:30.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.283+0000 7f5a7ca00640 1 Processor -- start 2024-03-26T08:33:30.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.283+0000 7f5a7ca00640 1 -- start start 2024-03-26T08:33:30.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.284+0000 7f5a7ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a78100480 0x7f5a78199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:30.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.284+0000 7f5a7ca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a78101670 0x7f5a78199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:30.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.284+0000 7f5a7ca00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a78102030 0x7f5a781a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:30.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.284+0000 7f5a7ca00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7810c010 con 0x7f5a78100480 2024-03-26T08:33:30.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.284+0000 7f5a7ca00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7810be90 con 0x7f5a78102030 2024-03-26T08:33:30.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.284+0000 7f5a7ca00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7810c190 con 0x7f5a78101670 2024-03-26T08:33:30.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.284+0000 7f5a77400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a78100480 0x7f5a78199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:30.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.284+0000 7f5a77400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a78100480 0x7f5a78199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52170/0 (socket says 172.21.15.43:52170) 2024-03-26T08:33:30.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.284+0000 7f5a77400640 1 -- 172.21.15.43:0/2999109064 learned_addr learned my addr 172.21.15.43:0/2999109064 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:30.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.284+0000 7f5a76a00640 1 --2- 172.21.15.43:0/2999109064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a78101670 0x7f5a78199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:30.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.284+0000 7f5a77e00640 1 --2- 172.21.15.43:0/2999109064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a78102030 0x7f5a781a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:30.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.285+0000 7f5a77400640 1 -- 172.21.15.43:0/2999109064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a78101670 msgr2=0x7f5a78199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:30.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.285+0000 7f5a77400640 1 --2- 172.21.15.43:0/2999109064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a78101670 0x7f5a78199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:30.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.285+0000 7f5a77400640 1 -- 172.21.15.43:0/2999109064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a78102030 msgr2=0x7f5a781a0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:30.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.285+0000 7f5a77400640 1 --2- 172.21.15.43:0/2999109064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a78102030 0x7f5a781a0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:30.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.285+0000 7f5a77400640 1 -- 172.21.15.43:0/2999109064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a781a1580 con 0x7f5a78100480 2024-03-26T08:33:30.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.285+0000 7f5a77400640 1 --2- 172.21.15.43:0/2999109064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a78100480 0x7f5a78199960 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5a6800c970 tx=0x7f5a6800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:30.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.285+0000 7f5a6fe00640 1 -- 172.21.15.43:0/2999109064 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a68007c10 con 0x7f5a78100480 2024-03-26T08:33:30.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.285+0000 7f5a6fe00640 1 -- 172.21.15.43:0/2999109064 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5a68007db0 con 0x7f5a78100480 2024-03-26T08:33:30.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.285+0000 7f5a7ca00640 1 -- 172.21.15.43:0/2999109064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a781a1870 con 0x7f5a78100480 2024-03-26T08:33:30.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.285+0000 7f5a6fe00640 1 -- 172.21.15.43:0/2999109064 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6800f870 con 0x7f5a78100480 2024-03-26T08:33:30.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.285+0000 7f5a7ca00640 1 -- 172.21.15.43:0/2999109064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a781a1dc0 con 0x7f5a78100480 2024-03-26T08:33:30.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.287+0000 7f5a7ca00640 1 -- 172.21.15.43:0/2999109064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a781a2020 con 0x7f5a78100480 2024-03-26T08:33:30.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.289+0000 7f5a6fe00640 1 -- 172.21.15.43:0/2999109064 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5a6800fa10 con 0x7f5a78100480 2024-03-26T08:33:30.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.290+0000 7f5a6fe00640 1 --2- 172.21.15.43:0/2999109064 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5a5c0a2800 0x7f5a5c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:30.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.290+0000 7f5a6fe00640 1 -- 172.21.15.43:0/2999109064 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f5a680cf520 con 0x7f5a78100480 2024-03-26T08:33:30.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.290+0000 7f5a76a00640 1 --2- 172.21.15.43:0/2999109064 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5a5c0a2800 0x7f5a5c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:30.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.291+0000 7f5a76a00640 1 --2- 172.21.15.43:0/2999109064 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5a5c0a2800 0x7f5a5c0a4cc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5a7819ae70 tx=0x7f5a64000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:30.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.293+0000 7f5a6fe00640 1 -- 172.21.15.43:0/2999109064 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5a6809c300 con 0x7f5a78100480 2024-03-26T08:33:30.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:29 smithi183 ceph-mon[35818]: pgmap v440: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:30.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:29 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/4051926787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:33:30.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:29 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/605509721' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:33:30.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:29 smithi052 ceph-mon[35071]: pgmap v440: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:30.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:29 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/4051926787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:33:30.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:29 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/605509721' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:33:30.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.504+0000 7f5a7ca00640 1 -- 172.21.15.43:0/2999109064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5a781a21f0 con 0x7f5a78100480 2024-03-26T08:33:30.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.505+0000 7f5a6fe00640 1 -- 172.21.15.43:0/2999109064 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5a6809bc00 con 0x7f5a78100480 2024-03-26T08:33:30.498 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-26T08:33:30.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.508+0000 7f5a7ca00640 1 -- 172.21.15.43:0/2999109064 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5a5c0a2800 msgr2=0x7f5a5c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:30.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.508+0000 7f5a7ca00640 1 --2- 172.21.15.43:0/2999109064 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5a5c0a2800 0x7f5a5c0a4cc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5a7819ae70 tx=0x7f5a64000bc0 comp rx=0 tx=0).stop 2024-03-26T08:33:30.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.508+0000 7f5a7ca00640 1 -- 172.21.15.43:0/2999109064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a78100480 msgr2=0x7f5a78199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:30.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.508+0000 7f5a7ca00640 1 --2- 172.21.15.43:0/2999109064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a78100480 0x7f5a78199960 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5a6800c970 tx=0x7f5a6800ce30 comp rx=0 tx=0).stop 2024-03-26T08:33:30.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.508+0000 7f5a7ca00640 1 -- 172.21.15.43:0/2999109064 shutdown_connections 2024-03-26T08:33:30.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.508+0000 7f5a7ca00640 1 --2- 172.21.15.43:0/2999109064 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5a5c0a2800 0x7f5a5c0a4cc0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:30.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.508+0000 7f5a7ca00640 1 --2- 172.21.15.43:0/2999109064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a78102030 0x7f5a781a0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:30.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.508+0000 7f5a7ca00640 1 --2- 172.21.15.43:0/2999109064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a78101670 0x7f5a78199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:30.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.508+0000 7f5a7ca00640 1 --2- 172.21.15.43:0/2999109064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a78100480 0x7f5a78199960 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:30.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.508+0000 7f5a7ca00640 1 -- 172.21.15.43:0/2999109064 >> 172.21.15.43:0/2999109064 conn(0x7f5a780fba10 msgr2=0x7f5a780fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:30.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.508+0000 7f5a7ca00640 1 -- 172.21.15.43:0/2999109064 shutdown_connections 2024-03-26T08:33:30.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.508+0000 7f5a7ca00640 1 -- 172.21.15.43:0/2999109064 wait complete. 2024-03-26T08:33:30.522 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:33:30.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.756+0000 7f3d7ce00640 1 -- 172.21.15.43:0/2417717830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d78107ce0 msgr2=0x7f3d78108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:30.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.756+0000 7f3d7ce00640 1 --2- 172.21.15.43:0/2417717830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d78107ce0 0x7f3d78108100 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3d60009900 tx=0x7f3d6002bc40 comp rx=0 tx=0).stop 2024-03-26T08:33:30.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.757+0000 7f3d7ce00640 1 -- 172.21.15.43:0/2417717830 shutdown_connections 2024-03-26T08:33:30.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.757+0000 7f3d7ce00640 1 --2- 172.21.15.43:0/2417717830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d78109890 0x7f3d78110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:30.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.757+0000 7f3d7ce00640 1 --2- 172.21.15.43:0/2417717830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d78108ed0 0x7f3d78109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:30.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.757+0000 7f3d7ce00640 1 --2- 172.21.15.43:0/2417717830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d78107ce0 0x7f3d78108100 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:30.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.757+0000 7f3d7ce00640 1 -- 172.21.15.43:0/2417717830 >> 172.21.15.43:0/2417717830 conn(0x7f3d78076cb0 msgr2=0x7f3d780790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:30.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.757+0000 7f3d7ce00640 1 -- 172.21.15.43:0/2417717830 shutdown_connections 2024-03-26T08:33:30.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.757+0000 7f3d7ce00640 1 -- 172.21.15.43:0/2417717830 wait complete. 2024-03-26T08:33:30.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.758+0000 7f3d7ce00640 1 Processor -- start 2024-03-26T08:33:30.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.758+0000 7f3d7ce00640 1 -- start start 2024-03-26T08:33:30.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.758+0000 7f3d7ce00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d78107ce0 0x7f3d7819df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:30.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.758+0000 7f3d7ce00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d78108ed0 0x7f3d7819e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:30.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.758+0000 7f3d7ce00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d78109890 0x7f3d781a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:30.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.758+0000 7f3d7ce00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d78113830 con 0x7f3d78107ce0 2024-03-26T08:33:30.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.758+0000 7f3d7ce00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d781136b0 con 0x7f3d78108ed0 2024-03-26T08:33:30.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.758+0000 7f3d7ce00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d781139b0 con 0x7f3d78109890 2024-03-26T08:33:30.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.758+0000 7f3d75e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d78107ce0 0x7f3d7819df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:30.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.758+0000 7f3d75e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d78107ce0 0x7f3d7819df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52188/0 (socket says 172.21.15.43:52188) 2024-03-26T08:33:30.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.758+0000 7f3d75e00640 1 -- 172.21.15.43:0/768312425 learned_addr learned my addr 172.21.15.43:0/768312425 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:30.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.759+0000 7f3d75400640 1 --2- 172.21.15.43:0/768312425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d78108ed0 0x7f3d7819e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:30.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.759+0000 7f3d76800640 1 --2- 172.21.15.43:0/768312425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d78109890 0x7f3d781a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:30.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.759+0000 7f3d75e00640 1 -- 172.21.15.43:0/768312425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d78109890 msgr2=0x7f3d781a54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:30.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.759+0000 7f3d75e00640 1 --2- 172.21.15.43:0/768312425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d78109890 0x7f3d781a54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:30.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.759+0000 7f3d75e00640 1 -- 172.21.15.43:0/768312425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d78108ed0 msgr2=0x7f3d7819e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:30.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.759+0000 7f3d75e00640 1 --2- 172.21.15.43:0/768312425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d78108ed0 0x7f3d7819e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:30.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.759+0000 7f3d75e00640 1 -- 172.21.15.43:0/768312425 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d781a5b50 con 0x7f3d78107ce0 2024-03-26T08:33:30.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.759+0000 7f3d75e00640 1 --2- 172.21.15.43:0/768312425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d78107ce0 0x7f3d7819df30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3d60005ab0 tx=0x7f3d600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:30.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.760+0000 7f3d6ea00640 1 -- 172.21.15.43:0/768312425 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d60043020 con 0x7f3d78107ce0 2024-03-26T08:33:30.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.760+0000 7f3d6ea00640 1 -- 172.21.15.43:0/768312425 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3d600054c0 con 0x7f3d78107ce0 2024-03-26T08:33:30.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.760+0000 7f3d6ea00640 1 -- 172.21.15.43:0/768312425 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d60003d10 con 0x7f3d78107ce0 2024-03-26T08:33:30.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.760+0000 7f3d7ce00640 1 -- 172.21.15.43:0/768312425 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d781a5de0 con 0x7f3d78107ce0 2024-03-26T08:33:30.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.760+0000 7f3d7ce00640 1 -- 172.21.15.43:0/768312425 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d781a61f0 con 0x7f3d78107ce0 2024-03-26T08:33:30.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.761+0000 7f3d7ce00640 1 -- 172.21.15.43:0/768312425 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d7810f050 con 0x7f3d78107ce0 2024-03-26T08:33:30.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.763+0000 7f3d6ea00640 1 -- 172.21.15.43:0/768312425 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f3d60046020 con 0x7f3d78107ce0 2024-03-26T08:33:30.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.764+0000 7f3d6ea00640 1 --2- 172.21.15.43:0/768312425 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3d5c0a2880 0x7f3d5c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:30.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.765+0000 7f3d6ea00640 1 -- 172.21.15.43:0/768312425 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f3d600f11a0 con 0x7f3d78107ce0 2024-03-26T08:33:30.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.765+0000 7f3d75400640 1 --2- 172.21.15.43:0/768312425 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3d5c0a2880 0x7f3d5c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:30.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.766+0000 7f3d75400640 1 --2- 172.21.15.43:0/768312425 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3d5c0a2880 0x7f3d5c0a4d40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3d68005fd0 tx=0x7f3d68005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:30.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:30.769+0000 7f3d6ea00640 1 -- 172.21.15.43:0/768312425 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3d6004d020 con 0x7f3d78107ce0 2024-03-26T08:33:31.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.014+0000 7f3d7ce00640 1 -- 172.21.15.43:0/768312425 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d7802d070 con 0x7f3d78107ce0 2024-03-26T08:33:31.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.015+0000 7f3d6ea00640 1 -- 172.21.15.43:0/768312425 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11331 (secure 0 0 0) 0x7f3d600bde60 con 0x7f3d78107ce0 2024-03-26T08:33:31.008 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:33:31.008 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1452206807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1452206807},{"type":"v1","addr":"172.21.15.43:6835","nonce":1452206807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3151775467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3151775467},{"type":"v1","addr":"172.21.15.43:6837","nonce":3151775467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/1284655682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":1284655682},{"type":"v1","addr":"172.21.15.43:6839","nonce":1284655682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3081085787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3081085787},{"type":"v1","addr":"172.21.15.43:6841","nonce":3081085787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1842295306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1842295306},{"type":"v1","addr":"172.21.15.183:6835","nonce":1842295306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2267781666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2267781666},{"type":"v1","addr":"172.21.15.183:6837","nonce":2267781666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3572429948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3572429948},{"type":"v1","addr":"172.21.15.183:6839","nonce":3572429948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34448,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/1108327179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108327179},{"type":"v1","addr":"172.21.15.52:6833","nonce":1108327179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34487,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/742057410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":742057410},{"type":"v1","addr":"172.21.15.52:6835","nonce":742057410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:31:21.819689+0000","modified":"2024-03-26T08:31:48.080233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34517":{"gid":34517,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/748239918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":748239918},{"type":"v1","addr":"172.21.15.52:6837","nonce":748239918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34559":{"gid":34559,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.52:6839/2551965101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2551965101},{"type":"v1","addr":"172.21.15.52:6839","nonce":2551965101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34559,"qdb_cluster":[34559]},"id":1}]} 2024-03-26T08:33:31.008 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 21 2024-03-26T08:33:31.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.018+0000 7f3d7ce00640 1 -- 172.21.15.43:0/768312425 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3d5c0a2880 msgr2=0x7f3d5c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:31.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.018+0000 7f3d7ce00640 1 --2- 172.21.15.43:0/768312425 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3d5c0a2880 0x7f3d5c0a4d40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3d68005fd0 tx=0x7f3d68005850 comp rx=0 tx=0).stop 2024-03-26T08:33:31.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.018+0000 7f3d7ce00640 1 -- 172.21.15.43:0/768312425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d78107ce0 msgr2=0x7f3d7819df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:31.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.018+0000 7f3d7ce00640 1 --2- 172.21.15.43:0/768312425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d78107ce0 0x7f3d7819df30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3d60005ab0 tx=0x7f3d600042f0 comp rx=0 tx=0).stop 2024-03-26T08:33:31.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.018+0000 7f3d7ce00640 1 -- 172.21.15.43:0/768312425 shutdown_connections 2024-03-26T08:33:31.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.018+0000 7f3d7ce00640 1 --2- 172.21.15.43:0/768312425 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3d5c0a2880 0x7f3d5c0a4d40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:31.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.018+0000 7f3d7ce00640 1 --2- 172.21.15.43:0/768312425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d78109890 0x7f3d781a54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:31.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.018+0000 7f3d7ce00640 1 --2- 172.21.15.43:0/768312425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d78108ed0 0x7f3d7819e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:31.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.018+0000 7f3d7ce00640 1 --2- 172.21.15.43:0/768312425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d78107ce0 0x7f3d7819df30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:31.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.018+0000 7f3d7ce00640 1 -- 172.21.15.43:0/768312425 >> 172.21.15.43:0/768312425 conn(0x7f3d78076cb0 msgr2=0x7f3d7810c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:31.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.019+0000 7f3d7ce00640 1 -- 172.21.15.43:0/768312425 shutdown_connections 2024-03-26T08:33:31.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.019+0000 7f3d7ce00640 1 -- 172.21.15.43:0/768312425 wait complete. 2024-03-26T08:33:31.031 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:33:31.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:30 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2366064316' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:33:31.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:30 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2999109064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T08:33:31.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.268+0000 7f6ce7000640 1 -- 172.21.15.43:0/1452951744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ce01036f0 msgr2=0x7f6ce0103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:31.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.268+0000 7f6ce7000640 1 --2- 172.21.15.43:0/1452951744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ce01036f0 0x7f6ce0103b10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4009900 tx=0x7f6cd402bc30 comp rx=0 tx=0).stop 2024-03-26T08:33:31.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.268+0000 7f6ce7000640 1 -- 172.21.15.43:0/1452951744 shutdown_connections 2024-03-26T08:33:31.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.268+0000 7f6ce7000640 1 --2- 172.21.15.43:0/1452951744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ce01052a0 0x7f6ce010bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:31.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.268+0000 7f6ce7000640 1 --2- 172.21.15.43:0/1452951744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ce01048e0 0x7f6ce0104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:31.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.268+0000 7f6ce7000640 1 --2- 172.21.15.43:0/1452951744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ce01036f0 0x7f6ce0103b10 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:31.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.268+0000 7f6ce7000640 1 -- 172.21.15.43:0/1452951744 >> 172.21.15.43:0/1452951744 conn(0x7f6ce00fec80 msgr2=0x7f6ce01010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:31.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.268+0000 7f6ce7000640 1 -- 172.21.15.43:0/1452951744 shutdown_connections 2024-03-26T08:33:31.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.268+0000 7f6ce7000640 1 -- 172.21.15.43:0/1452951744 wait complete. 2024-03-26T08:33:31.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.269+0000 7f6ce7000640 1 Processor -- start 2024-03-26T08:33:31.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.269+0000 7f6ce7000640 1 -- start start 2024-03-26T08:33:31.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.270+0000 7f6ce7000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ce01036f0 0x7f6ce01998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:31.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.270+0000 7f6ce7000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ce01048e0 0x7f6ce0199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:31.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.270+0000 7f6ce7000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ce01052a0 0x7f6ce01a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:31.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.270+0000 7f6ce7000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce010f220 con 0x7f6ce01052a0 2024-03-26T08:33:31.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.270+0000 7f6ce7000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce010f0a0 con 0x7f6ce01048e0 2024-03-26T08:33:31.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.270+0000 7f6ce7000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce010f3a0 con 0x7f6ce01036f0 2024-03-26T08:33:31.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.270+0000 7f6ce6600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ce01052a0 0x7f6ce01a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:31.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.270+0000 7f6ce6600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ce01052a0 0x7f6ce01a0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52200/0 (socket says 172.21.15.43:52200) 2024-03-26T08:33:31.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.270+0000 7f6ce6600640 1 -- 172.21.15.43:0/2845148707 learned_addr learned my addr 172.21.15.43:0/2845148707 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:31.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.270+0000 7f6ce5c00640 1 --2- 172.21.15.43:0/2845148707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ce01036f0 0x7f6ce01998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:31.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.270+0000 7f6ce5200640 1 --2- 172.21.15.43:0/2845148707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ce01048e0 0x7f6ce0199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:31.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.270+0000 7f6ce6600640 1 -- 172.21.15.43:0/2845148707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ce01036f0 msgr2=0x7f6ce01998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:31.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.270+0000 7f6ce6600640 1 --2- 172.21.15.43:0/2845148707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ce01036f0 0x7f6ce01998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:31.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.270+0000 7f6ce6600640 1 -- 172.21.15.43:0/2845148707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ce01048e0 msgr2=0x7f6ce0199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:31.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.270+0000 7f6ce6600640 1 --2- 172.21.15.43:0/2845148707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ce01048e0 0x7f6ce0199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:31.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.270+0000 7f6ce6600640 1 -- 172.21.15.43:0/2845148707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce01a14f0 con 0x7f6ce01052a0 2024-03-26T08:33:31.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.271+0000 7f6ce6600640 1 --2- 172.21.15.43:0/2845148707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ce01052a0 0x7f6ce01a0e80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6cd800eaa0 tx=0x7f6cd800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:31.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.271+0000 7f6cde000640 1 -- 172.21.15.43:0/2845148707 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd800cdb0 con 0x7f6ce01052a0 2024-03-26T08:33:31.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.271+0000 7f6cde000640 1 -- 172.21.15.43:0/2845148707 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6cd8005020 con 0x7f6ce01052a0 2024-03-26T08:33:31.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.271+0000 7f6ce7000640 1 -- 172.21.15.43:0/2845148707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce01a17e0 con 0x7f6ce01052a0 2024-03-26T08:33:31.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.271+0000 7f6cde000640 1 -- 172.21.15.43:0/2845148707 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd80116d0 con 0x7f6ce01052a0 2024-03-26T08:33:31.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.271+0000 7f6ce7000640 1 -- 172.21.15.43:0/2845148707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce01a1d30 con 0x7f6ce01052a0 2024-03-26T08:33:31.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.272+0000 7f6ce7000640 1 -- 172.21.15.43:0/2845148707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce010aa90 con 0x7f6ce01052a0 2024-03-26T08:33:31.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.274+0000 7f6cde000640 1 -- 172.21.15.43:0/2845148707 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f6cd80051d0 con 0x7f6ce01052a0 2024-03-26T08:33:31.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.276+0000 7f6cde000640 1 --2- 172.21.15.43:0/2845148707 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6cc80a2800 0x7f6cc80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:31.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.276+0000 7f6cde000640 1 -- 172.21.15.43:0/2845148707 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f6cd80cf0a0 con 0x7f6ce01052a0 2024-03-26T08:33:31.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.276+0000 7f6ce5c00640 1 --2- 172.21.15.43:0/2845148707 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6cc80a2800 0x7f6cc80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:31.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.277+0000 7f6ce5c00640 1 --2- 172.21.15.43:0/2845148707 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6cc80a2800 0x7f6cc80a4cc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4006010 tx=0x7f6cd4005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.280+0000 7f6cde000640 1 -- 172.21.15.43:0/2845148707 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6cd8016030 con 0x7f6ce01052a0 2024-03-26T08:33:31.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:30 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2366064316' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:33:31.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:30 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2999109064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T08:33:31.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:30 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2366064316' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:33:31.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:30 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2999109064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T08:33:31.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.489+0000 7f6ce7000640 1 -- 172.21.15.43:0/2845148707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ce00622a0 con 0x7f6ce01052a0 2024-03-26T08:33:31.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.492+0000 7f6cde000640 1 -- 172.21.15.43:0/2845148707 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24670 (secure 0 0 0) 0x7f6cd809bd60 con 0x7f6ce01052a0 2024-03-26T08:33:31.486 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:33:31.486 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":94,"fsid":"88759420-eb49-11ee-95d0-87774f69a715","created":"2024-03-26T08:20:27.817107+0000","modified":"2024-03-26T08:32:38.256877+0000","last_up_change":"2024-03-26T08:30:10.520976+0000","last_in_change":"2024-03-26T08:29:46.394360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:24:54.852427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:31:19.924902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:31:21.323881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T08:31:24.951015+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"28670673-9193-4fa3-9cc0-c1a5a6b27284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6803","nonce":936601146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6805","nonce":936601146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6809","nonce":936601146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6807","nonce":936601146}]},"public_addr":"172.21.15.43:6803/936601146","cluster_addr":"172.21.15.43:6805/936601146","heartbeat_back_addr":"172.21.15.43:6809/936601146","heartbeat_front_addr":"172.21.15.43:6807/936601146","state":["exists","up"]},{"osd":1,"uuid":"fa8ad6b6-277a-42b8-b0f8-b36fc219de2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6801","nonce":3268412328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6803","nonce":3268412328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6807","nonce":3268412328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6805","nonce":3268412328}]},"public_addr":"172.21.15.52:6801/3268412328","cluster_addr":"172.21.15.52:6803/3268412328","heartbeat_back_addr":"172.21.15.52:6807/3268412328","heartbeat_front_addr":"172.21.15.52:6805/3268412328","state":["exists","up"]},{"osd":2,"uuid":"ec0039c3-f9bd-419a-81de-fa4418359808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6801","nonce":2103180908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6803","nonce":2103180908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6807","nonce":2103180908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6805","nonce":2103180908}]},"public_addr":"172.21.15.183:6801/2103180908","cluster_addr":"172.21.15.183:6803/2103180908","heartbeat_back_addr":"172.21.15.183:6807/2103180908","heartbeat_front_addr":"172.21.15.183:6805/2103180908","state":["exists","up"]},{"osd":3,"uuid":"e7dcbf00-6586-4664-a69e-3cb0df8dfd30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6811","nonce":3722846033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6813","nonce":3722846033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6817","nonce":3722846033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6815","nonce":3722846033}]},"public_addr":"172.21.15.43:6811/3722846033","cluster_addr":"172.21.15.43:6813/3722846033","heartbeat_back_addr":"172.21.15.43:6817/3722846033","heartbeat_front_addr":"172.21.15.43:6815/3722846033","state":["exists","up"]},{"osd":4,"uuid":"db089a49-5e98-432d-9687-cdc044114c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6809","nonce":4028436523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6811","nonce":4028436523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6815","nonce":4028436523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6813","nonce":4028436523}]},"public_addr":"172.21.15.52:6809/4028436523","cluster_addr":"172.21.15.52:6811/4028436523","heartbeat_back_addr":"172.21.15.52:6815/4028436523","heartbeat_front_addr":"172.21.15.52:6813/4028436523","state":["exists","up"]},{"osd":5,"uuid":"556ac804-02ea-423b-a599-55baf5be4ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6809","nonce":787132252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6811","nonce":787132252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6815","nonce":787132252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6813","nonce":787132252}]},"public_addr":"172.21.15.183:6809/787132252","cluster_addr":"172.21.15.183:6811/787132252","heartbeat_back_addr":"172.21.15.183:6815/787132252","heartbeat_front_addr":"172.21.15.183:6813/787132252","state":["exists","up"]},{"osd":6,"uuid":"3de2a49d-6cef-4580-9d8a-f90cdfcecd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6819","nonce":4206861158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6821","nonce":4206861158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6825","nonce":4206861158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6823","nonce":4206861158}]},"public_addr":"172.21.15.43:6819/4206861158","cluster_addr":"172.21.15.43:6821/4206861158","heartbeat_back_addr":"172.21.15.43:6825/4206861158","heartbeat_front_addr":"172.21.15.43:6823/4206861158","state":["exists","up"]},{"osd":7,"uuid":"a3b832f7-7d7d-4a62-b679-db6da98275d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6817","nonce":3452064146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6819","nonce":3452064146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6823","nonce":3452064146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6821","nonce":3452064146}]},"public_addr":"172.21.15.52:6817/3452064146","cluster_addr":"172.21.15.52:6819/3452064146","heartbeat_back_addr":"172.21.15.52:6823/3452064146","heartbeat_front_addr":"172.21.15.52:6821/3452064146","state":["exists","up"]},{"osd":8,"uuid":"ddc6b07f-d703-414d-a59a-02316376b13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6817","nonce":1023131725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6819","nonce":1023131725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6823","nonce":1023131725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6821","nonce":1023131725}]},"public_addr":"172.21.15.183:6817/1023131725","cluster_addr":"172.21.15.183:6819/1023131725","heartbeat_back_addr":"172.21.15.183:6823/1023131725","heartbeat_front_addr":"172.21.15.183:6821/1023131725","state":["exists","up"]},{"osd":9,"uuid":"ac16bb0e-6232-45c8-b16a-4d878874c588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6827","nonce":3935195657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6829","nonce":3935195657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6833","nonce":3935195657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6831","nonce":3935195657}]},"public_addr":"172.21.15.43:6827/3935195657","cluster_addr":"172.21.15.43:6829/3935195657","heartbeat_back_addr":"172.21.15.43:6833/3935195657","heartbeat_front_addr":"172.21.15.43:6831/3935195657","state":["exists","up"]},{"osd":10,"uuid":"fd839188-f398-4cff-aa7b-36522ff1a8a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6825","nonce":377628320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6827","nonce":377628320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6831","nonce":377628320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6829","nonce":377628320}]},"public_addr":"172.21.15.52:6825/377628320","cluster_addr":"172.21.15.52:6827/377628320","heartbeat_back_addr":"172.21.15.52:6831/377628320","heartbeat_front_addr":"172.21.15.52:6829/377628320","state":["exists","up"]},{"osd":11,"uuid":"a40282a0-0c64-49ee-b64e-6f3f8119e827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6825","nonce":1373086433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6827","nonce":1373086433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6831","nonce":1373086433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6829","nonce":1373086433}]},"public_addr":"172.21.15.183:6825/1373086433","cluster_addr":"172.21.15.183:6827/1373086433","heartbeat_back_addr":"172.21.15.183:6831/1373086433","heartbeat_front_addr":"172.21.15.183:6829/1373086433","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:23:43.478905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:17.573532+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:50.286610+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:25:27.407663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:03.081294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:36.057416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:12.062013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:47.032271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:21.023201+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:57.364739+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:29:34.039995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:30:08.827577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6801/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/4240775295":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/510203140":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/4251957687":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:6801/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1885244665":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/298320867":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1301092625":"2024-03-27T08:21:36.149092+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:33:31.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.495+0000 7f6ce7000640 1 -- 172.21.15.43:0/2845148707 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6cc80a2800 msgr2=0x7f6cc80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:31.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.495+0000 7f6ce7000640 1 --2- 172.21.15.43:0/2845148707 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6cc80a2800 0x7f6cc80a4cc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4006010 tx=0x7f6cd4005be0 comp rx=0 tx=0).stop 2024-03-26T08:33:31.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.495+0000 7f6ce7000640 1 -- 172.21.15.43:0/2845148707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ce01052a0 msgr2=0x7f6ce01a0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:31.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.495+0000 7f6ce7000640 1 --2- 172.21.15.43:0/2845148707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ce01052a0 0x7f6ce01a0e80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6cd800eaa0 tx=0x7f6cd800ef60 comp rx=0 tx=0).stop 2024-03-26T08:33:31.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.495+0000 7f6ce7000640 1 -- 172.21.15.43:0/2845148707 shutdown_connections 2024-03-26T08:33:31.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.495+0000 7f6ce7000640 1 --2- 172.21.15.43:0/2845148707 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6cc80a2800 0x7f6cc80a4cc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:31.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.495+0000 7f6ce7000640 1 --2- 172.21.15.43:0/2845148707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ce01052a0 0x7f6ce01a0e80 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:31.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.495+0000 7f6ce7000640 1 --2- 172.21.15.43:0/2845148707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ce01048e0 0x7f6ce0199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:31.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.495+0000 7f6ce7000640 1 --2- 172.21.15.43:0/2845148707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ce01036f0 0x7f6ce01998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:31.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.495+0000 7f6ce7000640 1 -- 172.21.15.43:0/2845148707 >> 172.21.15.43:0/2845148707 conn(0x7f6ce00fec80 msgr2=0x7f6ce0107c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:31.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.495+0000 7f6ce7000640 1 -- 172.21.15.43:0/2845148707 shutdown_connections 2024-03-26T08:33:31.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.496+0000 7f6ce7000640 1 -- 172.21.15.43:0/2845148707 wait complete. 2024-03-26T08:33:31.509 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:33:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.742+0000 7faf0b000640 1 -- 172.21.15.43:0/44467539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf041028b0 msgr2=0x7faf04102d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.742+0000 7faf0b000640 1 --2- 172.21.15.43:0/44467539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf041028b0 0x7faf04102d30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7faeec009ca0 tx=0x7faeec02c0f0 comp rx=0 tx=0).stop 2024-03-26T08:33:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.742+0000 7faf0b000640 1 -- 172.21.15.43:0/44467539 shutdown_connections 2024-03-26T08:33:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.742+0000 7faf0b000640 1 --2- 172.21.15.43:0/44467539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf04103270 0x7faf04109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.742+0000 7faf0b000640 1 --2- 172.21.15.43:0/44467539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf041028b0 0x7faf04102d30 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.742+0000 7faf0b000640 1 --2- 172.21.15.43:0/44467539 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf041016c0 0x7faf04101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.742+0000 7faf0b000640 1 -- 172.21.15.43:0/44467539 >> 172.21.15.43:0/44467539 conn(0x7faf040fcc50 msgr2=0x7faf040ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.742+0000 7faf0b000640 1 -- 172.21.15.43:0/44467539 shutdown_connections 2024-03-26T08:33:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.742+0000 7faf0b000640 1 -- 172.21.15.43:0/44467539 wait complete. 2024-03-26T08:33:31.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.743+0000 7faf0b000640 1 Processor -- start 2024-03-26T08:33:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.743+0000 7faf0b000640 1 -- start start 2024-03-26T08:33:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.743+0000 7faf0b000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf041016c0 0x7faf04199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.743+0000 7faf0b000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf041028b0 0x7faf0419a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.744+0000 7faf0b000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf04103270 0x7faf041a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.744+0000 7faf0b000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0410d1f0 con 0x7faf04103270 2024-03-26T08:33:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.744+0000 7faf0b000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0410d070 con 0x7faf041016c0 2024-03-26T08:33:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.744+0000 7faf0b000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0410d370 con 0x7faf041028b0 2024-03-26T08:33:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.744+0000 7faf09000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf04103270 0x7faf041a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.744+0000 7faf09000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf04103270 0x7faf041a1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52226/0 (socket says 172.21.15.43:52226) 2024-03-26T08:33:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.744+0000 7faf09000640 1 -- 172.21.15.43:0/2389382140 learned_addr learned my addr 172.21.15.43:0/2389382140 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.744+0000 7faf03e00640 1 --2- 172.21.15.43:0/2389382140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf041016c0 0x7faf04199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:31.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.744+0000 7faf03400640 1 --2- 172.21.15.43:0/2389382140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf041028b0 0x7faf0419a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.744+0000 7faf09000640 1 -- 172.21.15.43:0/2389382140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf041028b0 msgr2=0x7faf0419a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.744+0000 7faf09000640 1 --2- 172.21.15.43:0/2389382140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf041028b0 0x7faf0419a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.744+0000 7faf09000640 1 -- 172.21.15.43:0/2389382140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf041016c0 msgr2=0x7faf04199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.744+0000 7faf09000640 1 --2- 172.21.15.43:0/2389382140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf041016c0 0x7faf04199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.744+0000 7faf09000640 1 -- 172.21.15.43:0/2389382140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf041a1790 con 0x7faf04103270 2024-03-26T08:33:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.745+0000 7faf09000640 1 --2- 172.21.15.43:0/2389382140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf04103270 0x7faf041a1120 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7faefc00d950 tx=0x7faefc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.745+0000 7faf00c00640 1 -- 172.21.15.43:0/2389382140 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc017030 con 0x7faf04103270 2024-03-26T08:33:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.745+0000 7faf00c00640 1 -- 172.21.15.43:0/2389382140 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faefc004020 con 0x7faf04103270 2024-03-26T08:33:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.745+0000 7faf0b000640 1 -- 172.21.15.43:0/2389382140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf041a1a80 con 0x7faf04103270 2024-03-26T08:33:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.745+0000 7faf00c00640 1 -- 172.21.15.43:0/2389382140 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc003b20 con 0x7faf04103270 2024-03-26T08:33:31.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.745+0000 7faf0b000640 1 -- 172.21.15.43:0/2389382140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf041a1f50 con 0x7faf04103270 2024-03-26T08:33:31.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.746+0000 7faf0b000640 1 -- 172.21.15.43:0/2389382140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf041a21e0 con 0x7faf04103270 2024-03-26T08:33:31.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.748+0000 7faf00c00640 1 -- 172.21.15.43:0/2389382140 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7faefc01a070 con 0x7faf04103270 2024-03-26T08:33:31.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.750+0000 7faf00c00640 1 --2- 172.21.15.43:0/2389382140 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7faee40a2800 0x7faee40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:31.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.750+0000 7faf00c00640 1 -- 172.21.15.43:0/2389382140 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7faefc0cea60 con 0x7faf04103270 2024-03-26T08:33:31.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.750+0000 7faf03e00640 1 --2- 172.21.15.43:0/2389382140 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7faee40a2800 0x7faee40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:31.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.751+0000 7faf03e00640 1 --2- 172.21.15.43:0/2389382140 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7faee40a2800 0x7faee40a4cc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faf04102710 tx=0x7faef4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:31.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.753+0000 7faf00c00640 1 -- 172.21.15.43:0/2389382140 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faefc021070 con 0x7faf04103270 2024-03-26T08:33:31.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:31.999+0000 7faf0b000640 1 -- 172.21.15.43:0/2389382140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf041a23b0 con 0x7faf04103270 2024-03-26T08:33:31.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.001+0000 7faf00c00640 1 -- 172.21.15.43:0/2389382140 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11331 (secure 0 0 0) 0x7faefc09b720 con 0x7faf04103270 2024-03-26T08:33:31.993 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:33:31.994 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1452206807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1452206807},{"type":"v1","addr":"172.21.15.43:6835","nonce":1452206807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3151775467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3151775467},{"type":"v1","addr":"172.21.15.43:6837","nonce":3151775467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/1284655682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":1284655682},{"type":"v1","addr":"172.21.15.43:6839","nonce":1284655682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3081085787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3081085787},{"type":"v1","addr":"172.21.15.43:6841","nonce":3081085787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1842295306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1842295306},{"type":"v1","addr":"172.21.15.183:6835","nonce":1842295306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2267781666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2267781666},{"type":"v1","addr":"172.21.15.183:6837","nonce":2267781666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3572429948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3572429948},{"type":"v1","addr":"172.21.15.183:6839","nonce":3572429948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34448,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/1108327179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108327179},{"type":"v1","addr":"172.21.15.52:6833","nonce":1108327179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34487,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/742057410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":742057410},{"type":"v1","addr":"172.21.15.52:6835","nonce":742057410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:31:21.819689+0000","modified":"2024-03-26T08:31:48.080233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34517":{"gid":34517,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/748239918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":748239918},{"type":"v1","addr":"172.21.15.52:6837","nonce":748239918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34559":{"gid":34559,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.52:6839/2551965101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2551965101},{"type":"v1","addr":"172.21.15.52:6839","nonce":2551965101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34559,"qdb_cluster":[34559]},"id":1}]} 2024-03-26T08:33:31.994 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 21 2024-03-26T08:33:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.003+0000 7faf0b000640 1 -- 172.21.15.43:0/2389382140 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7faee40a2800 msgr2=0x7faee40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.003+0000 7faf0b000640 1 --2- 172.21.15.43:0/2389382140 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7faee40a2800 0x7faee40a4cc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faf04102710 tx=0x7faef4005f90 comp rx=0 tx=0).stop 2024-03-26T08:33:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.003+0000 7faf0b000640 1 -- 172.21.15.43:0/2389382140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf04103270 msgr2=0x7faf041a1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.003+0000 7faf0b000640 1 --2- 172.21.15.43:0/2389382140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf04103270 0x7faf041a1120 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7faefc00d950 tx=0x7faefc00de10 comp rx=0 tx=0).stop 2024-03-26T08:33:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.003+0000 7faf0b000640 1 -- 172.21.15.43:0/2389382140 shutdown_connections 2024-03-26T08:33:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.004+0000 7faf0b000640 1 --2- 172.21.15.43:0/2389382140 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7faee40a2800 0x7faee40a4cc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:31.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.004+0000 7faf0b000640 1 --2- 172.21.15.43:0/2389382140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf04103270 0x7faf041a1120 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:31.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.004+0000 7faf0b000640 1 --2- 172.21.15.43:0/2389382140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf041028b0 0x7faf0419a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:31.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.004+0000 7faf0b000640 1 --2- 172.21.15.43:0/2389382140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf041016c0 0x7faf04199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:31.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.004+0000 7faf0b000640 1 -- 172.21.15.43:0/2389382140 >> 172.21.15.43:0/2389382140 conn(0x7faf040fcc50 msgr2=0x7faf040738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:31.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.004+0000 7faf0b000640 1 -- 172.21.15.43:0/2389382140 shutdown_connections 2024-03-26T08:33:31.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.004+0000 7faf0b000640 1 -- 172.21.15.43:0/2389382140 wait complete. 2024-03-26T08:33:32.017 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-26T08:31:21.819689+0000', 'modified': '2024-03-26T08:31:48.080233+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34559}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34517': {'gid': 34517, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.52:6837/748239918', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6836', 'nonce': 748239918}, {'type': 'v1', 'addr': '172.21.15.52:6837', 'nonce': 748239918}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34559': {'gid': 34559, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.52:6839/2551965101', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6838', 'nonce': 2551965101}, {'type': 'v1', 'addr': '172.21.15.52:6839', 'nonce': 2551965101}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34559, 'qdb_cluster': [34559]} 2024-03-26T08:33:32.017 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-26T08:33:32.017 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-26T08:33:32.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.254+0000 7f5c7be00640 1 -- 172.21.15.43:0/74723584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c7c1037a0 msgr2=0x7f5c7c103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:32.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.254+0000 7f5c7be00640 1 --2- 172.21.15.43:0/74723584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c7c1037a0 0x7f5c7c103bc0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009900 tx=0x7f5c6c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:33:32.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.254+0000 7f5c7be00640 1 -- 172.21.15.43:0/74723584 shutdown_connections 2024-03-26T08:33:32.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.254+0000 7f5c7be00640 1 --2- 172.21.15.43:0/74723584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c7c105350 0x7f5c7c10bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.254+0000 7f5c7be00640 1 --2- 172.21.15.43:0/74723584 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c7c104990 0x7f5c7c104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.254+0000 7f5c7be00640 1 --2- 172.21.15.43:0/74723584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c7c1037a0 0x7f5c7c103bc0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.254+0000 7f5c7be00640 1 -- 172.21.15.43:0/74723584 >> 172.21.15.43:0/74723584 conn(0x7f5c7c0fed10 msgr2=0x7f5c7c101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:32.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.254+0000 7f5c7be00640 1 -- 172.21.15.43:0/74723584 shutdown_connections 2024-03-26T08:33:32.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.254+0000 7f5c7be00640 1 -- 172.21.15.43:0/74723584 wait complete. 2024-03-26T08:33:32.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.255+0000 7f5c7be00640 1 Processor -- start 2024-03-26T08:33:32.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.255+0000 7f5c7be00640 1 -- start start 2024-03-26T08:33:32.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.255+0000 7f5c7be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c7c1037a0 0x7f5c7c199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:32.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.255+0000 7f5c7be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c7c104990 0x7f5c7c199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:32.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.255+0000 7f5c7be00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c7c105350 0x7f5c7c1a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:32.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.255+0000 7f5c7be00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c10f310 con 0x7f5c7c1037a0 2024-03-26T08:33:32.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.255+0000 7f5c7be00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c10f190 con 0x7f5c7c105350 2024-03-26T08:33:32.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.255+0000 7f5c7be00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c10f490 con 0x7f5c7c104990 2024-03-26T08:33:32.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.256+0000 7f5c7aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c7c1037a0 0x7f5c7c199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:32.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.256+0000 7f5c7aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c7c1037a0 0x7f5c7c199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52244/0 (socket says 172.21.15.43:52244) 2024-03-26T08:33:32.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.256+0000 7f5c7aa00640 1 -- 172.21.15.43:0/2068971079 learned_addr learned my addr 172.21.15.43:0/2068971079 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:32.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.256+0000 7f5c7b400640 1 --2- 172.21.15.43:0/2068971079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c7c105350 0x7f5c7c1a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:32.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.256+0000 7f5c7aa00640 1 -- 172.21.15.43:0/2068971079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c7c104990 msgr2=0x7f5c7c199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:32.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.256+0000 7f5c7aa00640 1 --2- 172.21.15.43:0/2068971079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c7c104990 0x7f5c7c199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.256+0000 7f5c7aa00640 1 -- 172.21.15.43:0/2068971079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c7c105350 msgr2=0x7f5c7c1a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:32.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.256+0000 7f5c7aa00640 1 --2- 172.21.15.43:0/2068971079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c7c105350 0x7f5c7c1a0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.256+0000 7f5c7aa00640 1 -- 172.21.15.43:0/2068971079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7c1a1610 con 0x7f5c7c1037a0 2024-03-26T08:33:32.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.256+0000 7f5c7aa00640 1 --2- 172.21.15.43:0/2068971079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c7c1037a0 0x7f5c7c199a40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c005ab0 tx=0x7f5c6c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:32.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.257+0000 7f5c73400640 1 -- 172.21.15.43:0/2068971079 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c043020 con 0x7f5c7c1037a0 2024-03-26T08:33:32.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.257+0000 7f5c73400640 1 -- 172.21.15.43:0/2068971079 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5c6c0054c0 con 0x7f5c7c1037a0 2024-03-26T08:33:32.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.257+0000 7f5c7be00640 1 -- 172.21.15.43:0/2068971079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c7c1a18a0 con 0x7f5c7c1037a0 2024-03-26T08:33:32.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.257+0000 7f5c73400640 1 -- 172.21.15.43:0/2068971079 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c003d10 con 0x7f5c7c1037a0 2024-03-26T08:33:32.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.257+0000 7f5c7be00640 1 -- 172.21.15.43:0/2068971079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7c1a1df0 con 0x7f5c7c1037a0 2024-03-26T08:33:32.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.260+0000 7f5c73400640 1 -- 172.21.15.43:0/2068971079 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5c6c046020 con 0x7f5c7c1037a0 2024-03-26T08:33:32.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.260+0000 7f5c8497c740 1 Processor -- start 2024-03-26T08:33:32.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.261+0000 7f5c8497c740 1 -- start start 2024-03-26T08:33:32.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.261+0000 7f5c8497c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9211d9d80 0x55b9211da1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:32.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.261+0000 7f5c8497c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9211da770 0x55b9211df270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:32.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.261+0000 7f5c8497c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b9211df8e0 0x55b9211e1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:32.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.261+0000 7f5c8497c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b92113ec90 con 0x55b9211d9d80 2024-03-26T08:33:32.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.261+0000 7f5c8497c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9210e7980 con 0x55b9211df8e0 2024-03-26T08:33:32.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.261+0000 7f5c8497c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b92113ad30 con 0x55b9211da770 2024-03-26T08:33:32.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.261+0000 7f5c7b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9211d9d80 0x55b9211da1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:32.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.261+0000 7f5c7b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9211d9d80 0x55b9211da1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52246/0 (socket says 172.21.15.43:52246) 2024-03-26T08:33:32.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.261+0000 7f5c7b400640 1 -- 172.21.15.43:0/3654162406 learned_addr learned my addr 172.21.15.43:0/3654162406 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:32.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.262+0000 7f5c73e00640 1 --2- 172.21.15.43:0/3654162406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b9211df8e0 0x55b9211e1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:32.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.262+0000 7f5c7b400640 1 -- 172.21.15.43:0/3654162406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9211da770 msgr2=0x55b9211df270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:33:32.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.262+0000 7f5c7b400640 1 --2- 172.21.15.43:0/3654162406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9211da770 0x55b9211df270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.262+0000 7f5c7b400640 1 -- 172.21.15.43:0/3654162406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b9211df8e0 msgr2=0x55b9211e1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:33:32.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.262+0000 7f5c7b400640 1 --2- 172.21.15.43:0/3654162406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b9211df8e0 0x55b9211e1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.262+0000 7f5c7b400640 1 -- 172.21.15.43:0/3654162406 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9210ff510 con 0x55b9211d9d80 2024-03-26T08:33:32.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.262+0000 7f5c73400640 1 --2- 172.21.15.43:0/2068971079 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5c600a2800 0x7f5c600a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:32.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.262+0000 7f5c73400640 1 -- 172.21.15.43:0/2068971079 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f5c6c0f1330 con 0x7f5c7c1037a0 2024-03-26T08:33:32.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.262+0000 7f5c7aa00640 1 --2- 172.21.15.43:0/2068971079 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5c600a2800 0x7f5c600a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:32.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.262+0000 7f5c7b400640 1 --2- 172.21.15.43:0/3654162406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9211d9d80 0x55b9211da1a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5c74009df0 tx=0x7f5c7400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5557ce889a3cb7d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:32.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.263+0000 7f5c6be00640 1 -- 172.21.15.43:0/3654162406 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c74003970 con 0x55b9211d9d80 2024-03-26T08:33:32.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.263+0000 7f5c6be00640 1 -- 172.21.15.43:0/3654162406 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5c74004030 con 0x55b9211d9d80 2024-03-26T08:33:32.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.263+0000 7f5c6be00640 1 -- 172.21.15.43:0/3654162406 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c74011720 con 0x55b9211d9d80 2024-03-26T08:33:32.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.263+0000 7f5c8497c740 1 -- 172.21.15.43:0/3654162406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9211d9d80 msgr2=0x55b9211da1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:32.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.263+0000 7f5c8497c740 1 --2- 172.21.15.43:0/3654162406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9211d9d80 0x55b9211da1a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5c74009df0 tx=0x7f5c7400f730 comp rx=0 tx=0).stop 2024-03-26T08:33:32.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.263+0000 7f5c7aa00640 1 --2- 172.21.15.43:0/2068971079 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5c600a2800 0x7f5c600a4cc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c0ba3e0 tx=0x7f5c6c02bc30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:32.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.264+0000 7f5c8497c740 1 -- 172.21.15.43:0/3654162406 shutdown_connections 2024-03-26T08:33:32.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.264+0000 7f5c8497c740 1 --2- 172.21.15.43:0/3654162406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b9211df8e0 0x55b9211e1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.264+0000 7f5c8497c740 1 --2- 172.21.15.43:0/3654162406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9211da770 0x55b9211df270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.264+0000 7f5c8497c740 1 --2- 172.21.15.43:0/3654162406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9211d9d80 0x55b9211da1a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.264+0000 7f5c8497c740 1 -- 172.21.15.43:0/3654162406 >> 172.21.15.43:0/3654162406 conn(0x55b9211d52f0 msgr2=0x55b9211d56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:32.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.264+0000 7f5c8497c740 1 -- 172.21.15.43:0/3654162406 shutdown_connections 2024-03-26T08:33:32.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.264+0000 7f5c8497c740 1 -- 172.21.15.43:0/3654162406 wait complete. 2024-03-26T08:33:32.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.264+0000 7f5c8497c740 1 -- start start 2024-03-26T08:33:32.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.264+0000 7f5c8497c740 1 Processor -- start 2024-03-26T08:33:32.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.265+0000 7f5c8497c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9211d9d80 0x55b9211e42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:32.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.265+0000 7f5c8497c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9211da770 0x55b9211e47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:32.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.265+0000 7f5c8497c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b9211df8e0 0x55b9211eb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:32.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.265+0000 7f5c8497c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9211e2890 con 0x55b9211da770 2024-03-26T08:33:32.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.265+0000 7f5c8497c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9210fadd0 con 0x55b9211df8e0 2024-03-26T08:33:32.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.265+0000 7f5c8497c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9211e2a10 con 0x55b9211d9d80 2024-03-26T08:33:32.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.265+0000 7f5c8497c740 10 client.0 _ll_register_callbacks cb 0x55b920e52ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:33:32.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.265+0000 7f5c73e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9211da770 0x55b9211e47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:32.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.265+0000 7f5c73e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9211da770 0x55b9211e47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52254/0 (socket says 172.21.15.43:52254) 2024-03-26T08:33:32.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.265+0000 7f5c73e00640 1 -- 172.21.15.43:0/795975190 learned_addr learned my addr 172.21.15.43:0/795975190 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:32.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.265+0000 7f5c7b400640 1 --2- 172.21.15.43:0/795975190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b9211df8e0 0x55b9211eb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:32.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.265+0000 7f5c73e00640 1 -- 172.21.15.43:0/795975190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9211d9d80 msgr2=0x55b9211e42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:33:32.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.265+0000 7f5c73e00640 1 --2- 172.21.15.43:0/795975190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9211d9d80 0x55b9211e42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.265+0000 7f5c73e00640 1 -- 172.21.15.43:0/795975190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b9211df8e0 msgr2=0x55b9211eb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:33:32.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.265+0000 7f5c73e00640 1 --2- 172.21.15.43:0/795975190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b9211df8e0 0x55b9211eb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.265+0000 7f5c73e00640 1 -- 172.21.15.43:0/795975190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b920e532d0 con 0x55b9211da770 2024-03-26T08:33:32.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.266+0000 7f5c73e00640 1 --2- 172.21.15.43:0/795975190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9211da770 0x55b9211e47e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400be00 tx=0x7f5c6400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980546f024d5fa80 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:32.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.266+0000 7f5c68c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-26T08:33:32.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.266+0000 7f5c68c00640 1 -- 172.21.15.43:0/795975190 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c64015070 con 0x55b9211da770 2024-03-26T08:33:32.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.266+0000 7f5c68c00640 1 -- 172.21.15.43:0/795975190 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f5c64011850 con 0x55b9211da770 2024-03-26T08:33:32.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.266+0000 7f5c8497c740 1 -- 172.21.15.43:0/795975190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c74004030 con 0x55b9211da770 2024-03-26T08:33:32.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.266+0000 7f5c68c00640 1 -- 172.21.15.43:0/795975190 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c64023050 con 0x55b9211da770 2024-03-26T08:33:32.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.266+0000 7f5c8497c740 10 client.15252 fetch_fsmap learned FSMap version 21 2024-03-26T08:33:32.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.266+0000 7f5c8497c740 1 -- 172.21.15.43:0/795975190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55b920f1d6b0 con 0x55b9211da770 2024-03-26T08:33:32.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.267+0000 7f5c68c00640 1 -- 172.21.15.43:0/795975190 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5c74004030 con 0x55b9211da770 2024-03-26T08:33:32.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.267+0000 7f5c68c00640 1 -- 172.21.15.43:0/795975190 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f5c6401aa50 con 0x55b9211da770 2024-03-26T08:33:32.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.267+0000 7f5c8497c740 10 client.15252 fetch_fsmap finished waiting for FSMap version 21 2024-03-26T08:33:32.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.267+0000 7f5c8497c740 10 client.15252 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T08:33:32.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.267+0000 7f5c8497c740 20 client.15252 populate_metadata read hostname 'smithi043' 2024-03-26T08:33:32.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.267+0000 7f5c8497c740 1 --2- 172.21.15.43:0/795975190 >> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] conn(0x55b9211f0af0 0x55b9211efbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:32.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.267+0000 7f5c8497c740 4 client.15252 mds_command: new command op to 34559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:33:32.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.267+0000 7f5c8497c740 1 -- 172.21.15.43:0/795975190 --> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b9210e0520 con 0x55b9211f0af0 2024-03-26T08:33:32.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.268+0000 7f5c7b400640 1 --2- 172.21.15.43:0/795975190 >> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] conn(0x55b9211f0af0 0x55b9211efbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:32.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.269+0000 7f5c7b400640 1 --2- 172.21.15.43:0/795975190 >> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] conn(0x55b9211f0af0 0x55b9211efbc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f62a7a3ee4eb96f server_cookie=2396531bb5fbc040 in_seq=0 out_seq=0 2024-03-26T08:33:32.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.269+0000 7f5c68c00640 10 client.15252 ms_handle_connect on v2:172.21.15.52:6838/2551965101 2024-03-26T08:33:32.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.271+0000 7f5c68c00640 1 -- 172.21.15.43:0/795975190 <== mds.0 v2:172.21.15.52:6838/2551965101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b9210e0520 con 0x55b9211f0af0 2024-03-26T08:33:32.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.271+0000 7f5c68c00640 10 client.15252 handle_command_reply: tid=0 2024-03-26T08:33:32.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.271+0000 7f5c8497c740 1 client.15252 shutdown 2024-03-26T08:33:32.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.271+0000 7f5c8497c740 1 -- 172.21.15.43:0/795975190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9211da770 msgr2=0x55b9211e47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:32.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.271+0000 7f5c8497c740 1 --2- 172.21.15.43:0/795975190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9211da770 0x55b9211e47e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400be00 tx=0x7f5c6400f730 comp rx=0 tx=0).stop 2024-03-26T08:33:32.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.271+0000 7f5c8497c740 1 -- 172.21.15.43:0/795975190 shutdown_connections 2024-03-26T08:33:32.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.271+0000 7f5c8497c740 1 --2- 172.21.15.43:0/795975190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b9211df8e0 0x55b9211eb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.271+0000 7f5c8497c740 1 --2- 172.21.15.43:0/795975190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9211da770 0x55b9211e47e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.271+0000 7f5c8497c740 1 --2- 172.21.15.43:0/795975190 >> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] conn(0x55b9211f0af0 0x55b9211efbc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.271+0000 7f5c8497c740 1 --2- 172.21.15.43:0/795975190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9211d9d80 0x55b9211e42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.271+0000 7f5c68c00640 0 client.15252 ms_handle_reset on v2:172.21.15.52:6838/2551965101 2024-03-26T08:33:32.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.271+0000 7f5c8497c740 1 -- 172.21.15.43:0/795975190 >> 172.21.15.43:0/795975190 conn(0x55b9211d52f0 msgr2=0x55b9211d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:32.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.271+0000 7f5c8497c740 1 -- 172.21.15.43:0/795975190 shutdown_connections 2024-03-26T08:33:32.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.271+0000 7f5c8497c740 1 -- 172.21.15.43:0/795975190 wait complete. 2024-03-26T08:33:32.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.271+0000 7f5c8497c740 20 client.15252 trim_cache size 0 max 16384 2024-03-26T08:33:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.290+0000 7f5c8497c740 1 Processor -- start 2024-03-26T08:33:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.290+0000 7f5c8497c740 1 -- start start 2024-03-26T08:33:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.290+0000 7f5c8497c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b9211d9d80 0x55b9211e2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.290+0000 7f5c8497c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9211da770 0x55b9211db210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.290+0000 7f5c8497c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9211df8e0 0x55b9211d56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.290+0000 7f5c8497c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9211f0310 con 0x55b9211da770 2024-03-26T08:33:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.290+0000 7f5c8497c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9211f0970 con 0x55b9211d9d80 2024-03-26T08:33:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.290+0000 7f5c8497c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9211f0fd0 con 0x55b9211df8e0 2024-03-26T08:33:32.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.291+0000 7f5c73e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9211da770 0x55b9211db210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:32.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.291+0000 7f5c73e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9211da770 0x55b9211db210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52262/0 (socket says 172.21.15.43:52262) 2024-03-26T08:33:32.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.291+0000 7f5c73e00640 1 -- 172.21.15.43:0/2060278707 learned_addr learned my addr 172.21.15.43:0/2060278707 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:32.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.291+0000 7f5c7b400640 1 --2- 172.21.15.43:0/2060278707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b9211d9d80 0x55b9211e2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:32.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.291+0000 7f5c7b400640 1 --2- 172.21.15.43:0/2060278707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9211df8e0 0x55b9211d56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:32.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.291+0000 7f5c73e00640 1 -- 172.21.15.43:0/2060278707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9211df8e0 msgr2=0x55b9211d56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:33:32.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.291+0000 7f5c73e00640 1 --2- 172.21.15.43:0/2060278707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9211df8e0 0x55b9211d56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.291+0000 7f5c73e00640 1 -- 172.21.15.43:0/2060278707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b9211d9d80 msgr2=0x55b9211e2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:33:32.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.291+0000 7f5c73e00640 1 --2- 172.21.15.43:0/2060278707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b9211d9d80 0x55b9211e2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.291+0000 7f5c73e00640 1 -- 172.21.15.43:0/2060278707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9211d6060 con 0x55b9211da770 2024-03-26T08:33:32.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.291+0000 7f5c7b400640 1 --2- 172.21.15.43:0/2060278707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b9211d9d80 0x55b9211e2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:33:32.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.291+0000 7f5c73e00640 1 --2- 172.21.15.43:0/2060278707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9211da770 0x55b9211db210 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5c64000c00 tx=0x7f5c6400b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8c8233b142153f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:32.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.292+0000 7f5c6b400640 1 -- 172.21.15.43:0/2060278707 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c64020040 con 0x55b9211da770 2024-03-26T08:33:32.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.292+0000 7f5c6b400640 1 -- 172.21.15.43:0/2060278707 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5c640044f0 con 0x55b9211da770 2024-03-26T08:33:32.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.292+0000 7f5c6b400640 1 -- 172.21.15.43:0/2060278707 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c64015040 con 0x55b9211da770 2024-03-26T08:33:32.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.292+0000 7f5c8497c740 1 -- 172.21.15.43:0/2060278707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9211da770 msgr2=0x55b9211db210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:32.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.292+0000 7f5c8497c740 1 --2- 172.21.15.43:0/2060278707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9211da770 0x55b9211db210 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5c64000c00 tx=0x7f5c6400b9f0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.293+0000 7f5c8497c740 1 -- 172.21.15.43:0/2060278707 shutdown_connections 2024-03-26T08:33:32.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.293+0000 7f5c8497c740 1 --2- 172.21.15.43:0/2060278707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9211df8e0 0x55b9211d56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.293+0000 7f5c8497c740 1 --2- 172.21.15.43:0/2060278707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9211da770 0x55b9211db210 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.293+0000 7f5c8497c740 1 --2- 172.21.15.43:0/2060278707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b9211d9d80 0x55b9211e2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.293+0000 7f5c8497c740 1 -- 172.21.15.43:0/2060278707 >> 172.21.15.43:0/2060278707 conn(0x55b9211d52f0 msgr2=0x55b9211efbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:32.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.293+0000 7f5c8497c740 1 -- 172.21.15.43:0/2060278707 shutdown_connections 2024-03-26T08:33:32.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.293+0000 7f5c8497c740 1 -- 172.21.15.43:0/2060278707 wait complete. 2024-03-26T08:33:32.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.293+0000 7f5c8497c740 1 -- start start 2024-03-26T08:33:32.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.293+0000 7f5c8497c740 1 Processor -- start 2024-03-26T08:33:32.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.294+0000 7f5c8497c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9211d9d80 0x55b9211e9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:32.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.294+0000 7f5c8497c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b9211da770 0x55b9211e9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:32.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.294+0000 7f5c8497c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9211df8e0 0x55b9211ea1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:32.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.294+0000 7f5c8497c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9211f1c90 con 0x55b9211d9d80 2024-03-26T08:33:32.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.294+0000 7f5c8497c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9211f1630 con 0x55b9211da770 2024-03-26T08:33:32.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.294+0000 7f5c8497c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9211f22f0 con 0x55b9211df8e0 2024-03-26T08:33:32.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.294+0000 7f5c8497c740 10 client.0 _ll_register_callbacks cb 0x7f5c6400b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:33:32.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.294+0000 7f5c7b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9211d9d80 0x55b9211e9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:32.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.294+0000 7f5c7b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9211d9d80 0x55b9211e9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52268/0 (socket says 172.21.15.43:52268) 2024-03-26T08:33:32.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.294+0000 7f5c7b400640 1 -- 172.21.15.43:0/1365427984 learned_addr learned my addr 172.21.15.43:0/1365427984 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:32.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.294+0000 7f5c73e00640 1 --2- 172.21.15.43:0/1365427984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b9211da770 0x55b9211e9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:32.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.294+0000 7f5c7b400640 1 -- 172.21.15.43:0/1365427984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9211df8e0 msgr2=0x55b9211ea1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:33:32.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.294+0000 7f5c7b400640 1 --2- 172.21.15.43:0/1365427984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9211df8e0 0x55b9211ea1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.294+0000 7f5c7b400640 1 -- 172.21.15.43:0/1365427984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b9211da770 msgr2=0x55b9211e9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:33:32.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.294+0000 7f5c7b400640 1 --2- 172.21.15.43:0/1365427984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b9211da770 0x55b9211e9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.294+0000 7f5c7b400640 1 -- 172.21.15.43:0/1365427984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b9211ea6f0 con 0x55b9211d9d80 2024-03-26T08:33:32.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.295+0000 7f5c73e00640 1 --2- 172.21.15.43:0/1365427984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b9211da770 0x55b9211e9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:33:32.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.295+0000 7f5c7b400640 1 --2- 172.21.15.43:0/1365427984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9211d9d80 0x55b9211e9730 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x55b9211dafd0 tx=0x7f5c74009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1c2aa507fa8a19 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:32.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.295+0000 7f5c68c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-26T08:33:32.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.295+0000 7f5c68c00640 1 -- 172.21.15.43:0/1365427984 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c74003d80 con 0x55b9211d9d80 2024-03-26T08:33:32.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.295+0000 7f5c68c00640 1 -- 172.21.15.43:0/1365427984 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f5c74025610 con 0x55b9211d9d80 2024-03-26T08:33:32.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.295+0000 7f5c8497c740 1 -- 172.21.15.43:0/1365427984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c640044f0 con 0x55b9211d9d80 2024-03-26T08:33:32.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.295+0000 7f5c68c00640 1 -- 172.21.15.43:0/1365427984 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c74025810 con 0x55b9211d9d80 2024-03-26T08:33:32.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.295+0000 7f5c8497c740 10 client.15258 fetch_fsmap learned FSMap version 21 2024-03-26T08:33:32.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.295+0000 7f5c8497c740 1 -- 172.21.15.43:0/1365427984 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55b92124c060 con 0x55b9211d9d80 2024-03-26T08:33:32.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.296+0000 7f5c68c00640 1 -- 172.21.15.43:0/1365427984 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5c640044f0 con 0x55b9211d9d80 2024-03-26T08:33:32.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.296+0000 7f5c68c00640 1 -- 172.21.15.43:0/1365427984 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f5c740055b0 con 0x55b9211d9d80 2024-03-26T08:33:32.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.296+0000 7f5c8497c740 10 client.15258 fetch_fsmap finished waiting for FSMap version 21 2024-03-26T08:33:32.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.296+0000 7f5c8497c740 10 client.15258 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T08:33:32.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.296+0000 7f5c8497c740 20 client.15258 populate_metadata read hostname 'smithi043' 2024-03-26T08:33:32.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.296+0000 7f5c8497c740 1 --2- 172.21.15.43:0/1365427984 >> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] conn(0x55b9212503f0 0x55b92124c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:32.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.296+0000 7f5c8497c740 4 client.15258 mds_command: new command op to 34559 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T08:33:32.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.296+0000 7f5c8497c740 1 -- 172.21.15.43:0/1365427984 --> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b92114c1d0 con 0x55b9212503f0 2024-03-26T08:33:32.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.297+0000 7f5c73e00640 1 --2- 172.21.15.43:0/1365427984 >> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] conn(0x55b9212503f0 0x55b92124c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:32.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.298+0000 7f5c73e00640 1 --2- 172.21.15.43:0/1365427984 >> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] conn(0x55b9212503f0 0x55b92124c340 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba48bba99110c7c4 server_cookie=5e86e8745bf436bc in_seq=0 out_seq=0 2024-03-26T08:33:32.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.298+0000 7f5c68c00640 10 client.15258 ms_handle_connect on v2:172.21.15.52:6838/2551965101 2024-03-26T08:33:32.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.298+0000 7f5c68c00640 1 -- 172.21.15.43:0/1365427984 <== mds.0 v2:172.21.15.52:6838/2551965101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55b92114c1d0 con 0x55b9212503f0 2024-03-26T08:33:32.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.298+0000 7f5c68c00640 10 client.15258 handle_command_reply: tid=0 2024-03-26T08:33:32.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.298+0000 7f5c8497c740 1 client.15258 shutdown 2024-03-26T08:33:32.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.299+0000 7f5c8497c740 1 -- 172.21.15.43:0/1365427984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9211d9d80 msgr2=0x55b9211e9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:32.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.299+0000 7f5c8497c740 1 --2- 172.21.15.43:0/1365427984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9211d9d80 0x55b9211e9730 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x55b9211dafd0 tx=0x7f5c74009b30 comp rx=0 tx=0).stop 2024-03-26T08:33:32.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.299+0000 7f5c8497c740 1 -- 172.21.15.43:0/1365427984 shutdown_connections 2024-03-26T08:33:32.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.299+0000 7f5c8497c740 1 --2- 172.21.15.43:0/1365427984 >> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] conn(0x55b9212503f0 0x55b92124c340 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.299+0000 7f5c8497c740 1 --2- 172.21.15.43:0/1365427984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9211df8e0 0x55b9211ea1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.299+0000 7f5c8497c740 1 --2- 172.21.15.43:0/1365427984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b9211da770 0x55b9211e9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.299+0000 7f5c8497c740 1 --2- 172.21.15.43:0/1365427984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b9211d9d80 0x55b9211e9730 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.299+0000 7f5c8497c740 1 -- 172.21.15.43:0/1365427984 >> 172.21.15.43:0/1365427984 conn(0x55b9211d52f0 msgr2=0x55b9211e2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:32.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.299+0000 7f5c68c00640 0 client.15258 ms_handle_reset on v2:172.21.15.52:6838/2551965101 2024-03-26T08:33:32.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.299+0000 7f5c8497c740 1 -- 172.21.15.43:0/1365427984 shutdown_connections 2024-03-26T08:33:32.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.299+0000 7f5c8497c740 1 -- 172.21.15.43:0/1365427984 wait complete. 2024-03-26T08:33:32.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.299+0000 7f5c8497c740 20 client.15258 trim_cache size 0 max 16384 2024-03-26T08:33:32.296 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-26T08:33:32.296 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "88759420-eb49-11ee-95d0-87774f69a715", 2024-03-26T08:33:32.296 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-03-26T08:33:32.296 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 34559, 2024-03-26T08:33:32.296 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-03-26T08:33:32.296 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-03-26T08:33:32.297 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-03-26T08:33:32.297 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 129.925503397, 2024-03-26T08:33:32.297 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 21, 2024-03-26T08:33:32.297 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 94, 2024-03-26T08:33:32.297 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 92, 2024-03-26T08:33:32.297 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 138.10737036899999 2024-03-26T08:33:32.297 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-26T08:33:32.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.300+0000 7f5c7be00640 1 -- 172.21.15.43:0/2068971079 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5c600a2800 msgr2=0x7f5c600a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:32.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.300+0000 7f5c7be00640 1 --2- 172.21.15.43:0/2068971079 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5c600a2800 0x7f5c600a4cc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c0ba3e0 tx=0x7f5c6c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:33:32.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.300+0000 7f5c7be00640 1 -- 172.21.15.43:0/2068971079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c7c1037a0 msgr2=0x7f5c7c199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:32.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.300+0000 7f5c7be00640 1 --2- 172.21.15.43:0/2068971079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c7c1037a0 0x7f5c7c199a40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c005ab0 tx=0x7f5c6c0042f0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.300+0000 7f5c7be00640 1 -- 172.21.15.43:0/2068971079 shutdown_connections 2024-03-26T08:33:32.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.300+0000 7f5c7be00640 1 --2- 172.21.15.43:0/2068971079 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5c600a2800 0x7f5c600a4cc0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.300+0000 7f5c7be00640 1 --2- 172.21.15.43:0/2068971079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c7c105350 0x7f5c7c1a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.300+0000 7f5c7be00640 1 --2- 172.21.15.43:0/2068971079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c7c104990 0x7f5c7c199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.300+0000 7f5c7be00640 1 --2- 172.21.15.43:0/2068971079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c7c1037a0 0x7f5c7c199a40 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.300+0000 7f5c7be00640 1 -- 172.21.15.43:0/2068971079 >> 172.21.15.43:0/2068971079 conn(0x7f5c7c0fed10 msgr2=0x7f5c7c107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:32.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.300+0000 7f5c7be00640 1 -- 172.21.15.43:0/2068971079 shutdown_connections 2024-03-26T08:33:32.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:32.300+0000 7f5c7be00640 1 -- 172.21.15.43:0/2068971079 wait complete. 2024-03-26T08:33:32.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:31 smithi183 ceph-mon[35818]: pgmap v441: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:33:32.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:31 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/768312425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:33:32.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:31 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2845148707' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:33:32.307 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:31 smithi043 ceph-mon[31929]: pgmap v441: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:33:32.308 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:31 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/768312425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:33:32.308 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:31 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2845148707' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:33:32.310 INFO:tasks.cephadm:Setup complete, yielding 2024-03-26T08:33:32.310 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-26T08:33:32.320 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-03-26T08:33:32.320 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- bash -c 'ceph orch status' 2024-03-26T08:33:32.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:31 smithi052 ceph-mon[35071]: pgmap v441: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:33:32.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:31 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/768312425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:33:32.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:31 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2845148707' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:33:32.773 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:33:33.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:32 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2389382140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:33:33.301 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:32 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2389382140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:33:33.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:32 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2389382140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:33:34.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:33 smithi043 ceph-mon[31929]: pgmap v442: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:34.224 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:33 smithi052 ceph-mon[35071]: pgmap v442: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:34.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:33 smithi183 ceph-mon[35818]: pgmap v442: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:35.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.211+0000 7fb64ac00700 1 -- 172.21.15.43:0/3695454863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6441033d0 msgr2=0x7fb644103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:35.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.211+0000 7fb64ac00700 1 --2- 172.21.15.43:0/3695454863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6441033d0 0x7fb644103830 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb6300099c0 tx=0x7fb630009cd0 comp rx=0 tx=0).stop 2024-03-26T08:33:35.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.211+0000 7fb64ac00700 1 -- 172.21.15.43:0/3695454863 shutdown_connections 2024-03-26T08:33:35.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.211+0000 7fb64ac00700 1 --2- 172.21.15.43:0/3695454863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb644103d70 0x7fb64410a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:35.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.211+0000 7fb64ac00700 1 --2- 172.21.15.43:0/3695454863 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6441021d0 0x7fb6441025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:35.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.211+0000 7fb64ac00700 1 --2- 172.21.15.43:0/3695454863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6441033d0 0x7fb644103830 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:35.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.211+0000 7fb64ac00700 1 -- 172.21.15.43:0/3695454863 >> 172.21.15.43:0/3695454863 conn(0x7fb6440fd730 msgr2=0x7fb6440ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:35.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.211+0000 7fb64ac00700 1 -- 172.21.15.43:0/3695454863 shutdown_connections 2024-03-26T08:33:35.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.211+0000 7fb64ac00700 1 -- 172.21.15.43:0/3695454863 wait complete. 2024-03-26T08:33:35.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.212+0000 7fb64ac00700 1 Processor -- start 2024-03-26T08:33:35.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.213+0000 7fb64ac00700 1 -- start start 2024-03-26T08:33:35.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.213+0000 7fb64ac00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6441021d0 0x7fb64419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:35.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.213+0000 7fb64ac00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6441033d0 0x7fb64419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:35.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.213+0000 7fb64ac00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb644103d70 0x7fb6441a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:35.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.213+0000 7fb64ac00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64410dc90 con 0x7fb644103d70 2024-03-26T08:33:35.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.213+0000 7fb64ac00700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64410db10 con 0x7fb6441021d0 2024-03-26T08:33:35.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.213+0000 7fb64ac00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64410de10 con 0x7fb6441033d0 2024-03-26T08:33:35.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.213+0000 7fb648c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb644103d70 0x7fb6441a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:35.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.213+0000 7fb648c00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb644103d70 0x7fb6441a1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59112/0 (socket says 172.21.15.43:59112) 2024-03-26T08:33:35.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.213+0000 7fb648c00700 1 -- 172.21.15.43:0/1891257593 learned_addr learned my addr 172.21.15.43:0/1891257593 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:35.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.214+0000 7fb643400700 1 --2- 172.21.15.43:0/1891257593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6441033d0 0x7fb64419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:35.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.214+0000 7fb643e00700 1 --2- 172.21.15.43:0/1891257593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6441021d0 0x7fb64419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:35.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.214+0000 7fb648c00700 1 -- 172.21.15.43:0/1891257593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6441033d0 msgr2=0x7fb64419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:35.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.214+0000 7fb648c00700 1 --2- 172.21.15.43:0/1891257593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6441033d0 0x7fb64419aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:35.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.214+0000 7fb648c00700 1 -- 172.21.15.43:0/1891257593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6441021d0 msgr2=0x7fb64419a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:35.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.214+0000 7fb648c00700 1 --2- 172.21.15.43:0/1891257593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6441021d0 0x7fb64419a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:35.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.214+0000 7fb648c00700 1 -- 172.21.15.43:0/1891257593 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6441a21e0 con 0x7fb644103d70 2024-03-26T08:33:35.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.214+0000 7fb648c00700 1 --2- 172.21.15.43:0/1891257593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb644103d70 0x7fb6441a1b40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb63c007f10 tx=0x7fb63c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:35.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.215+0000 7fb640c00700 1 -- 172.21.15.43:0/1891257593 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c004bb0 con 0x7fb644103d70 2024-03-26T08:33:35.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.215+0000 7fb640c00700 1 -- 172.21.15.43:0/1891257593 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb63c004d50 con 0x7fb644103d70 2024-03-26T08:33:35.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.215+0000 7fb64ac00700 1 -- 172.21.15.43:0/1891257593 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6441a24d0 con 0x7fb644103d70 2024-03-26T08:33:35.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.215+0000 7fb640c00700 1 -- 172.21.15.43:0/1891257593 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c0058e0 con 0x7fb644103d70 2024-03-26T08:33:35.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.215+0000 7fb64ac00700 1 -- 172.21.15.43:0/1891257593 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6441a2a60 con 0x7fb644103d70 2024-03-26T08:33:35.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.216+0000 7fb64ac00700 1 -- 172.21.15.43:0/1891257593 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb644068e50 con 0x7fb644103d70 2024-03-26T08:33:35.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.218+0000 7fb640c00700 1 -- 172.21.15.43:0/1891257593 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb63c01a020 con 0x7fb644103d70 2024-03-26T08:33:35.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.220+0000 7fb640c00700 1 --2- 172.21.15.43:0/1891257593 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb6280a2910 0x7fb6280a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:35.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.220+0000 7fb640c00700 1 -- 172.21.15.43:0/1891257593 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7fb63c0cea70 con 0x7fb644103d70 2024-03-26T08:33:35.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.220+0000 7fb643e00700 1 --2- 172.21.15.43:0/1891257593 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb6280a2910 0x7fb6280a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:35.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.221+0000 7fb643e00700 1 --2- 172.21.15.43:0/1891257593 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb6280a2910 0x7fb6280a4dd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb644103230 tx=0x7fb634009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:35.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.223+0000 7fb640c00700 1 -- 172.21.15.43:0/1891257593 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb63c010070 con 0x7fb644103d70 2024-03-26T08:33:35.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.469+0000 7fb64ac00700 1 -- 172.21.15.43:0/1891257593 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb644075ae0 con 0x7fb6280a2910 2024-03-26T08:33:35.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.473+0000 7fb640c00700 1 -- 172.21.15.43:0/1891257593 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fb644075ae0 con 0x7fb6280a2910 2024-03-26T08:33:35.466 INFO:teuthology.orchestra.run.smithi043.stdout:Backend: cephadm 2024-03-26T08:33:35.466 INFO:teuthology.orchestra.run.smithi043.stdout:Available: Yes 2024-03-26T08:33:35.467 INFO:teuthology.orchestra.run.smithi043.stdout:Paused: No 2024-03-26T08:33:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.477+0000 7fb64ac00700 1 -- 172.21.15.43:0/1891257593 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb6280a2910 msgr2=0x7fb6280a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.477+0000 7fb64ac00700 1 --2- 172.21.15.43:0/1891257593 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb6280a2910 0x7fb6280a4dd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb644103230 tx=0x7fb634009260 comp rx=0 tx=0).stop 2024-03-26T08:33:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.477+0000 7fb64ac00700 1 -- 172.21.15.43:0/1891257593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb644103d70 msgr2=0x7fb6441a1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:35.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.477+0000 7fb64ac00700 1 --2- 172.21.15.43:0/1891257593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb644103d70 0x7fb6441a1b40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb63c007f10 tx=0x7fb63c00d3b0 comp rx=0 tx=0).stop 2024-03-26T08:33:35.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.477+0000 7fb64ac00700 1 -- 172.21.15.43:0/1891257593 shutdown_connections 2024-03-26T08:33:35.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.477+0000 7fb64ac00700 1 --2- 172.21.15.43:0/1891257593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb644103d70 0x7fb6441a1b40 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:35.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.477+0000 7fb64ac00700 1 --2- 172.21.15.43:0/1891257593 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb6280a2910 0x7fb6280a4dd0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:35.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.477+0000 7fb64ac00700 1 --2- 172.21.15.43:0/1891257593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6441021d0 0x7fb64419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:35.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.477+0000 7fb64ac00700 1 --2- 172.21.15.43:0/1891257593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6441033d0 0x7fb64419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:35.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.477+0000 7fb64ac00700 1 -- 172.21.15.43:0/1891257593 >> 172.21.15.43:0/1891257593 conn(0x7fb6440fd730 msgr2=0x7fb6440743c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:35.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.478+0000 7fb64ac00700 1 -- 172.21.15.43:0/1891257593 shutdown_connections 2024-03-26T08:33:35.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:35.478+0000 7fb64ac00700 1 -- 172.21.15.43:0/1891257593 wait complete. 2024-03-26T08:33:36.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:35 smithi043 ceph-mon[31929]: pgmap v443: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:36.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:35 smithi183 ceph-mon[35818]: pgmap v443: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:36.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:35 smithi052 ceph-mon[35071]: pgmap v443: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:36.326 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-26T08:33:36.779 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:33:37.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:36 smithi043 ceph-mon[31929]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:33:37.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:36 smithi183 ceph-mon[35818]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:33:37.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:36 smithi052 ceph-mon[35071]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:33:38.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:37 smithi043 ceph-mon[31929]: pgmap v444: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:33:38.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:37 smithi183 ceph-mon[35818]: pgmap v444: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:33:38.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:37 smithi052 ceph-mon[35071]: pgmap v444: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:33:39.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.320+0000 7f5c4e200700 1 -- 172.21.15.43:0/927020381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c4810e7b0 msgr2=0x7f5c48110ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:39.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.320+0000 7f5c4e200700 1 --2- 172.21.15.43:0/927020381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c4810e7b0 0x7f5c48110ba0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000b0a0 tx=0x7f5c4000b3b0 comp rx=0 tx=0).stop 2024-03-26T08:33:39.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.320+0000 7f5c4e200700 1 -- 172.21.15.43:0/927020381 shutdown_connections 2024-03-26T08:33:39.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.320+0000 7f5c4e200700 1 --2- 172.21.15.43:0/927020381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c4810e7b0 0x7f5c48110ba0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:39.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.320+0000 7f5c4e200700 1 --2- 172.21.15.43:0/927020381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c480761c0 0x7f5c48074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:39.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.320+0000 7f5c4e200700 1 --2- 172.21.15.43:0/927020381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c48074bf0 0x7f5c48075070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:39.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.320+0000 7f5c4e200700 1 -- 172.21.15.43:0/927020381 >> 172.21.15.43:0/927020381 conn(0x7f5c480fd730 msgr2=0x7f5c480ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:39.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.320+0000 7f5c4e200700 1 -- 172.21.15.43:0/927020381 shutdown_connections 2024-03-26T08:33:39.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.320+0000 7f5c4e200700 1 -- 172.21.15.43:0/927020381 wait complete. 2024-03-26T08:33:39.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.321+0000 7f5c4e200700 1 Processor -- start 2024-03-26T08:33:39.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.322+0000 7f5c4e200700 1 -- start start 2024-03-26T08:33:39.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.322+0000 7f5c4e200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c48074bf0 0x7f5c4819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:39.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.322+0000 7f5c4e200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c480761c0 0x7f5c4819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:39.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.322+0000 7f5c4e200700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c4810e7b0 0x7f5c481a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:39.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.322+0000 7f5c4e200700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c481142d0 con 0x7f5c48074bf0 2024-03-26T08:33:39.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.322+0000 7f5c4e200700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c48114150 con 0x7f5c4810e7b0 2024-03-26T08:33:39.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.322+0000 7f5c4e200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c48114450 con 0x7f5c480761c0 2024-03-26T08:33:39.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.322+0000 7f5c47400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c48074bf0 0x7f5c4819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:39.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.322+0000 7f5c47400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c48074bf0 0x7f5c4819e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59120/0 (socket says 172.21.15.43:59120) 2024-03-26T08:33:39.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.322+0000 7f5c47400700 1 -- 172.21.15.43:0/2481754113 learned_addr learned my addr 172.21.15.43:0/2481754113 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:39.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.323+0000 7f5c46a00700 1 --2- 172.21.15.43:0/2481754113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c480761c0 0x7f5c4819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:39.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.323+0000 7f5c47e00700 1 --2- 172.21.15.43:0/2481754113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c4810e7b0 0x7f5c481a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:39.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.323+0000 7f5c47400700 1 -- 172.21.15.43:0/2481754113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c480761c0 msgr2=0x7f5c4819ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:39.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.323+0000 7f5c47400700 1 --2- 172.21.15.43:0/2481754113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c480761c0 0x7f5c4819ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:39.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.323+0000 7f5c47400700 1 -- 172.21.15.43:0/2481754113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c4810e7b0 msgr2=0x7f5c481a5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:39.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.323+0000 7f5c47400700 1 --2- 172.21.15.43:0/2481754113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c4810e7b0 0x7f5c481a5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:39.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.323+0000 7f5c47400700 1 -- 172.21.15.43:0/2481754113 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c481a6590 con 0x7f5c48074bf0 2024-03-26T08:33:39.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.323+0000 7f5c47400700 1 --2- 172.21.15.43:0/2481754113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c48074bf0 0x7f5c4819e930 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800d980 tx=0x7f5c3800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:39.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.324+0000 7f5c3fe00700 1 -- 172.21.15.43:0/2481754113 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c38017070 con 0x7f5c48074bf0 2024-03-26T08:33:39.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.324+0000 7f5c3fe00700 1 -- 172.21.15.43:0/2481754113 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5c3800de60 con 0x7f5c48074bf0 2024-03-26T08:33:39.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.324+0000 7f5c4e200700 1 -- 172.21.15.43:0/2481754113 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c481a6880 con 0x7f5c48074bf0 2024-03-26T08:33:39.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.324+0000 7f5c3fe00700 1 -- 172.21.15.43:0/2481754113 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c380053a0 con 0x7f5c48074bf0 2024-03-26T08:33:39.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.324+0000 7f5c4e200700 1 -- 172.21.15.43:0/2481754113 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c481a6dc0 con 0x7f5c48074bf0 2024-03-26T08:33:39.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.325+0000 7f5c4e200700 1 -- 172.21.15.43:0/2481754113 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c48068e50 con 0x7f5c48074bf0 2024-03-26T08:33:39.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.327+0000 7f5c3fe00700 1 -- 172.21.15.43:0/2481754113 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5c38005540 con 0x7f5c48074bf0 2024-03-26T08:33:39.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.328+0000 7f5c3fe00700 1 --2- 172.21.15.43:0/2481754113 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5c2c0a2910 0x7f5c2c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:39.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.328+0000 7f5c3fe00700 1 -- 172.21.15.43:0/2481754113 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f5c380ceaa0 con 0x7f5c48074bf0 2024-03-26T08:33:39.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.329+0000 7f5c46a00700 1 --2- 172.21.15.43:0/2481754113 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5c2c0a2910 0x7f5c2c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:39.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.329+0000 7f5c46a00700 1 --2- 172.21.15.43:0/2481754113 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5c2c0a2910 0x7f5c2c0a4dd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5c34005950 tx=0x7f5c340058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:39.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.331+0000 7f5c3fe00700 1 -- 172.21.15.43:0/2481754113 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5c3809b7e0 con 0x7f5c48074bf0 2024-03-26T08:33:39.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.569+0000 7f5c4e200700 1 -- 172.21.15.43:0/2481754113 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c48062690 con 0x7f5c2c0a2910 2024-03-26T08:33:39.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.612+0000 7f5c3fe00700 1 -- 172.21.15.43:0/2481754113 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4123 (secure 0 0 0) 0x7f5c48062690 con 0x7f5c2c0a2910 2024-03-26T08:33:39.605 INFO:teuthology.orchestra.run.smithi043.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-26T08:33:39.606 INFO:teuthology.orchestra.run.smithi043.stdout:mds.a smithi043 running (2m) 31s ago 2m 12.2M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 e7393de82697 2024-03-26T08:33:39.606 INFO:teuthology.orchestra.run.smithi043.stdout:mds.b smithi052 running (2m) 72s ago 2m 24.6M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 233fc0c0ef08 2024-03-26T08:33:39.606 INFO:teuthology.orchestra.run.smithi043.stdout:mds.c smithi183 running (2m) 73s ago 2m 12.3M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 acfb6a572373 2024-03-26T08:33:39.606 INFO:teuthology.orchestra.run.smithi043.stdout:mds.d smithi043 running (2m) 31s ago 2m 12.5M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 68d36da8a4dc 2024-03-26T08:33:39.606 INFO:teuthology.orchestra.run.smithi043.stdout:mds.e smithi052 running (2m) 72s ago 2m 12.4M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 7f77c1bd618d 2024-03-26T08:33:39.606 INFO:teuthology.orchestra.run.smithi043.stdout:mds.f smithi183 running (2m) 73s ago 2m 12.4M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 321fef0effb1 2024-03-26T08:33:39.606 INFO:teuthology.orchestra.run.smithi043.stdout:mds.g smithi043 running (2m) 31s ago 2m 12.3M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 27c24afd4d05 2024-03-26T08:33:39.606 INFO:teuthology.orchestra.run.smithi043.stdout:mds.h smithi052 running (2m) 72s ago 2m 15.7M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 c09851e5d51e 2024-03-26T08:33:39.606 INFO:teuthology.orchestra.run.smithi043.stdout:mds.i smithi183 running (2m) 73s ago 2m 12.2M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 756510d7e47e 2024-03-26T08:33:39.606 INFO:teuthology.orchestra.run.smithi043.stdout:mds.j smithi043 running (3m) 31s ago 3m 12.6M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 de9f0ba93782 2024-03-26T08:33:39.606 INFO:teuthology.orchestra.run.smithi043.stdout:mds.k smithi052 running (3m) 72s ago 3m 12.4M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 b0d46f3ecabb 2024-03-26T08:33:39.606 INFO:teuthology.orchestra.run.smithi043.stdout:mds.l smithi183 running (3m) 73s ago 3m 12.5M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 476ecfbdfdd3 2024-03-26T08:33:39.607 INFO:teuthology.orchestra.run.smithi043.stdout:mgr.x smithi043 *:9283,8765 running (13m) 31s ago 13m 514M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 ec1554ba1ef0 2024-03-26T08:33:39.607 INFO:teuthology.orchestra.run.smithi043.stdout:mgr.y smithi052 *:8443,8765 running (10m) 72s ago 10m 451M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 1bdae692a339 2024-03-26T08:33:39.607 INFO:teuthology.orchestra.run.smithi043.stdout:mgr.z smithi183 *:8443,8765 running (10m) 73s ago 10m 451M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 5eb7852de1c7 2024-03-26T08:33:39.607 INFO:teuthology.orchestra.run.smithi043.stdout:mon.a smithi043 running (13m) 31s ago 13m 49.6M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 b43fc9922941 2024-03-26T08:33:39.607 INFO:teuthology.orchestra.run.smithi043.stdout:mon.b smithi052 running (10m) 72s ago 10m 39.7M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 02f8c34e593d 2024-03-26T08:33:39.607 INFO:teuthology.orchestra.run.smithi043.stdout:mon.c smithi183 running (10m) 73s ago 10m 39.5M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 fc6a11acaf9f 2024-03-26T08:33:39.607 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0 smithi043 running (10m) 31s ago 10m 45.3M 4273M 19.0.0-2145-g3be9a5c9 c664afe56bb0 259b59fc83dd 2024-03-26T08:33:39.607 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1 smithi052 running (9m) 72s ago 9m 46.0M 4273M 19.0.0-2145-g3be9a5c9 c664afe56bb0 08dd3012fb58 2024-03-26T08:33:39.607 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2 smithi183 running (8m) 73s ago 8m 55.4M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 9820bbd3b513 2024-03-26T08:33:39.607 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3 smithi043 running (8m) 31s ago 8m 52.7M 4273M 19.0.0-2145-g3be9a5c9 c664afe56bb0 3ae00888cfc2 2024-03-26T08:33:39.607 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4 smithi052 running (7m) 72s ago 7m 52.8M 4273M 19.0.0-2145-g3be9a5c9 c664afe56bb0 25370584c031 2024-03-26T08:33:39.607 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5 smithi183 running (7m) 73s ago 7m 55.0M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 f6e9d594387f 2024-03-26T08:33:39.607 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6 smithi043 running (6m) 31s ago 6m 52.6M 4273M 19.0.0-2145-g3be9a5c9 c664afe56bb0 0b46c1b15612 2024-03-26T08:33:39.607 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7 smithi052 running (5m) 72s ago 5m 54.8M 4273M 19.0.0-2145-g3be9a5c9 c664afe56bb0 fb994d47de12 2024-03-26T08:33:39.608 INFO:teuthology.orchestra.run.smithi043.stdout:osd.8 smithi183 running (5m) 73s ago 5m 41.6M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 1fc6782bae0f 2024-03-26T08:33:39.608 INFO:teuthology.orchestra.run.smithi043.stdout:osd.9 smithi043 running (4m) 31s ago 4m 42.2M 4273M 19.0.0-2145-g3be9a5c9 c664afe56bb0 fdddf9ad3486 2024-03-26T08:33:39.608 INFO:teuthology.orchestra.run.smithi043.stdout:osd.10 smithi052 running (4m) 72s ago 4m 53.3M 4273M 19.0.0-2145-g3be9a5c9 c664afe56bb0 d9204bafa8f7 2024-03-26T08:33:39.608 INFO:teuthology.orchestra.run.smithi043.stdout:osd.11 smithi183 running (3m) 73s ago 3m 42.4M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 08d09c25149a 2024-03-26T08:33:39.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.615+0000 7f5c4e200700 1 -- 172.21.15.43:0/2481754113 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5c2c0a2910 msgr2=0x7f5c2c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:39.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.615+0000 7f5c4e200700 1 --2- 172.21.15.43:0/2481754113 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5c2c0a2910 0x7f5c2c0a4dd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5c34005950 tx=0x7f5c340058e0 comp rx=0 tx=0).stop 2024-03-26T08:33:39.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.615+0000 7f5c4e200700 1 -- 172.21.15.43:0/2481754113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c48074bf0 msgr2=0x7f5c4819e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:39.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.615+0000 7f5c4e200700 1 --2- 172.21.15.43:0/2481754113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c48074bf0 0x7f5c4819e930 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800d980 tx=0x7f5c3800dd40 comp rx=0 tx=0).stop 2024-03-26T08:33:39.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.615+0000 7f5c4e200700 1 -- 172.21.15.43:0/2481754113 shutdown_connections 2024-03-26T08:33:39.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.615+0000 7f5c4e200700 1 --2- 172.21.15.43:0/2481754113 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5c2c0a2910 0x7f5c2c0a4dd0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:39.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.615+0000 7f5c4e200700 1 --2- 172.21.15.43:0/2481754113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c4810e7b0 0x7f5c481a5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:39.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.615+0000 7f5c4e200700 1 --2- 172.21.15.43:0/2481754113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c48074bf0 0x7f5c4819e930 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:39.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.615+0000 7f5c4e200700 1 --2- 172.21.15.43:0/2481754113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c480761c0 0x7f5c4819ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:39.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.615+0000 7f5c4e200700 1 -- 172.21.15.43:0/2481754113 >> 172.21.15.43:0/2481754113 conn(0x7f5c480fd730 msgr2=0x7f5c48101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:39.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.616+0000 7f5c4e200700 1 -- 172.21.15.43:0/2481754113 shutdown_connections 2024-03-26T08:33:39.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:39.616+0000 7f5c4e200700 1 -- 172.21.15.43:0/2481754113 wait complete. 2024-03-26T08:33:40.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:39 smithi183 ceph-mon[35818]: pgmap v445: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:40.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:39 smithi052 ceph-mon[35071]: pgmap v445: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:40.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:39 smithi043 ceph-mon[31929]: pgmap v445: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:40.564 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-26T08:33:41.024 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:33:41.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:40 smithi043 ceph-mon[31929]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:33:41.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:40 smithi183 ceph-mon[35818]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:33:41.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:40 smithi052 ceph-mon[35071]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:33:42.253 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:41 smithi043 ceph-mon[31929]: pgmap v446: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:33:42.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:41 smithi183 ceph-mon[35818]: pgmap v446: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:33:42.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:41 smithi052 ceph-mon[35071]: pgmap v446: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:33:43.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.538+0000 7f6d4d400700 1 -- 172.21.15.43:0/1398492844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d48107eb0 msgr2=0x7f6d4810a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:43.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.538+0000 7f6d4d400700 1 --2- 172.21.15.43:0/1398492844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d48107eb0 0x7f6d4810a320 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6d4000b0a0 tx=0x7f6d4000b3b0 comp rx=0 tx=0).stop 2024-03-26T08:33:43.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.538+0000 7f6d4d400700 1 -- 172.21.15.43:0/1398492844 shutdown_connections 2024-03-26T08:33:43.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.538+0000 7f6d4d400700 1 --2- 172.21.15.43:0/1398492844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d48107eb0 0x7f6d4810a320 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:43.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.538+0000 7f6d4d400700 1 --2- 172.21.15.43:0/1398492844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d48102c50 0x7f6d48105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:43.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.538+0000 7f6d4d400700 1 --2- 172.21.15.43:0/1398492844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d48105580 0x7f6d48107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:43.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.538+0000 7f6d4d400700 1 -- 172.21.15.43:0/1398492844 >> 172.21.15.43:0/1398492844 conn(0x7f6d480fc660 msgr2=0x7f6d480feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:43.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.539+0000 7f6d4d400700 1 -- 172.21.15.43:0/1398492844 shutdown_connections 2024-03-26T08:33:43.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.539+0000 7f6d4d400700 1 -- 172.21.15.43:0/1398492844 wait complete. 2024-03-26T08:33:43.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.540+0000 7f6d4d400700 1 Processor -- start 2024-03-26T08:33:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.540+0000 7f6d4d400700 1 -- start start 2024-03-26T08:33:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.540+0000 7f6d4d400700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d48102c50 0x7f6d4819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.540+0000 7f6d4d400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d48105580 0x7f6d4819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.540+0000 7f6d4d400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d48107eb0 0x7f6d481a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.540+0000 7f6d4d400700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4810dca0 con 0x7f6d48107eb0 2024-03-26T08:33:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.540+0000 7f6d4d400700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4810db20 con 0x7f6d48102c50 2024-03-26T08:33:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.540+0000 7f6d4d400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4810de20 con 0x7f6d48105580 2024-03-26T08:33:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.541+0000 7f6d47200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d48107eb0 0x7f6d481a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.541+0000 7f6d47200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d48107eb0 0x7f6d481a1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59142/0 (socket says 172.21.15.43:59142) 2024-03-26T08:33:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.541+0000 7f6d47200700 1 -- 172.21.15.43:0/906270572 learned_addr learned my addr 172.21.15.43:0/906270572 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.541+0000 7f6d46800700 1 --2- 172.21.15.43:0/906270572 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d48102c50 0x7f6d4819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.541+0000 7f6d3fe00700 1 --2- 172.21.15.43:0/906270572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d48105580 0x7f6d4819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.541+0000 7f6d47200700 1 -- 172.21.15.43:0/906270572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d48105580 msgr2=0x7f6d4819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:43.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.541+0000 7f6d47200700 1 --2- 172.21.15.43:0/906270572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d48105580 0x7f6d4819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:43.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.541+0000 7f6d47200700 1 -- 172.21.15.43:0/906270572 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d48102c50 msgr2=0x7f6d4819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:43.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.541+0000 7f6d47200700 1 --2- 172.21.15.43:0/906270572 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d48102c50 0x7f6d4819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:43.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.541+0000 7f6d47200700 1 -- 172.21.15.43:0/906270572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d481a2140 con 0x7f6d48107eb0 2024-03-26T08:33:43.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.542+0000 7f6d47200700 1 --2- 172.21.15.43:0/906270572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d48107eb0 0x7f6d481a1aa0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6d40006070 tx=0x7f6d40004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:43.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.542+0000 7f6d3f400700 1 -- 172.21.15.43:0/906270572 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d40021070 con 0x7f6d48107eb0 2024-03-26T08:33:43.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.542+0000 7f6d3f400700 1 -- 172.21.15.43:0/906270572 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6d40007db0 con 0x7f6d48107eb0 2024-03-26T08:33:43.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.542+0000 7f6d3f400700 1 -- 172.21.15.43:0/906270572 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4001d450 con 0x7f6d48107eb0 2024-03-26T08:33:43.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.542+0000 7f6d4d400700 1 -- 172.21.15.43:0/906270572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d481a23d0 con 0x7f6d48107eb0 2024-03-26T08:33:43.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.542+0000 7f6d4d400700 1 -- 172.21.15.43:0/906270572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d481a28d0 con 0x7f6d48107eb0 2024-03-26T08:33:43.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.543+0000 7f6d4d400700 1 -- 172.21.15.43:0/906270572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d48068e50 con 0x7f6d48107eb0 2024-03-26T08:33:43.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.545+0000 7f6d3f400700 1 -- 172.21.15.43:0/906270572 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f6d40004140 con 0x7f6d48107eb0 2024-03-26T08:33:43.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.548+0000 7f6d3f400700 1 --2- 172.21.15.43:0/906270572 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6d2c0a2910 0x7f6d2c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:43.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.548+0000 7f6d3f400700 1 -- 172.21.15.43:0/906270572 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f6d4000f020 con 0x7f6d48107eb0 2024-03-26T08:33:43.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.548+0000 7f6d46800700 1 --2- 172.21.15.43:0/906270572 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6d2c0a2910 0x7f6d2c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:43.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.549+0000 7f6d46800700 1 --2- 172.21.15.43:0/906270572 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6d2c0a2910 0x7f6d2c0a4dd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6d38005950 tx=0x7f6d380058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:43.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.552+0000 7f6d3f400700 1 -- 172.21.15.43:0/906270572 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6d4002a020 con 0x7f6d48107eb0 2024-03-26T08:33:43.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.793+0000 7f6d4d400700 1 -- 172.21.15.43:0/906270572 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d480625d0 con 0x7f6d2c0a2910 2024-03-26T08:33:43.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.804+0000 7f6d3f400700 1 -- 172.21.15.43:0/906270572 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f6d480625d0 con 0x7f6d2c0a2910 2024-03-26T08:33:43.797 INFO:teuthology.orchestra.run.smithi043.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-26T08:33:43.798 INFO:teuthology.orchestra.run.smithi043.stdout:mds.all 12/12 77s ago 3m smithi043=a;smithi043=d;smithi043=g;smithi043=j;smithi052=b;smithi052=e;smithi052=h;smithi052=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2024-03-26T08:33:43.798 INFO:teuthology.orchestra.run.smithi043.stdout:mgr 3/3 77s ago 10m smithi043=x;smithi052=y;smithi183=z;count:3 2024-03-26T08:33:43.798 INFO:teuthology.orchestra.run.smithi043.stdout:mon 3/3 77s ago 11m smithi043:172.21.15.43=a;smithi052:172.21.15.52=b;smithi183:172.21.15.183=c;count:3 2024-03-26T08:33:43.798 INFO:teuthology.orchestra.run.smithi043.stdout:osd 12 77s ago - 2024-03-26T08:33:43.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.807+0000 7f6d4d400700 1 -- 172.21.15.43:0/906270572 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6d2c0a2910 msgr2=0x7f6d2c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:43.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.807+0000 7f6d4d400700 1 --2- 172.21.15.43:0/906270572 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6d2c0a2910 0x7f6d2c0a4dd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6d38005950 tx=0x7f6d380058e0 comp rx=0 tx=0).stop 2024-03-26T08:33:43.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.807+0000 7f6d4d400700 1 -- 172.21.15.43:0/906270572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d48107eb0 msgr2=0x7f6d481a1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:43.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.807+0000 7f6d4d400700 1 --2- 172.21.15.43:0/906270572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d48107eb0 0x7f6d481a1aa0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6d40006070 tx=0x7f6d40004ef0 comp rx=0 tx=0).stop 2024-03-26T08:33:43.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.808+0000 7f6d4d400700 1 -- 172.21.15.43:0/906270572 shutdown_connections 2024-03-26T08:33:43.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.808+0000 7f6d4d400700 1 --2- 172.21.15.43:0/906270572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d48107eb0 0x7f6d481a1aa0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:43.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.808+0000 7f6d4d400700 1 --2- 172.21.15.43:0/906270572 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f6d2c0a2910 0x7f6d2c0a4dd0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:43.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.808+0000 7f6d4d400700 1 --2- 172.21.15.43:0/906270572 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d48102c50 0x7f6d4819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:43.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.808+0000 7f6d4d400700 1 --2- 172.21.15.43:0/906270572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d48105580 0x7f6d4819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:43.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.808+0000 7f6d4d400700 1 -- 172.21.15.43:0/906270572 >> 172.21.15.43:0/906270572 conn(0x7f6d480fc660 msgr2=0x7f6d48102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:43.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.808+0000 7f6d4d400700 1 -- 172.21.15.43:0/906270572 shutdown_connections 2024-03-26T08:33:43.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:43.808+0000 7f6d4d400700 1 -- 172.21.15.43:0/906270572 wait complete. 2024-03-26T08:33:44.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:43 smithi043 ceph-mon[31929]: pgmap v447: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:44.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:43 smithi183 ceph-mon[35818]: pgmap v447: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:44.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:43 smithi052 ceph-mon[35071]: pgmap v447: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:44.840 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-26T08:33:45.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:44 smithi043 ceph-mon[31929]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:33:45.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:44 smithi183 ceph-mon[35818]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:33:45.312 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:33:45.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:44 smithi052 ceph-mon[35071]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:33:46.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:45 smithi043 ceph-mon[31929]: pgmap v448: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:46.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:45 smithi183 ceph-mon[35818]: pgmap v448: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:46.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:45 smithi052 ceph-mon[35071]: pgmap v448: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:47.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.652+0000 7f3420600700 1 -- 172.21.15.43:0/1266694611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3418103d50 msgr2=0x7f341810a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:47.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.652+0000 7f3420600700 1 --2- 172.21.15.43:0/1266694611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3418103d50 0x7f341810a600 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f341000b0d0 tx=0x7f341000b3e0 comp rx=0 tx=0).stop 2024-03-26T08:33:47.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.652+0000 7f3420600700 1 -- 172.21.15.43:0/1266694611 shutdown_connections 2024-03-26T08:33:47.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.652+0000 7f3420600700 1 --2- 172.21.15.43:0/1266694611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3418103d50 0x7f341810a600 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:47.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.652+0000 7f3420600700 1 --2- 172.21.15.43:0/1266694611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34181021b0 0x7f34181025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:47.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.652+0000 7f3420600700 1 --2- 172.21.15.43:0/1266694611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34181033b0 0x7f3418103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:47.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.652+0000 7f3420600700 1 -- 172.21.15.43:0/1266694611 >> 172.21.15.43:0/1266694611 conn(0x7f34180fd770 msgr2=0x7f34180ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:47.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.652+0000 7f3420600700 1 -- 172.21.15.43:0/1266694611 shutdown_connections 2024-03-26T08:33:47.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.652+0000 7f3420600700 1 -- 172.21.15.43:0/1266694611 wait complete. 2024-03-26T08:33:47.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.653+0000 7f3420600700 1 Processor -- start 2024-03-26T08:33:47.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.654+0000 7f3420600700 1 -- start start 2024-03-26T08:33:47.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.654+0000 7f3420600700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34181021b0 0x7f341819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:47.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.654+0000 7f3420600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34181033b0 0x7f341819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:47.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.654+0000 7f3420600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3418103d50 0x7f34181a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:47.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.654+0000 7f3420600700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341810dcb0 con 0x7f3418103d50 2024-03-26T08:33:47.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.654+0000 7f3420600700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341810db30 con 0x7f34181021b0 2024-03-26T08:33:47.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.654+0000 7f3420600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341810de30 con 0x7f34181033b0 2024-03-26T08:33:47.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.654+0000 7f341e600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3418103d50 0x7f34181a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:47.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.655+0000 7f341e600700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3418103d50 0x7f34181a1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57412/0 (socket says 172.21.15.43:57412) 2024-03-26T08:33:47.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.655+0000 7f341e600700 1 -- 172.21.15.43:0/2863212992 learned_addr learned my addr 172.21.15.43:0/2863212992 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:47.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.655+0000 7f341dc00700 1 --2- 172.21.15.43:0/2863212992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34181021b0 0x7f341819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:47.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.655+0000 7f341d200700 1 --2- 172.21.15.43:0/2863212992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34181033b0 0x7f341819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:47.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.655+0000 7f341e600700 1 -- 172.21.15.43:0/2863212992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34181033b0 msgr2=0x7f341819aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:47.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.655+0000 7f341e600700 1 --2- 172.21.15.43:0/2863212992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34181033b0 0x7f341819aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:47.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.655+0000 7f341e600700 1 -- 172.21.15.43:0/2863212992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34181021b0 msgr2=0x7f341819a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:47.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.655+0000 7f341e600700 1 --2- 172.21.15.43:0/2863212992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34181021b0 0x7f341819a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:47.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.655+0000 7f341e600700 1 -- 172.21.15.43:0/2863212992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34181a2170 con 0x7f3418103d50 2024-03-26T08:33:47.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.655+0000 7f341dc00700 1 --2- 172.21.15.43:0/2863212992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34181021b0 0x7f341819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:33:47.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.655+0000 7f341d200700 1 --2- 172.21.15.43:0/2863212992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34181033b0 0x7f341819aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:33:47.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.655+0000 7f341e600700 1 --2- 172.21.15.43:0/2863212992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3418103d50 0x7f34181a1ad0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3410004cb0 tx=0x7f3410004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:47.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.656+0000 7f3416000700 1 -- 172.21.15.43:0/2863212992 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341001f030 con 0x7f3418103d50 2024-03-26T08:33:47.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.656+0000 7f3416000700 1 -- 172.21.15.43:0/2863212992 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3410007ac0 con 0x7f3418103d50 2024-03-26T08:33:47.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.656+0000 7f3416000700 1 -- 172.21.15.43:0/2863212992 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341001b690 con 0x7f3418103d50 2024-03-26T08:33:47.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.656+0000 7f3420600700 1 -- 172.21.15.43:0/2863212992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34181a2400 con 0x7f3418103d50 2024-03-26T08:33:47.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.656+0000 7f3420600700 1 -- 172.21.15.43:0/2863212992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34181a2800 con 0x7f3418103d50 2024-03-26T08:33:47.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.657+0000 7f3420600700 1 -- 172.21.15.43:0/2863212992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3418068e80 con 0x7f3418103d50 2024-03-26T08:33:47.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.659+0000 7f3416000700 1 -- 172.21.15.43:0/2863212992 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f3410022030 con 0x7f3418103d50 2024-03-26T08:33:47.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.661+0000 7f3416000700 1 --2- 172.21.15.43:0/2863212992 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f34000a2910 0x7f34000a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:47.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.661+0000 7f3416000700 1 -- 172.21.15.43:0/2863212992 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f341000f020 con 0x7f3418103d50 2024-03-26T08:33:47.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.661+0000 7f341dc00700 1 --2- 172.21.15.43:0/2863212992 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f34000a2910 0x7f34000a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:47.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.662+0000 7f341dc00700 1 --2- 172.21.15.43:0/2863212992 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f34000a2910 0x7f34000a4dd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f340c005950 tx=0x7f340c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:47.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.666+0000 7f3416000700 1 -- 172.21.15.43:0/2863212992 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3418068e80 con 0x7f3418103d50 2024-03-26T08:33:47.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.902+0000 7f3420600700 1 -- 172.21.15.43:0/2863212992 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3418075ae0 con 0x7f34000a2910 2024-03-26T08:33:47.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.906+0000 7f3416000700 1 -- 172.21.15.43:0/2863212992 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f3418075ae0 con 0x7f34000a2910 2024-03-26T08:33:47.899 INFO:teuthology.orchestra.run.smithi043.stdout:HOST ADDR LABELS STATUS 2024-03-26T08:33:47.899 INFO:teuthology.orchestra.run.smithi043.stdout:smithi043 172.21.15.43 2024-03-26T08:33:47.899 INFO:teuthology.orchestra.run.smithi043.stdout:smithi052 172.21.15.52 2024-03-26T08:33:47.899 INFO:teuthology.orchestra.run.smithi043.stdout:smithi183 172.21.15.183 2024-03-26T08:33:47.900 INFO:teuthology.orchestra.run.smithi043.stdout:3 hosts in cluster 2024-03-26T08:33:47.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.910+0000 7f3420600700 1 -- 172.21.15.43:0/2863212992 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f34000a2910 msgr2=0x7f34000a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:47.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.910+0000 7f3420600700 1 --2- 172.21.15.43:0/2863212992 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f34000a2910 0x7f34000a4dd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f340c005950 tx=0x7f340c0058e0 comp rx=0 tx=0).stop 2024-03-26T08:33:47.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.910+0000 7f3420600700 1 -- 172.21.15.43:0/2863212992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3418103d50 msgr2=0x7f34181a1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:47.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.910+0000 7f3420600700 1 --2- 172.21.15.43:0/2863212992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3418103d50 0x7f34181a1ad0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3410004cb0 tx=0x7f3410004d90 comp rx=0 tx=0).stop 2024-03-26T08:33:47.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.910+0000 7f3420600700 1 -- 172.21.15.43:0/2863212992 shutdown_connections 2024-03-26T08:33:47.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.910+0000 7f3420600700 1 --2- 172.21.15.43:0/2863212992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3418103d50 0x7f34181a1ad0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:47.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.910+0000 7f3420600700 1 --2- 172.21.15.43:0/2863212992 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f34000a2910 0x7f34000a4dd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:47.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.910+0000 7f3420600700 1 --2- 172.21.15.43:0/2863212992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34181021b0 0x7f341819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:47.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.910+0000 7f3420600700 1 --2- 172.21.15.43:0/2863212992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34181033b0 0x7f341819aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:47.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.910+0000 7f3420600700 1 -- 172.21.15.43:0/2863212992 >> 172.21.15.43:0/2863212992 conn(0x7f34180fd770 msgr2=0x7f34180743c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:47.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.910+0000 7f3420600700 1 -- 172.21.15.43:0/2863212992 shutdown_connections 2024-03-26T08:33:47.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:47.910+0000 7f3420600700 1 -- 172.21.15.43:0/2863212992 wait complete. 2024-03-26T08:33:48.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:47 smithi043 ceph-mon[31929]: pgmap v449: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:33:48.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:47 smithi183 ceph-mon[35818]: pgmap v449: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:33:48.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:47 smithi052 ceph-mon[35071]: pgmap v449: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:33:48.902 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-26T08:33:49.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:49 smithi043 ceph-mon[31929]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:33:49.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:49 smithi183 ceph-mon[35818]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:33:49.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:49 smithi052 ceph-mon[35071]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:33:49.373 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:33:50.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:50 smithi043 ceph-mon[31929]: pgmap v450: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:50.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:50 smithi183 ceph-mon[35818]: pgmap v450: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:50.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:50 smithi052 ceph-mon[35071]: pgmap v450: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.720+0000 7f5f95400700 1 -- 172.21.15.43:0/2378514869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f90103410 msgr2=0x7f5f90103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.720+0000 7f5f95400700 1 --2- 172.21.15.43:0/2378514869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f90103410 0x7f5f90103870 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5f780099c0 tx=0x7f5f78009cd0 comp rx=0 tx=0).stop 2024-03-26T08:33:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.720+0000 7f5f95400700 1 -- 172.21.15.43:0/2378514869 shutdown_connections 2024-03-26T08:33:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.720+0000 7f5f95400700 1 --2- 172.21.15.43:0/2378514869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f90103db0 0x7f5f9010a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.720+0000 7f5f95400700 1 --2- 172.21.15.43:0/2378514869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f90102210 0x7f5f90102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.720+0000 7f5f95400700 1 --2- 172.21.15.43:0/2378514869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f90103410 0x7f5f90103870 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.720+0000 7f5f95400700 1 -- 172.21.15.43:0/2378514869 >> 172.21.15.43:0/2378514869 conn(0x7f5f900fd7b0 msgr2=0x7f5f900ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.720+0000 7f5f95400700 1 -- 172.21.15.43:0/2378514869 shutdown_connections 2024-03-26T08:33:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.720+0000 7f5f95400700 1 -- 172.21.15.43:0/2378514869 wait complete. 2024-03-26T08:33:51.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.721+0000 7f5f95400700 1 Processor -- start 2024-03-26T08:33:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.722+0000 7f5f95400700 1 -- start start 2024-03-26T08:33:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.722+0000 7f5f95400700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f90102210 0x7f5f90075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.722+0000 7f5f95400700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f90103410 0x7f5f90074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.722+0000 7f5f95400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f90103db0 0x7f5f90074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.722+0000 7f5f95400700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9010dd00 con 0x7f5f90103db0 2024-03-26T08:33:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.722+0000 7f5f95400700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9010db80 con 0x7f5f90102210 2024-03-26T08:33:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.722+0000 7f5f95400700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9010de80 con 0x7f5f90103410 2024-03-26T08:33:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.722+0000 7f5f8f200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f90103db0 0x7f5f90074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.722+0000 7f5f8f200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f90103db0 0x7f5f90074ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57432/0 (socket says 172.21.15.43:57432) 2024-03-26T08:33:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.722+0000 7f5f8f200700 1 -- 172.21.15.43:0/768686923 learned_addr learned my addr 172.21.15.43:0/768686923 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.723+0000 7f5f8e800700 1 --2- 172.21.15.43:0/768686923 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f90102210 0x7f5f90075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.723+0000 7f5f87e00700 1 --2- 172.21.15.43:0/768686923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f90103410 0x7f5f90074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.723+0000 7f5f8f200700 1 -- 172.21.15.43:0/768686923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f90103410 msgr2=0x7f5f90074580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.723+0000 7f5f8f200700 1 --2- 172.21.15.43:0/768686923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f90103410 0x7f5f90074580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.723+0000 7f5f8f200700 1 -- 172.21.15.43:0/768686923 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f90102210 msgr2=0x7f5f90075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.723+0000 7f5f8f200700 1 --2- 172.21.15.43:0/768686923 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f90102210 0x7f5f90075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.723+0000 7f5f8f200700 1 -- 172.21.15.43:0/768686923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f90075370 con 0x7f5f90103db0 2024-03-26T08:33:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.723+0000 7f5f87e00700 1 --2- 172.21.15.43:0/768686923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f90103410 0x7f5f90074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:33:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.723+0000 7f5f8e800700 1 --2- 172.21.15.43:0/768686923 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f90102210 0x7f5f90075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:33:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.724+0000 7f5f8f200700 1 --2- 172.21.15.43:0/768686923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f90103db0 0x7f5f90074ac0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5f88007d70 tx=0x7f5f8800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.724+0000 7f5f87400700 1 -- 172.21.15.43:0/768686923 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f88016020 con 0x7f5f90103db0 2024-03-26T08:33:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.724+0000 7f5f87400700 1 -- 172.21.15.43:0/768686923 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5f8800ad40 con 0x7f5f90103db0 2024-03-26T08:33:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.724+0000 7f5f87400700 1 -- 172.21.15.43:0/768686923 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f88011620 con 0x7f5f90103db0 2024-03-26T08:33:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.724+0000 7f5f95400700 1 -- 172.21.15.43:0/768686923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f901a2450 con 0x7f5f90103db0 2024-03-26T08:33:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.724+0000 7f5f95400700 1 -- 172.21.15.43:0/768686923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f901a2910 con 0x7f5f90103db0 2024-03-26T08:33:51.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.725+0000 7f5f95400700 1 -- 172.21.15.43:0/768686923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f90068e50 con 0x7f5f90103db0 2024-03-26T08:33:51.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.728+0000 7f5f87400700 1 -- 172.21.15.43:0/768686923 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5f88019020 con 0x7f5f90103db0 2024-03-26T08:33:51.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.729+0000 7f5f87400700 1 --2- 172.21.15.43:0/768686923 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5f740a2910 0x7f5f740a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:51.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.729+0000 7f5f87400700 1 -- 172.21.15.43:0/768686923 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f5f880cedc0 con 0x7f5f90103db0 2024-03-26T08:33:51.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.729+0000 7f5f8e800700 1 --2- 172.21.15.43:0/768686923 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5f740a2910 0x7f5f740a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:51.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.730+0000 7f5f8e800700 1 --2- 172.21.15.43:0/768686923 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5f740a2910 0x7f5f740a4dd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5f80005950 tx=0x7f5f800058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:51.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.734+0000 7f5f87400700 1 -- 172.21.15.43:0/768686923 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5f88020020 con 0x7f5f90103db0 2024-03-26T08:33:51.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.972+0000 7f5f95400700 1 -- 172.21.15.43:0/768686923 --> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f900625d0 con 0x7f5f740a2910 2024-03-26T08:33:51.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.975+0000 7f5f87400700 1 -- 172.21.15.43:0/768686923 <== mgr.14152 v2:172.21.15.43:6800/3390816575 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f5f900625d0 con 0x7f5f740a2910 2024-03-26T08:33:51.968 INFO:teuthology.orchestra.run.smithi043.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-26T08:33:51.968 INFO:teuthology.orchestra.run.smithi043.stdout:smithi043 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300GH480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-03-26T08:33:51.968 INFO:teuthology.orchestra.run.smithi043.stdout:smithi052 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513010N480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-03-26T08:33:51.968 INFO:teuthology.orchestra.run.smithi043.stdout:smithi183 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500QJ400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-26T08:33:51.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.980+0000 7f5f95400700 1 -- 172.21.15.43:0/768686923 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5f740a2910 msgr2=0x7f5f740a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:51.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.980+0000 7f5f95400700 1 --2- 172.21.15.43:0/768686923 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5f740a2910 0x7f5f740a4dd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5f80005950 tx=0x7f5f800058e0 comp rx=0 tx=0).stop 2024-03-26T08:33:51.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.980+0000 7f5f95400700 1 -- 172.21.15.43:0/768686923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f90103db0 msgr2=0x7f5f90074ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:51.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.980+0000 7f5f95400700 1 --2- 172.21.15.43:0/768686923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f90103db0 0x7f5f90074ac0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5f88007d70 tx=0x7f5f8800a340 comp rx=0 tx=0).stop 2024-03-26T08:33:51.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.980+0000 7f5f95400700 1 -- 172.21.15.43:0/768686923 shutdown_connections 2024-03-26T08:33:51.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.980+0000 7f5f95400700 1 --2- 172.21.15.43:0/768686923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f90103db0 0x7f5f90074ac0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:51.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.980+0000 7f5f95400700 1 --2- 172.21.15.43:0/768686923 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f5f740a2910 0x7f5f740a4dd0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:51.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.980+0000 7f5f95400700 1 --2- 172.21.15.43:0/768686923 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5f90102210 0x7f5f90075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:51.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.980+0000 7f5f95400700 1 --2- 172.21.15.43:0/768686923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f90103410 0x7f5f90074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:51.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.980+0000 7f5f95400700 1 -- 172.21.15.43:0/768686923 >> 172.21.15.43:0/768686923 conn(0x7f5f900fd7b0 msgr2=0x7f5f90106710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:51.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.980+0000 7f5f95400700 1 -- 172.21.15.43:0/768686923 shutdown_connections 2024-03-26T08:33:51.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:51.980+0000 7f5f95400700 1 -- 172.21.15.43:0/768686923 wait complete. 2024-03-26T08:33:52.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:52 smithi183 ceph-mon[35818]: pgmap v451: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:33:52.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:52 smithi052 ceph-mon[35071]: pgmap v451: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:33:52.344 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:52 smithi043 ceph-mon[31929]: pgmap v451: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:33:52.826 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-26T08:33:52.836 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi043.front.sepia.ceph.com 2024-03-26T08:33:52.836 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-26T08:33:53.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:53 smithi183 ceph-mon[35818]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:33:53.310 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:33:53.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:53 smithi052 ceph-mon[35071]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:33:53.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:53 smithi043 ceph-mon[31929]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:33:54.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:54 smithi183 ceph-mon[35818]: pgmap v452: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:54.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:54 smithi052 ceph-mon[35071]: pgmap v452: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:54.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:54 smithi043 ceph-mon[31929]: pgmap v452: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:55.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.836+0000 7f8c5a200700 1 -- 172.21.15.43:0/1476286264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c54102cf0 msgr2=0x7f8c541050e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:55.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.836+0000 7f8c5a200700 1 --2- 172.21.15.43:0/1476286264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c54102cf0 0x7f8c541050e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009990 tx=0x7f8c44009ca0 comp rx=0 tx=0).stop 2024-03-26T08:33:55.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.836+0000 7f8c5a200700 1 -- 172.21.15.43:0/1476286264 shutdown_connections 2024-03-26T08:33:55.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.836+0000 7f8c5a200700 1 --2- 172.21.15.43:0/1476286264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c54107f50 0x7f8c5410a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:55.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.836+0000 7f8c5a200700 1 --2- 172.21.15.43:0/1476286264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c54102cf0 0x7f8c541050e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:55.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.836+0000 7f8c5a200700 1 --2- 172.21.15.43:0/1476286264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c54105620 0x7f8c54107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:55.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.836+0000 7f8c5a200700 1 -- 172.21.15.43:0/1476286264 >> 172.21.15.43:0/1476286264 conn(0x7f8c540fc660 msgr2=0x7f8c540feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:55.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.836+0000 7f8c5a200700 1 -- 172.21.15.43:0/1476286264 shutdown_connections 2024-03-26T08:33:55.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.837+0000 7f8c5a200700 1 -- 172.21.15.43:0/1476286264 wait complete. 2024-03-26T08:33:55.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.838+0000 7f8c5a200700 1 Processor -- start 2024-03-26T08:33:55.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.838+0000 7f8c5a200700 1 -- start start 2024-03-26T08:33:55.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.838+0000 7f8c5a200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c54102cf0 0x7f8c5419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:55.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.838+0000 7f8c5a200700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c54105620 0x7f8c5419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:55.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.838+0000 7f8c5a200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c54107f50 0x7f8c541a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:55.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.838+0000 7f8c5a200700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5410dca0 con 0x7f8c54107f50 2024-03-26T08:33:55.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.838+0000 7f8c5a200700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5410db20 con 0x7f8c54105620 2024-03-26T08:33:55.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.838+0000 7f8c5a200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5410de20 con 0x7f8c54102cf0 2024-03-26T08:33:55.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.839+0000 7f8c53e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c54107f50 0x7f8c541a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:55.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.839+0000 7f8c53e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c54107f50 0x7f8c541a1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55558/0 (socket says 172.21.15.43:55558) 2024-03-26T08:33:55.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.839+0000 7f8c53e00700 1 -- 172.21.15.43:0/1300878294 learned_addr learned my addr 172.21.15.43:0/1300878294 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:55.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.839+0000 7f8c4aa00700 1 --2- 172.21.15.43:0/1300878294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c54105620 0x7f8c5419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:55.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.839+0000 7f8c53400700 1 --2- 172.21.15.43:0/1300878294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c54102cf0 0x7f8c5419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:55.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.839+0000 7f8c53e00700 1 -- 172.21.15.43:0/1300878294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c54102cf0 msgr2=0x7f8c5419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:55.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.839+0000 7f8c53e00700 1 --2- 172.21.15.43:0/1300878294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c54102cf0 0x7f8c5419a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:55.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.839+0000 7f8c53e00700 1 -- 172.21.15.43:0/1300878294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c54105620 msgr2=0x7f8c5419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:55.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.839+0000 7f8c53e00700 1 --2- 172.21.15.43:0/1300878294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c54105620 0x7f8c5419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:55.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.839+0000 7f8c53e00700 1 -- 172.21.15.43:0/1300878294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c541a2180 con 0x7f8c54107f50 2024-03-26T08:33:55.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.840+0000 7f8c53e00700 1 --2- 172.21.15.43:0/1300878294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c54107f50 0x7f8c541a1ae0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c007ee0 tx=0x7f8c4c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:55.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.840+0000 7f8c50c00700 1 -- 172.21.15.43:0/1300878294 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c004db0 con 0x7f8c54107f50 2024-03-26T08:33:55.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.840+0000 7f8c5a200700 1 -- 172.21.15.43:0/1300878294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c541a2470 con 0x7f8c54107f50 2024-03-26T08:33:55.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.840+0000 7f8c50c00700 1 -- 172.21.15.43:0/1300878294 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8c4c00dbd0 con 0x7f8c54107f50 2024-03-26T08:33:55.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.840+0000 7f8c50c00700 1 -- 172.21.15.43:0/1300878294 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c015400 con 0x7f8c54107f50 2024-03-26T08:33:55.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.840+0000 7f8c5a200700 1 -- 172.21.15.43:0/1300878294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c541a29b0 con 0x7f8c54107f50 2024-03-26T08:33:55.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.841+0000 7f8c5a200700 1 -- 172.21.15.43:0/1300878294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c54068e50 con 0x7f8c54107f50 2024-03-26T08:33:55.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.843+0000 7f8c50c00700 1 -- 172.21.15.43:0/1300878294 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f8c4c005000 con 0x7f8c54107f50 2024-03-26T08:33:55.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.845+0000 7f8c50c00700 1 --2- 172.21.15.43:0/1300878294 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8c340a2890 0x7f8c340a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:55.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.845+0000 7f8c50c00700 1 -- 172.21.15.43:0/1300878294 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f8c4c0cf180 con 0x7f8c54107f50 2024-03-26T08:33:55.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.845+0000 7f8c53400700 1 --2- 172.21.15.43:0/1300878294 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8c340a2890 0x7f8c340a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:55.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.846+0000 7f8c53400700 1 --2- 172.21.15.43:0/1300878294 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8c340a2890 0x7f8c340a4d50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8c44006010 tx=0x7f8c44003870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:55.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:55.848+0000 7f8c50c00700 1 -- 172.21.15.43:0/1300878294 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8c4c010070 con 0x7f8c54107f50 2024-03-26T08:33:56.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:56.127+0000 7f8c5a200700 1 -- 172.21.15.43:0/1300878294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8c5402da30 con 0x7f8c54107f50 2024-03-26T08:33:56.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:56.128+0000 7f8c50c00700 1 -- 172.21.15.43:0/1300878294 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+2952 (secure 0 0 0) 0x7f8c4c09be40 con 0x7f8c54107f50 2024-03-26T08:33:56.121 INFO:teuthology.orchestra.run.smithi043.stdout:e21 2024-03-26T08:33:56.122 INFO:teuthology.orchestra.run.smithi043.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-26T08:33:56.122 INFO:teuthology.orchestra.run.smithi043.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-26T08:33:56.122 INFO:teuthology.orchestra.run.smithi043.stdout:legacy client fscid: 1 2024-03-26T08:33:56.122 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:33:56.122 INFO:teuthology.orchestra.run.smithi043.stdout:Filesystem 'cephfs' (1) 2024-03-26T08:33:56.122 INFO:teuthology.orchestra.run.smithi043.stdout:fs_name cephfs 2024-03-26T08:33:56.122 INFO:teuthology.orchestra.run.smithi043.stdout:epoch 21 2024-03-26T08:33:56.122 INFO:teuthology.orchestra.run.smithi043.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-26T08:33:56.122 INFO:teuthology.orchestra.run.smithi043.stdout:created 2024-03-26T08:31:21.819689+0000 2024-03-26T08:33:56.122 INFO:teuthology.orchestra.run.smithi043.stdout:modified 2024-03-26T08:31:48.080233+0000 2024-03-26T08:33:56.122 INFO:teuthology.orchestra.run.smithi043.stdout:tableserver 0 2024-03-26T08:33:56.122 INFO:teuthology.orchestra.run.smithi043.stdout:root 0 2024-03-26T08:33:56.123 INFO:teuthology.orchestra.run.smithi043.stdout:session_timeout 300 2024-03-26T08:33:56.123 INFO:teuthology.orchestra.run.smithi043.stdout:session_autoclose 300 2024-03-26T08:33:56.123 INFO:teuthology.orchestra.run.smithi043.stdout:max_file_size 1099511627776 2024-03-26T08:33:56.123 INFO:teuthology.orchestra.run.smithi043.stdout:max_xattr_size 65536 2024-03-26T08:33:56.123 INFO:teuthology.orchestra.run.smithi043.stdout:required_client_features {} 2024-03-26T08:33:56.123 INFO:teuthology.orchestra.run.smithi043.stdout:last_failure 0 2024-03-26T08:33:56.123 INFO:teuthology.orchestra.run.smithi043.stdout:last_failure_osd_epoch 0 2024-03-26T08:33:56.123 INFO:teuthology.orchestra.run.smithi043.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-26T08:33:56.123 INFO:teuthology.orchestra.run.smithi043.stdout:max_mds 1 2024-03-26T08:33:56.123 INFO:teuthology.orchestra.run.smithi043.stdout:in 0 2024-03-26T08:33:56.123 INFO:teuthology.orchestra.run.smithi043.stdout:up {0=34559} 2024-03-26T08:33:56.123 INFO:teuthology.orchestra.run.smithi043.stdout:failed 2024-03-26T08:33:56.123 INFO:teuthology.orchestra.run.smithi043.stdout:damaged 2024-03-26T08:33:56.123 INFO:teuthology.orchestra.run.smithi043.stdout:stopped 2024-03-26T08:33:56.123 INFO:teuthology.orchestra.run.smithi043.stdout:data_pools [3,4] 2024-03-26T08:33:56.124 INFO:teuthology.orchestra.run.smithi043.stdout:metadata_pool 2 2024-03-26T08:33:56.124 INFO:teuthology.orchestra.run.smithi043.stdout:inline_data disabled 2024-03-26T08:33:56.124 INFO:teuthology.orchestra.run.smithi043.stdout:balancer 2024-03-26T08:33:56.124 INFO:teuthology.orchestra.run.smithi043.stdout:bal_rank_mask -1 2024-03-26T08:33:56.124 INFO:teuthology.orchestra.run.smithi043.stdout:standby_count_wanted 0 2024-03-26T08:33:56.124 INFO:teuthology.orchestra.run.smithi043.stdout:qdb_cluster leader: 34559 members: 34559 2024-03-26T08:33:56.124 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.b{0:34559} state up:active seq 4 addr [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T08:33:56.124 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.h{0:34517} state up:standby-replay seq 1 addr [v2:172.21.15.52:6836/748239918,v1:172.21.15.52:6837/748239918] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T08:33:56.124 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:33:56.124 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:33:56.124 INFO:teuthology.orchestra.run.smithi043.stdout:Standby daemons: 2024-03-26T08:33:56.124 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:33:56.124 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.j{-1:14553} state up:standby seq 1 addr [v2:172.21.15.43:6834/1452206807,v1:172.21.15.43:6835/1452206807] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T08:33:56.124 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.d{-1:14565} state up:standby seq 1 addr [v2:172.21.15.43:6836/3151775467,v1:172.21.15.43:6837/3151775467] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T08:33:56.124 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.g{-1:14646} state up:standby seq 1 addr [v2:172.21.15.43:6838/1284655682,v1:172.21.15.43:6839/1284655682] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T08:33:56.125 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.a{-1:14664} state up:standby seq 1 addr [v2:172.21.15.43:6840/3081085787,v1:172.21.15.43:6841/3081085787] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T08:33:56.125 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.l{-1:24409} state up:standby seq 1 addr [v2:172.21.15.183:6832/3522741573,v1:172.21.15.183:6833/3522741573] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T08:33:56.125 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.f{-1:24469} state up:standby seq 1 addr [v2:172.21.15.183:6834/1842295306,v1:172.21.15.183:6835/1842295306] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T08:33:56.125 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.c{-1:24484} state up:standby seq 1 addr [v2:172.21.15.183:6836/2267781666,v1:172.21.15.183:6837/2267781666] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T08:33:56.125 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.i{-1:24511} state up:standby seq 1 addr [v2:172.21.15.183:6838/3572429948,v1:172.21.15.183:6839/3572429948] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T08:33:56.125 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.k{-1:34448} state up:standby seq 1 addr [v2:172.21.15.52:6832/1108327179,v1:172.21.15.52:6833/1108327179] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T08:33:56.125 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.e{-1:34487} state up:standby seq 1 addr [v2:172.21.15.52:6834/742057410,v1:172.21.15.52:6835/742057410] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T08:33:56.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:56.131+0000 7f8c5a200700 1 -- 172.21.15.43:0/1300878294 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8c340a2890 msgr2=0x7f8c340a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:56.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:56.131+0000 7f8c5a200700 1 --2- 172.21.15.43:0/1300878294 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8c340a2890 0x7f8c340a4d50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8c44006010 tx=0x7f8c44003870 comp rx=0 tx=0).stop 2024-03-26T08:33:56.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:56.131+0000 7f8c5a200700 1 -- 172.21.15.43:0/1300878294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c54107f50 msgr2=0x7f8c541a1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:56.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:56.131+0000 7f8c5a200700 1 --2- 172.21.15.43:0/1300878294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c54107f50 0x7f8c541a1ae0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c007ee0 tx=0x7f8c4c00d3b0 comp rx=0 tx=0).stop 2024-03-26T08:33:56.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:56.131+0000 7f8c5a200700 1 -- 172.21.15.43:0/1300878294 shutdown_connections 2024-03-26T08:33:56.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:56.131+0000 7f8c5a200700 1 --2- 172.21.15.43:0/1300878294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c54107f50 0x7f8c541a1ae0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:56.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:56.131+0000 7f8c5a200700 1 --2- 172.21.15.43:0/1300878294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c54102cf0 0x7f8c5419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:56.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:56.131+0000 7f8c5a200700 1 --2- 172.21.15.43:0/1300878294 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f8c340a2890 0x7f8c340a4d50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:56.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:56.131+0000 7f8c5a200700 1 --2- 172.21.15.43:0/1300878294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c54105620 0x7f8c5419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:56.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:56.131+0000 7f8c5a200700 1 -- 172.21.15.43:0/1300878294 >> 172.21.15.43:0/1300878294 conn(0x7f8c540fc660 msgr2=0x7f8c54102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:56.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:56.131+0000 7f8c5a200700 1 -- 172.21.15.43:0/1300878294 shutdown_connections 2024-03-26T08:33:56.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:56.131+0000 7f8c5a200700 1 -- 172.21.15.43:0/1300878294 wait complete. 2024-03-26T08:33:56.127 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 21 2024-03-26T08:33:56.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:56 smithi183 ceph-mon[35818]: pgmap v453: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:56.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:56 smithi052 ceph-mon[35071]: pgmap v453: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:56.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:56 smithi043 ceph-mon[31929]: pgmap v453: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:33:57.073 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-26T08:33:57.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:57 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1300878294' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-26T08:33:57.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:57 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1300878294' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-26T08:33:57.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:57 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1300878294' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-26T08:33:57.528 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:33:58.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:33:58 smithi183 ceph-mon[35818]: pgmap v454: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:33:58.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:33:58 smithi052 ceph-mon[35071]: pgmap v454: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:33:58.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:33:58 smithi043 ceph-mon[31929]: pgmap v454: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:33:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.912+0000 7fe4e6a00700 1 -- 172.21.15.43:0/3979204376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e0102c50 msgr2=0x7fe4e0105040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.912+0000 7fe4e6a00700 1 --2- 172.21.15.43:0/3979204376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e0102c50 0x7fe4e0105040 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009990 tx=0x7fe4d0009ca0 comp rx=0 tx=0).stop 2024-03-26T08:33:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.913+0000 7fe4e6a00700 1 -- 172.21.15.43:0/3979204376 shutdown_connections 2024-03-26T08:33:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.913+0000 7fe4e6a00700 1 --2- 172.21.15.43:0/3979204376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4e0107eb0 0x7fe4e010a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.913+0000 7fe4e6a00700 1 --2- 172.21.15.43:0/3979204376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e0102c50 0x7fe4e0105040 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.913+0000 7fe4e6a00700 1 --2- 172.21.15.43:0/3979204376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4e0105580 0x7fe4e0107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.913+0000 7fe4e6a00700 1 -- 172.21.15.43:0/3979204376 >> 172.21.15.43:0/3979204376 conn(0x7fe4e00fc660 msgr2=0x7fe4e00feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.913+0000 7fe4e6a00700 1 -- 172.21.15.43:0/3979204376 shutdown_connections 2024-03-26T08:33:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.913+0000 7fe4e6a00700 1 -- 172.21.15.43:0/3979204376 wait complete. 2024-03-26T08:33:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.914+0000 7fe4e6a00700 1 Processor -- start 2024-03-26T08:33:59.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.914+0000 7fe4e6a00700 1 -- start start 2024-03-26T08:33:59.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.914+0000 7fe4e6a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e0102c50 0x7fe4e019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:59.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.914+0000 7fe4e6a00700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4e0105580 0x7fe4e019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:59.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.915+0000 7fe4e6a00700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4e0107eb0 0x7fe4e01a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:59.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.915+0000 7fe4e6a00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e010dca0 con 0x7fe4e0102c50 2024-03-26T08:33:59.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.915+0000 7fe4e6a00700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e010db20 con 0x7fe4e0105580 2024-03-26T08:33:59.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.915+0000 7fe4e6a00700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e010de20 con 0x7fe4e0107eb0 2024-03-26T08:33:59.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.915+0000 7fe4dfe00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e0102c50 0x7fe4e019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.915+0000 7fe4dfe00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e0102c50 0x7fe4e019a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55578/0 (socket says 172.21.15.43:55578) 2024-03-26T08:33:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.915+0000 7fe4dfe00700 1 -- 172.21.15.43:0/792133126 learned_addr learned my addr 172.21.15.43:0/792133126 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:33:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.915+0000 7fe4df400700 1 --2- 172.21.15.43:0/792133126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4e0105580 0x7fe4e019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.915+0000 7fe4e4a00700 1 --2- 172.21.15.43:0/792133126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4e0107eb0 0x7fe4e01a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.915+0000 7fe4dfe00700 1 -- 172.21.15.43:0/792133126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4e0107eb0 msgr2=0x7fe4e01a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.915+0000 7fe4dfe00700 1 --2- 172.21.15.43:0/792133126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4e0107eb0 0x7fe4e01a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.915+0000 7fe4dfe00700 1 -- 172.21.15.43:0/792133126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4e0105580 msgr2=0x7fe4e019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.915+0000 7fe4dfe00700 1 --2- 172.21.15.43:0/792133126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4e0105580 0x7fe4e019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.915+0000 7fe4dfe00700 1 -- 172.21.15.43:0/792133126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e01a21a0 con 0x7fe4e0102c50 2024-03-26T08:33:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.915+0000 7fe4e4a00700 1 --2- 172.21.15.43:0/792133126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4e0107eb0 0x7fe4e01a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:33:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.916+0000 7fe4dfe00700 1 --2- 172.21.15.43:0/792133126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e0102c50 0x7fe4e019a540 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0000c00 tx=0x7fe4d0009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.916+0000 7fe4dcc00700 1 -- 172.21.15.43:0/792133126 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0027070 con 0x7fe4e0102c50 2024-03-26T08:33:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.916+0000 7fe4dcc00700 1 -- 172.21.15.43:0/792133126 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe4d0004190 con 0x7fe4e0102c50 2024-03-26T08:33:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.916+0000 7fe4e6a00700 1 -- 172.21.15.43:0/792133126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e01a2430 con 0x7fe4e0102c50 2024-03-26T08:33:59.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.916+0000 7fe4dcc00700 1 -- 172.21.15.43:0/792133126 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d000f460 con 0x7fe4e0102c50 2024-03-26T08:33:59.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.916+0000 7fe4e6a00700 1 -- 172.21.15.43:0/792133126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e01a28e0 con 0x7fe4e0102c50 2024-03-26T08:33:59.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.917+0000 7fe4e6a00700 1 -- 172.21.15.43:0/792133126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4e0109a30 con 0x7fe4e0102c50 2024-03-26T08:33:59.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.919+0000 7fe4dcc00700 1 -- 172.21.15.43:0/792133126 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fe4d002a070 con 0x7fe4e0102c50 2024-03-26T08:33:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.921+0000 7fe4dcc00700 1 --2- 172.21.15.43:0/792133126 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fe4c40a2910 0x7fe4c40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.921+0000 7fe4dcc00700 1 -- 172.21.15.43:0/792133126 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7fe4d00d0060 con 0x7fe4e0102c50 2024-03-26T08:33:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.921+0000 7fe4df400700 1 --2- 172.21.15.43:0/792133126 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fe4c40a2910 0x7fe4c40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:59.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.922+0000 7fe4df400700 1 --2- 172.21.15.43:0/792133126 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fe4c40a2910 0x7fe4c40a4dd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc005fd0 tx=0x7fe4cc00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:59.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:33:59.925+0000 7fe4dcc00700 1 -- 172.21.15.43:0/792133126 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe4d0013070 con 0x7fe4e0102c50 2024-03-26T08:34:00.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:00.156+0000 7fe4e6a00700 1 -- 172.21.15.43:0/792133126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fe4e0068e50 con 0x7fe4e0102c50 2024-03-26T08:34:00.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:00.159+0000 7fe4dcc00700 1 -- 172.21.15.43:0/792133126 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v94) v1 ==== 56+0+5614 (secure 0 0 0) 0x7fe4d00bc020 con 0x7fe4e0102c50 2024-03-26T08:34:00.152 INFO:teuthology.orchestra.run.smithi043.stdout:epoch 94 2024-03-26T08:34:00.152 INFO:teuthology.orchestra.run.smithi043.stdout:fsid 88759420-eb49-11ee-95d0-87774f69a715 2024-03-26T08:34:00.152 INFO:teuthology.orchestra.run.smithi043.stdout:created 2024-03-26T08:20:27.817107+0000 2024-03-26T08:34:00.153 INFO:teuthology.orchestra.run.smithi043.stdout:modified 2024-03-26T08:32:38.256877+0000 2024-03-26T08:34:00.153 INFO:teuthology.orchestra.run.smithi043.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-26T08:34:00.153 INFO:teuthology.orchestra.run.smithi043.stdout:crush_version 27 2024-03-26T08:34:00.153 INFO:teuthology.orchestra.run.smithi043.stdout:full_ratio 0.95 2024-03-26T08:34:00.153 INFO:teuthology.orchestra.run.smithi043.stdout:backfillfull_ratio 0.9 2024-03-26T08:34:00.153 INFO:teuthology.orchestra.run.smithi043.stdout:nearfull_ratio 0.85 2024-03-26T08:34:00.153 INFO:teuthology.orchestra.run.smithi043.stdout:require_min_compat_client luminous 2024-03-26T08:34:00.153 INFO:teuthology.orchestra.run.smithi043.stdout:min_compat_client luminous 2024-03-26T08:34:00.153 INFO:teuthology.orchestra.run.smithi043.stdout:require_osd_release squid 2024-03-26T08:34:00.153 INFO:teuthology.orchestra.run.smithi043.stdout:stretch_mode_enabled false 2024-03-26T08:34:00.153 INFO:teuthology.orchestra.run.smithi043.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-26T08:34:00.153 INFO:teuthology.orchestra.run.smithi043.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-26T08:34:00.153 INFO:teuthology.orchestra.run.smithi043.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-03-26T08:34:00.154 INFO:teuthology.orchestra.run.smithi043.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-03-26T08:34:00.154 INFO:teuthology.orchestra.run.smithi043.stdout:max_osd 12 2024-03-26T08:34:00.154 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0 up in weight 1 up_from 8 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6802/936601146,v1:172.21.15.43:6803/936601146] [v2:172.21.15.43:6804/936601146,v1:172.21.15.43:6805/936601146] exists,up 28670673-9193-4fa3-9cc0-c1a5a6b27284 2024-03-26T08:34:00.154 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1 up in weight 1 up_from 13 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.52:6800/3268412328,v1:172.21.15.52:6801/3268412328] [v2:172.21.15.52:6802/3268412328,v1:172.21.15.52:6803/3268412328] exists,up fa8ad6b6-277a-42b8-b0f8-b36fc219de2f 2024-03-26T08:34:00.154 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2 up in weight 1 up_from 18 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6800/2103180908,v1:172.21.15.183:6801/2103180908] [v2:172.21.15.183:6802/2103180908,v1:172.21.15.183:6803/2103180908] exists,up ec0039c3-f9bd-419a-81de-fa4418359808 2024-03-26T08:34:00.154 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3 up in weight 1 up_from 26 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6810/3722846033,v1:172.21.15.43:6811/3722846033] [v2:172.21.15.43:6812/3722846033,v1:172.21.15.43:6813/3722846033] exists,up e7dcbf00-6586-4664-a69e-3cb0df8dfd30 2024-03-26T08:34:00.154 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4 up in weight 1 up_from 34 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.52:6808/4028436523,v1:172.21.15.52:6809/4028436523] [v2:172.21.15.52:6810/4028436523,v1:172.21.15.52:6811/4028436523] exists,up db089a49-5e98-432d-9687-cdc044114c63 2024-03-26T08:34:00.154 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5 up in weight 1 up_from 39 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6808/787132252,v1:172.21.15.183:6809/787132252] [v2:172.21.15.183:6810/787132252,v1:172.21.15.183:6811/787132252] exists,up 556ac804-02ea-423b-a599-55baf5be4ca1 2024-03-26T08:34:00.154 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6 up in weight 1 up_from 47 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6818/4206861158,v1:172.21.15.43:6819/4206861158] [v2:172.21.15.43:6820/4206861158,v1:172.21.15.43:6821/4206861158] exists,up 3de2a49d-6cef-4580-9d8a-f90cdfcecd9a 2024-03-26T08:34:00.154 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7 up in weight 1 up_from 52 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.52:6816/3452064146,v1:172.21.15.52:6817/3452064146] [v2:172.21.15.52:6818/3452064146,v1:172.21.15.52:6819/3452064146] exists,up a3b832f7-7d7d-4a62-b679-db6da98275d9 2024-03-26T08:34:00.154 INFO:teuthology.orchestra.run.smithi043.stdout:osd.8 up in weight 1 up_from 60 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6816/1023131725,v1:172.21.15.183:6817/1023131725] [v2:172.21.15.183:6818/1023131725,v1:172.21.15.183:6819/1023131725] exists,up ddc6b07f-d703-414d-a59a-02316376b13d 2024-03-26T08:34:00.155 INFO:teuthology.orchestra.run.smithi043.stdout:osd.9 up in weight 1 up_from 68 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.43:6826/3935195657,v1:172.21.15.43:6827/3935195657] [v2:172.21.15.43:6828/3935195657,v1:172.21.15.43:6829/3935195657] exists,up ac16bb0e-6232-45c8-b16a-4d878874c588 2024-03-26T08:34:00.155 INFO:teuthology.orchestra.run.smithi043.stdout:osd.10 up in weight 1 up_from 73 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.52:6824/377628320,v1:172.21.15.52:6825/377628320] [v2:172.21.15.52:6826/377628320,v1:172.21.15.52:6827/377628320] exists,up fd839188-f398-4cff-aa7b-36522ff1a8a9 2024-03-26T08:34:00.155 INFO:teuthology.orchestra.run.smithi043.stdout:osd.11 up in weight 1 up_from 81 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6824/1373086433,v1:172.21.15.183:6825/1373086433] [v2:172.21.15.183:6826/1373086433,v1:172.21.15.183:6827/1373086433] exists,up a40282a0-0c64-49ee-b64e-6f3f8119e827 2024-03-26T08:34:00.155 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 2.8 [2,11] 2024-03-26T08:34:00.155 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 2.f [8,11] 2024-03-26T08:34:00.155 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.e [4,8] 2024-03-26T08:34:00.155 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.14 [4,6] 2024-03-26T08:34:00.155 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.36 [4,9] 2024-03-26T08:34:00.155 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.37 [1,9] 2024-03-26T08:34:00.155 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 3.38 [3,9,5,7] 2024-03-26T08:34:00.155 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 4.5 [9,10] 2024-03-26T08:34:00.155 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 4.b [9,8] 2024-03-26T08:34:00.155 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 4.e [5,8] 2024-03-26T08:34:00.155 INFO:teuthology.orchestra.run.smithi043.stdout:pg_upmap_items 4.17 [9,8] 2024-03-26T08:34:00.156 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:6801/2446912488 expires 2024-03-27T08:21:36.149092+0000 2024-03-26T08:34:00.156 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:6800/2446912488 expires 2024-03-27T08:21:36.149092+0000 2024-03-26T08:34:00.156 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/4240775295 expires 2024-03-27T08:21:36.149092+0000 2024-03-26T08:34:00.156 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/510203140 expires 2024-03-27T08:21:01.256355+0000 2024-03-26T08:34:00.156 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/4251957687 expires 2024-03-27T08:21:36.149092+0000 2024-03-26T08:34:00.156 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:6800/4108777098 expires 2024-03-27T08:21:01.256355+0000 2024-03-26T08:34:00.156 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:6801/4108777098 expires 2024-03-27T08:21:01.256355+0000 2024-03-26T08:34:00.156 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/1885244665 expires 2024-03-27T08:21:01.256355+0000 2024-03-26T08:34:00.156 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/298320867 expires 2024-03-27T08:21:01.256355+0000 2024-03-26T08:34:00.156 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/1301092625 expires 2024-03-27T08:21:36.149092+0000 2024-03-26T08:34:00.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:00.162+0000 7fe4e6a00700 1 -- 172.21.15.43:0/792133126 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fe4c40a2910 msgr2=0x7fe4c40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:00.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:00.162+0000 7fe4e6a00700 1 --2- 172.21.15.43:0/792133126 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fe4c40a2910 0x7fe4c40a4dd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc005fd0 tx=0x7fe4cc00a560 comp rx=0 tx=0).stop 2024-03-26T08:34:00.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:00.162+0000 7fe4e6a00700 1 -- 172.21.15.43:0/792133126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e0102c50 msgr2=0x7fe4e019a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:00.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:00.162+0000 7fe4e6a00700 1 --2- 172.21.15.43:0/792133126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e0102c50 0x7fe4e019a540 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0000c00 tx=0x7fe4d0009f70 comp rx=0 tx=0).stop 2024-03-26T08:34:00.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:00.162+0000 7fe4e6a00700 1 -- 172.21.15.43:0/792133126 shutdown_connections 2024-03-26T08:34:00.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:00.162+0000 7fe4e6a00700 1 --2- 172.21.15.43:0/792133126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4e0107eb0 0x7fe4e01a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:00.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:00.162+0000 7fe4e6a00700 1 --2- 172.21.15.43:0/792133126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe4e0102c50 0x7fe4e019a540 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:00.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:00.162+0000 7fe4e6a00700 1 --2- 172.21.15.43:0/792133126 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fe4c40a2910 0x7fe4c40a4dd0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:00.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:00.162+0000 7fe4e6a00700 1 --2- 172.21.15.43:0/792133126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4e0105580 0x7fe4e019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:00.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:00.162+0000 7fe4e6a00700 1 -- 172.21.15.43:0/792133126 >> 172.21.15.43:0/792133126 conn(0x7fe4e00fc660 msgr2=0x7fe4e01023b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:00.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:00.162+0000 7fe4e6a00700 1 -- 172.21.15.43:0/792133126 shutdown_connections 2024-03-26T08:34:00.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:00.162+0000 7fe4e6a00700 1 -- 172.21.15.43:0/792133126 wait complete. 2024-03-26T08:34:00.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:00 smithi052 ceph-mon[35071]: pgmap v455: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:00.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:00 smithi043 ceph-mon[31929]: pgmap v455: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:00.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:00 smithi183 ceph-mon[35818]: pgmap v455: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:01.085 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-26T08:34:01.096 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:34:01.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:01 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/792133126' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-26T08:34:01.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.335+0000 7f6726c00640 1 -- 172.21.15.43:0/2652531523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6720074080 msgr2=0x7f67200744e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:01.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.335+0000 7f6726c00640 1 --2- 172.21.15.43:0/2652531523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6720074080 0x7f67200744e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f670c009900 tx=0x7f670c02bcc0 comp rx=0 tx=0).stop 2024-03-26T08:34:01.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.335+0000 7f6726c00640 1 -- 172.21.15.43:0/2652531523 shutdown_connections 2024-03-26T08:34:01.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.335+0000 7f6726c00640 1 --2- 172.21.15.43:0/2652531523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67201093b0 0x7f672010b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:01.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.335+0000 7f6726c00640 1 --2- 172.21.15.43:0/2652531523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6720074080 0x7f67200744e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:01.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.335+0000 7f6726c00640 1 --2- 172.21.15.43:0/2652531523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6720075660 0x7f6720073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:01.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.335+0000 7f6726c00640 1 -- 172.21.15.43:0/2652531523 >> 172.21.15.43:0/2652531523 conn(0x7f67200fca70 msgr2=0x7f67200feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:01.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.335+0000 7f6726c00640 1 -- 172.21.15.43:0/2652531523 shutdown_connections 2024-03-26T08:34:01.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.335+0000 7f6726c00640 1 -- 172.21.15.43:0/2652531523 wait complete. 2024-03-26T08:34:01.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.336+0000 7f6726c00640 1 Processor -- start 2024-03-26T08:34:01.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.336+0000 7f6726c00640 1 -- start start 2024-03-26T08:34:01.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.337+0000 7f6726c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6720074080 0x7f67201998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:01.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.337+0000 7f6726c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6720075660 0x7f6720199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:01.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.337+0000 7f6726c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67201093b0 0x7f67201a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:01.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.337+0000 7f6726c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672010f260 con 0x7f6720074080 2024-03-26T08:34:01.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.337+0000 7f6726c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672010f0e0 con 0x7f6720075660 2024-03-26T08:34:01.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.337+0000 7f6726c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672010f3e0 con 0x7f67201093b0 2024-03-26T08:34:01.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.337+0000 7f6725800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6720074080 0x7f67201998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:01.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.337+0000 7f6725800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6720074080 0x7f67201998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55598/0 (socket says 172.21.15.43:55598) 2024-03-26T08:34:01.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.337+0000 7f6725800640 1 -- 172.21.15.43:0/3577482801 learned_addr learned my addr 172.21.15.43:0/3577482801 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:34:01.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.337+0000 7f6724e00640 1 --2- 172.21.15.43:0/3577482801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6720075660 0x7f6720199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:01.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.337+0000 7f6725800640 1 -- 172.21.15.43:0/3577482801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67201093b0 msgr2=0x7f67201a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:01.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.337+0000 7f6725800640 1 --2- 172.21.15.43:0/3577482801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67201093b0 0x7f67201a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:01.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.337+0000 7f6725800640 1 -- 172.21.15.43:0/3577482801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6720075660 msgr2=0x7f6720199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:01.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.337+0000 7f6725800640 1 --2- 172.21.15.43:0/3577482801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6720075660 0x7f6720199e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:01.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.337+0000 7f6725800640 1 -- 172.21.15.43:0/3577482801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67201a14b0 con 0x7f6720074080 2024-03-26T08:34:01.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.338+0000 7f6725800640 1 --2- 172.21.15.43:0/3577482801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6720074080 0x7f67201998e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f671400d950 tx=0x7f671400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:01.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.338+0000 7f671e000640 1 -- 172.21.15.43:0/3577482801 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6714017070 con 0x7f6720074080 2024-03-26T08:34:01.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.338+0000 7f671e000640 1 -- 172.21.15.43:0/3577482801 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6714009c30 con 0x7f6720074080 2024-03-26T08:34:01.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.338+0000 7f671e000640 1 -- 172.21.15.43:0/3577482801 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671400f620 con 0x7f6720074080 2024-03-26T08:34:01.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.338+0000 7f6726c00640 1 -- 172.21.15.43:0/3577482801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67201a17a0 con 0x7f6720074080 2024-03-26T08:34:01.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.338+0000 7f6726c00640 1 -- 172.21.15.43:0/3577482801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67201a1ca0 con 0x7f6720074080 2024-03-26T08:34:01.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.339+0000 7f6726c00640 1 -- 172.21.15.43:0/3577482801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6720108170 con 0x7f6720074080 2024-03-26T08:34:01.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.342+0000 7f671e000640 1 -- 172.21.15.43:0/3577482801 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f67140109a0 con 0x7f6720074080 2024-03-26T08:34:01.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.344+0000 7f671e000640 1 --2- 172.21.15.43:0/3577482801 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f67080a2800 0x7f67080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:01.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.345+0000 7f671e000640 1 -- 172.21.15.43:0/3577482801 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f67140ceb10 con 0x7f6720074080 2024-03-26T08:34:01.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.345+0000 7f6724e00640 1 --2- 172.21.15.43:0/3577482801 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f67080a2800 0x7f67080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:01.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.345+0000 7f6724e00640 1 --2- 172.21.15.43:0/3577482801 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f67080a2800 0x7f67080a4cc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f670c0096c0 tx=0x7f670c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:01.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.347+0000 7f671e000640 1 -- 172.21.15.43:0/3577482801 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6714009de0 con 0x7f6720074080 2024-03-26T08:34:01.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:01 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/792133126' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-26T08:34:01.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.555+0000 7f6726c00640 1 -- 172.21.15.43:0/3577482801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67200623a0 con 0x7f6720074080 2024-03-26T08:34:01.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:01 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/792133126' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-26T08:34:01.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.560+0000 7f671e000640 1 -- 172.21.15.43:0/3577482801 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24670 (secure 0 0 0) 0x7f671409b7d0 con 0x7f6720074080 2024-03-26T08:34:01.553 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:34:01.553 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":94,"fsid":"88759420-eb49-11ee-95d0-87774f69a715","created":"2024-03-26T08:20:27.817107+0000","modified":"2024-03-26T08:32:38.256877+0000","last_up_change":"2024-03-26T08:30:10.520976+0000","last_in_change":"2024-03-26T08:29:46.394360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:24:54.852427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:31:19.924902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:31:21.323881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T08:31:24.951015+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"28670673-9193-4fa3-9cc0-c1a5a6b27284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6803","nonce":936601146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6805","nonce":936601146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6809","nonce":936601146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6807","nonce":936601146}]},"public_addr":"172.21.15.43:6803/936601146","cluster_addr":"172.21.15.43:6805/936601146","heartbeat_back_addr":"172.21.15.43:6809/936601146","heartbeat_front_addr":"172.21.15.43:6807/936601146","state":["exists","up"]},{"osd":1,"uuid":"fa8ad6b6-277a-42b8-b0f8-b36fc219de2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6801","nonce":3268412328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6803","nonce":3268412328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6807","nonce":3268412328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6805","nonce":3268412328}]},"public_addr":"172.21.15.52:6801/3268412328","cluster_addr":"172.21.15.52:6803/3268412328","heartbeat_back_addr":"172.21.15.52:6807/3268412328","heartbeat_front_addr":"172.21.15.52:6805/3268412328","state":["exists","up"]},{"osd":2,"uuid":"ec0039c3-f9bd-419a-81de-fa4418359808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6801","nonce":2103180908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6803","nonce":2103180908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6807","nonce":2103180908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6805","nonce":2103180908}]},"public_addr":"172.21.15.183:6801/2103180908","cluster_addr":"172.21.15.183:6803/2103180908","heartbeat_back_addr":"172.21.15.183:6807/2103180908","heartbeat_front_addr":"172.21.15.183:6805/2103180908","state":["exists","up"]},{"osd":3,"uuid":"e7dcbf00-6586-4664-a69e-3cb0df8dfd30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6811","nonce":3722846033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6813","nonce":3722846033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6817","nonce":3722846033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6815","nonce":3722846033}]},"public_addr":"172.21.15.43:6811/3722846033","cluster_addr":"172.21.15.43:6813/3722846033","heartbeat_back_addr":"172.21.15.43:6817/3722846033","heartbeat_front_addr":"172.21.15.43:6815/3722846033","state":["exists","up"]},{"osd":4,"uuid":"db089a49-5e98-432d-9687-cdc044114c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6809","nonce":4028436523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6811","nonce":4028436523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6815","nonce":4028436523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6813","nonce":4028436523}]},"public_addr":"172.21.15.52:6809/4028436523","cluster_addr":"172.21.15.52:6811/4028436523","heartbeat_back_addr":"172.21.15.52:6815/4028436523","heartbeat_front_addr":"172.21.15.52:6813/4028436523","state":["exists","up"]},{"osd":5,"uuid":"556ac804-02ea-423b-a599-55baf5be4ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6809","nonce":787132252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6811","nonce":787132252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6815","nonce":787132252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6813","nonce":787132252}]},"public_addr":"172.21.15.183:6809/787132252","cluster_addr":"172.21.15.183:6811/787132252","heartbeat_back_addr":"172.21.15.183:6815/787132252","heartbeat_front_addr":"172.21.15.183:6813/787132252","state":["exists","up"]},{"osd":6,"uuid":"3de2a49d-6cef-4580-9d8a-f90cdfcecd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6819","nonce":4206861158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6821","nonce":4206861158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6825","nonce":4206861158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6823","nonce":4206861158}]},"public_addr":"172.21.15.43:6819/4206861158","cluster_addr":"172.21.15.43:6821/4206861158","heartbeat_back_addr":"172.21.15.43:6825/4206861158","heartbeat_front_addr":"172.21.15.43:6823/4206861158","state":["exists","up"]},{"osd":7,"uuid":"a3b832f7-7d7d-4a62-b679-db6da98275d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6817","nonce":3452064146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6819","nonce":3452064146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6823","nonce":3452064146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6821","nonce":3452064146}]},"public_addr":"172.21.15.52:6817/3452064146","cluster_addr":"172.21.15.52:6819/3452064146","heartbeat_back_addr":"172.21.15.52:6823/3452064146","heartbeat_front_addr":"172.21.15.52:6821/3452064146","state":["exists","up"]},{"osd":8,"uuid":"ddc6b07f-d703-414d-a59a-02316376b13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6817","nonce":1023131725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6819","nonce":1023131725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6823","nonce":1023131725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6821","nonce":1023131725}]},"public_addr":"172.21.15.183:6817/1023131725","cluster_addr":"172.21.15.183:6819/1023131725","heartbeat_back_addr":"172.21.15.183:6823/1023131725","heartbeat_front_addr":"172.21.15.183:6821/1023131725","state":["exists","up"]},{"osd":9,"uuid":"ac16bb0e-6232-45c8-b16a-4d878874c588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6827","nonce":3935195657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6829","nonce":3935195657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6833","nonce":3935195657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6831","nonce":3935195657}]},"public_addr":"172.21.15.43:6827/3935195657","cluster_addr":"172.21.15.43:6829/3935195657","heartbeat_back_addr":"172.21.15.43:6833/3935195657","heartbeat_front_addr":"172.21.15.43:6831/3935195657","state":["exists","up"]},{"osd":10,"uuid":"fd839188-f398-4cff-aa7b-36522ff1a8a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6825","nonce":377628320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6827","nonce":377628320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6831","nonce":377628320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6829","nonce":377628320}]},"public_addr":"172.21.15.52:6825/377628320","cluster_addr":"172.21.15.52:6827/377628320","heartbeat_back_addr":"172.21.15.52:6831/377628320","heartbeat_front_addr":"172.21.15.52:6829/377628320","state":["exists","up"]},{"osd":11,"uuid":"a40282a0-0c64-49ee-b64e-6f3f8119e827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6825","nonce":1373086433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6827","nonce":1373086433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6831","nonce":1373086433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6829","nonce":1373086433}]},"public_addr":"172.21.15.183:6825/1373086433","cluster_addr":"172.21.15.183:6827/1373086433","heartbeat_back_addr":"172.21.15.183:6831/1373086433","heartbeat_front_addr":"172.21.15.183:6829/1373086433","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:23:43.478905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:17.573532+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:50.286610+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:25:27.407663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:03.081294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:36.057416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:12.062013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:47.032271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:21.023201+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:57.364739+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:29:34.039995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:30:08.827577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6801/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/4240775295":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/510203140":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/4251957687":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:6801/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1885244665":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/298320867":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1301092625":"2024-03-27T08:21:36.149092+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:34:01.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.562+0000 7f6726c00640 1 -- 172.21.15.43:0/3577482801 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f67080a2800 msgr2=0x7f67080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:01.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.562+0000 7f6726c00640 1 --2- 172.21.15.43:0/3577482801 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f67080a2800 0x7f67080a4cc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f670c0096c0 tx=0x7f670c005fd0 comp rx=0 tx=0).stop 2024-03-26T08:34:01.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.562+0000 7f6726c00640 1 -- 172.21.15.43:0/3577482801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6720074080 msgr2=0x7f67201998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:01.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.562+0000 7f6726c00640 1 --2- 172.21.15.43:0/3577482801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6720074080 0x7f67201998e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f671400d950 tx=0x7f671400de10 comp rx=0 tx=0).stop 2024-03-26T08:34:01.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.562+0000 7f6726c00640 1 -- 172.21.15.43:0/3577482801 shutdown_connections 2024-03-26T08:34:01.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.562+0000 7f6726c00640 1 --2- 172.21.15.43:0/3577482801 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f67080a2800 0x7f67080a4cc0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:01.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.562+0000 7f6726c00640 1 --2- 172.21.15.43:0/3577482801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67201093b0 0x7f67201a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:01.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.562+0000 7f6726c00640 1 --2- 172.21.15.43:0/3577482801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6720075660 0x7f6720199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:01.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.562+0000 7f6726c00640 1 --2- 172.21.15.43:0/3577482801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6720074080 0x7f67201998e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:01.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.562+0000 7f6726c00640 1 -- 172.21.15.43:0/3577482801 >> 172.21.15.43:0/3577482801 conn(0x7f67200fca70 msgr2=0x7f6720105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:01.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.563+0000 7f6726c00640 1 -- 172.21.15.43:0/3577482801 shutdown_connections 2024-03-26T08:34:01.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.563+0000 7f6726c00640 1 -- 172.21.15.43:0/3577482801 wait complete. 2024-03-26T08:34:01.577 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T08:24:54.852427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T08:31:19.924902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T08:31:21.323881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T08:31:24.951015+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T08:34:01.578 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T08:34:01.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.810+0000 7fe693e00640 1 -- 172.21.15.43:0/4254109822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe694107bc0 msgr2=0x7fe694107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:01.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.810+0000 7fe693e00640 1 --2- 172.21.15.43:0/4254109822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe694107bc0 0x7fe694107fe0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe684009900 tx=0x7fe68402bc30 comp rx=0 tx=0).stop 2024-03-26T08:34:01.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.810+0000 7fe693e00640 1 -- 172.21.15.43:0/4254109822 shutdown_connections 2024-03-26T08:34:01.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.810+0000 7fe693e00640 1 --2- 172.21.15.43:0/4254109822 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe694109770 0x7fe694110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:01.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.810+0000 7fe693e00640 1 --2- 172.21.15.43:0/4254109822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe694108db0 0x7fe694109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:01.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.810+0000 7fe693e00640 1 --2- 172.21.15.43:0/4254109822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe694107bc0 0x7fe694107fe0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:01.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.810+0000 7fe693e00640 1 -- 172.21.15.43:0/4254109822 >> 172.21.15.43:0/4254109822 conn(0x7fe694076b70 msgr2=0x7fe694078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:01.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.810+0000 7fe693e00640 1 -- 172.21.15.43:0/4254109822 shutdown_connections 2024-03-26T08:34:01.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.810+0000 7fe693e00640 1 -- 172.21.15.43:0/4254109822 wait complete. 2024-03-26T08:34:01.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.811+0000 7fe693e00640 1 Processor -- start 2024-03-26T08:34:01.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.811+0000 7fe693e00640 1 -- start start 2024-03-26T08:34:01.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.812+0000 7fe693e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe694107bc0 0x7fe69419de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:01.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.812+0000 7fe693e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe694108db0 0x7fe69419e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:01.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.812+0000 7fe693e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe694109770 0x7fe6941a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:01.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.812+0000 7fe693e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe694113730 con 0x7fe694107bc0 2024-03-26T08:34:01.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.812+0000 7fe693e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6941135b0 con 0x7fe694109770 2024-03-26T08:34:01.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.812+0000 7fe693e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6941138b0 con 0x7fe694108db0 2024-03-26T08:34:01.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.812+0000 7fe692a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe694107bc0 0x7fe69419de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:01.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.812+0000 7fe692a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe694107bc0 0x7fe69419de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55612/0 (socket says 172.21.15.43:55612) 2024-03-26T08:34:01.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.812+0000 7fe692a00640 1 -- 172.21.15.43:0/4115895584 learned_addr learned my addr 172.21.15.43:0/4115895584 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:34:01.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.812+0000 7fe693400640 1 --2- 172.21.15.43:0/4115895584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe694109770 0x7fe6941a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:01.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.812+0000 7fe692000640 1 --2- 172.21.15.43:0/4115895584 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe694108db0 0x7fe69419e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:01.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.813+0000 7fe692a00640 1 -- 172.21.15.43:0/4115895584 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe694108db0 msgr2=0x7fe69419e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:01.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.813+0000 7fe692a00640 1 --2- 172.21.15.43:0/4115895584 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe694108db0 0x7fe69419e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:01.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.813+0000 7fe692a00640 1 -- 172.21.15.43:0/4115895584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe694109770 msgr2=0x7fe6941a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:01.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.813+0000 7fe692a00640 1 --2- 172.21.15.43:0/4115895584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe694109770 0x7fe6941a5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:01.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.813+0000 7fe692a00640 1 -- 172.21.15.43:0/4115895584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6941a5a80 con 0x7fe694107bc0 2024-03-26T08:34:01.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.813+0000 7fe692a00640 1 --2- 172.21.15.43:0/4115895584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe694107bc0 0x7fe69419de60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe684005ab0 tx=0x7fe6840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:01.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.813+0000 7fe68b400640 1 -- 172.21.15.43:0/4115895584 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684043020 con 0x7fe694107bc0 2024-03-26T08:34:01.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.813+0000 7fe68b400640 1 -- 172.21.15.43:0/4115895584 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe6840054c0 con 0x7fe694107bc0 2024-03-26T08:34:01.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.813+0000 7fe693e00640 1 -- 172.21.15.43:0/4115895584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6941a5d10 con 0x7fe694107bc0 2024-03-26T08:34:01.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.813+0000 7fe68b400640 1 -- 172.21.15.43:0/4115895584 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684003d10 con 0x7fe694107bc0 2024-03-26T08:34:01.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.814+0000 7fe693e00640 1 -- 172.21.15.43:0/4115895584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6941a61b0 con 0x7fe694107bc0 2024-03-26T08:34:01.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.815+0000 7fe693e00640 1 -- 172.21.15.43:0/4115895584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe69410ef50 con 0x7fe694107bc0 2024-03-26T08:34:01.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.817+0000 7fe68b400640 1 -- 172.21.15.43:0/4115895584 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fe684046020 con 0x7fe694107bc0 2024-03-26T08:34:01.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.818+0000 7fe68b400640 1 --2- 172.21.15.43:0/4115895584 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fe6780a2800 0x7fe6780a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:01.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.819+0000 7fe68b400640 1 -- 172.21.15.43:0/4115895584 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7fe6840f1650 con 0x7fe694107bc0 2024-03-26T08:34:01.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.819+0000 7fe692000640 1 --2- 172.21.15.43:0/4115895584 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fe6780a2800 0x7fe6780a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:01.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.819+0000 7fe692000640 1 --2- 172.21.15.43:0/4115895584 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fe6780a2800 0x7fe6780a4cc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe680005fd0 tx=0x7fe680005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:01.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:01.821+0000 7fe68b400640 1 -- 172.21.15.43:0/4115895584 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe68404d020 con 0x7fe694107bc0 2024-03-26T08:34:02.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.021+0000 7fe693e00640 1 -- 172.21.15.43:0/4115895584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe6940622a0 con 0x7fe694107bc0 2024-03-26T08:34:02.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.023+0000 7fe68b400640 1 -- 172.21.15.43:0/4115895584 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe6840be310 con 0x7fe694107bc0 2024-03-26T08:34:02.015 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-26T08:34:02.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.025+0000 7fe693e00640 1 -- 172.21.15.43:0/4115895584 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fe6780a2800 msgr2=0x7fe6780a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:02.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.025+0000 7fe693e00640 1 --2- 172.21.15.43:0/4115895584 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fe6780a2800 0x7fe6780a4cc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe680005fd0 tx=0x7fe680005850 comp rx=0 tx=0).stop 2024-03-26T08:34:02.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.025+0000 7fe693e00640 1 -- 172.21.15.43:0/4115895584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe694107bc0 msgr2=0x7fe69419de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:02.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.025+0000 7fe693e00640 1 --2- 172.21.15.43:0/4115895584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe694107bc0 0x7fe69419de60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe684005ab0 tx=0x7fe6840042f0 comp rx=0 tx=0).stop 2024-03-26T08:34:02.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.025+0000 7fe693e00640 1 -- 172.21.15.43:0/4115895584 shutdown_connections 2024-03-26T08:34:02.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.025+0000 7fe693e00640 1 --2- 172.21.15.43:0/4115895584 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fe6780a2800 0x7fe6780a4cc0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:02.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.025+0000 7fe693e00640 1 --2- 172.21.15.43:0/4115895584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe694109770 0x7fe6941a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:02.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.025+0000 7fe693e00640 1 --2- 172.21.15.43:0/4115895584 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe694108db0 0x7fe69419e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:02.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.025+0000 7fe693e00640 1 --2- 172.21.15.43:0/4115895584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe694107bc0 0x7fe69419de60 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:02.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.025+0000 7fe693e00640 1 -- 172.21.15.43:0/4115895584 >> 172.21.15.43:0/4115895584 conn(0x7fe694076b70 msgr2=0x7fe69410c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:02.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.026+0000 7fe693e00640 1 -- 172.21.15.43:0/4115895584 shutdown_connections 2024-03-26T08:34:02.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.026+0000 7fe693e00640 1 -- 172.21.15.43:0/4115895584 wait complete. 2024-03-26T08:34:02.039 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T08:34:02.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.274+0000 7f45c7e00640 1 -- 172.21.15.43:0/3597393821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45c8107bc0 msgr2=0x7f45c8107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:02.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.274+0000 7f45c7e00640 1 --2- 172.21.15.43:0/3597393821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45c8107bc0 0x7f45c8107fe0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009900 tx=0x7f45b802bc30 comp rx=0 tx=0).stop 2024-03-26T08:34:02.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.275+0000 7f45c7e00640 1 -- 172.21.15.43:0/3597393821 shutdown_connections 2024-03-26T08:34:02.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.275+0000 7f45c7e00640 1 --2- 172.21.15.43:0/3597393821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45c8109770 0x7f45c8110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:02.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.275+0000 7f45c7e00640 1 --2- 172.21.15.43:0/3597393821 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45c8108db0 0x7f45c8109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:02.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.275+0000 7f45c7e00640 1 --2- 172.21.15.43:0/3597393821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45c8107bc0 0x7f45c8107fe0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:02.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.275+0000 7f45c7e00640 1 -- 172.21.15.43:0/3597393821 >> 172.21.15.43:0/3597393821 conn(0x7f45c8076b70 msgr2=0x7f45c8078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:02.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.275+0000 7f45c7e00640 1 -- 172.21.15.43:0/3597393821 shutdown_connections 2024-03-26T08:34:02.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.275+0000 7f45c7e00640 1 -- 172.21.15.43:0/3597393821 wait complete. 2024-03-26T08:34:02.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.276+0000 7f45c7e00640 1 Processor -- start 2024-03-26T08:34:02.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.276+0000 7f45c7e00640 1 -- start start 2024-03-26T08:34:02.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.276+0000 7f45c7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45c8107bc0 0x7f45c819ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:02.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.276+0000 7f45c7e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45c8108db0 0x7f45c819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:02.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.276+0000 7f45c7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45c8109770 0x7f45c81a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:02.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.276+0000 7f45c7e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c8113740 con 0x7f45c8109770 2024-03-26T08:34:02.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.276+0000 7f45c7e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c81135c0 con 0x7f45c8108db0 2024-03-26T08:34:02.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.276+0000 7f45c7e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c81138c0 con 0x7f45c8107bc0 2024-03-26T08:34:02.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.276+0000 7f45c7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45c8109770 0x7f45c81a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:02.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.277+0000 7f45c7400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45c8109770 0x7f45c81a5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55622/0 (socket says 172.21.15.43:55622) 2024-03-26T08:34:02.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.277+0000 7f45c7400640 1 -- 172.21.15.43:0/4058506668 learned_addr learned my addr 172.21.15.43:0/4058506668 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:34:02.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.277+0000 7f45c6a00640 1 --2- 172.21.15.43:0/4058506668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45c8107bc0 0x7f45c819ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:02.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.277+0000 7f45c6000640 1 --2- 172.21.15.43:0/4058506668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45c8108db0 0x7f45c819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:02.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.277+0000 7f45c7400640 1 -- 172.21.15.43:0/4058506668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45c8107bc0 msgr2=0x7f45c819ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:02.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.277+0000 7f45c7400640 1 --2- 172.21.15.43:0/4058506668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45c8107bc0 0x7f45c819ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:02.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.277+0000 7f45c7400640 1 -- 172.21.15.43:0/4058506668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45c8108db0 msgr2=0x7f45c819e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:02.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.277+0000 7f45c7400640 1 --2- 172.21.15.43:0/4058506668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45c8108db0 0x7f45c819e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:02.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.277+0000 7f45c7400640 1 -- 172.21.15.43:0/4058506668 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c81a5aa0 con 0x7f45c8109770 2024-03-26T08:34:02.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.277+0000 7f45c7400640 1 --2- 172.21.15.43:0/4058506668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45c8109770 0x7f45c81a5430 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f45c000b970 tx=0x7f45c000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:02.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.278+0000 7f45cca00640 1 -- 172.21.15.43:0/4058506668 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c0019070 con 0x7f45c8109770 2024-03-26T08:34:02.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.278+0000 7f45cca00640 1 -- 172.21.15.43:0/4058506668 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f45c0014440 con 0x7f45c8109770 2024-03-26T08:34:02.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.278+0000 7f45c7e00640 1 -- 172.21.15.43:0/4058506668 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45c81a5d90 con 0x7f45c8109770 2024-03-26T08:34:02.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.278+0000 7f45cca00640 1 -- 172.21.15.43:0/4058506668 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c00075d0 con 0x7f45c8109770 2024-03-26T08:34:02.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.278+0000 7f45c7e00640 1 -- 172.21.15.43:0/4058506668 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c81a6300 con 0x7f45c8109770 2024-03-26T08:34:02.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.279+0000 7f45c7e00640 1 -- 172.21.15.43:0/4058506668 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45c810f000 con 0x7f45c8109770 2024-03-26T08:34:02.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.281+0000 7f45cca00640 1 -- 172.21.15.43:0/4058506668 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f45c001c070 con 0x7f45c8109770 2024-03-26T08:34:02.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.283+0000 7f45cca00640 1 --2- 172.21.15.43:0/4058506668 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f45ac0a2800 0x7f45ac0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:02.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.283+0000 7f45cca00640 1 -- 172.21.15.43:0/4058506668 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f45c00ce440 con 0x7f45c8109770 2024-03-26T08:34:02.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.283+0000 7f45c6a00640 1 --2- 172.21.15.43:0/4058506668 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f45ac0a2800 0x7f45ac0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:02.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.284+0000 7f45c6a00640 1 --2- 172.21.15.43:0/4058506668 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f45ac0a2800 0x7f45ac0a4cc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f45b8006010 tx=0x7f45b802c800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:02.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.288+0000 7f45cca00640 1 -- 172.21.15.43:0/4058506668 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f45c009b100 con 0x7f45c8109770 2024-03-26T08:34:02.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:02 smithi052 ceph-mon[35071]: pgmap v456: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:34:02.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:02 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3577482801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:34:02.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:02 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/4115895584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:34:02.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:02 smithi043 ceph-mon[31929]: pgmap v456: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:34:02.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:02 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3577482801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:34:02.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:02 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/4115895584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:34:02.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.487+0000 7f45c7e00640 1 -- 172.21.15.43:0/4058506668 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f45c80622a0 con 0x7f45c8109770 2024-03-26T08:34:02.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.488+0000 7f45cca00640 1 -- 172.21.15.43:0/4058506668 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f45c009aa00 con 0x7f45c8109770 2024-03-26T08:34:02.481 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-26T08:34:02.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.491+0000 7f45c7e00640 1 -- 172.21.15.43:0/4058506668 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f45ac0a2800 msgr2=0x7f45ac0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:02.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.491+0000 7f45c7e00640 1 --2- 172.21.15.43:0/4058506668 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f45ac0a2800 0x7f45ac0a4cc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f45b8006010 tx=0x7f45b802c800 comp rx=0 tx=0).stop 2024-03-26T08:34:02.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.491+0000 7f45c7e00640 1 -- 172.21.15.43:0/4058506668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45c8109770 msgr2=0x7f45c81a5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:02.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.491+0000 7f45c7e00640 1 --2- 172.21.15.43:0/4058506668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45c8109770 0x7f45c81a5430 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f45c000b970 tx=0x7f45c000be30 comp rx=0 tx=0).stop 2024-03-26T08:34:02.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.491+0000 7f45c7e00640 1 -- 172.21.15.43:0/4058506668 shutdown_connections 2024-03-26T08:34:02.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.491+0000 7f45c7e00640 1 --2- 172.21.15.43:0/4058506668 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f45ac0a2800 0x7f45ac0a4cc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:02.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.491+0000 7f45c7e00640 1 --2- 172.21.15.43:0/4058506668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45c8109770 0x7f45c81a5430 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:02.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.491+0000 7f45c7e00640 1 --2- 172.21.15.43:0/4058506668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f45c8108db0 0x7f45c819e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:02.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.491+0000 7f45c7e00640 1 --2- 172.21.15.43:0/4058506668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45c8107bc0 0x7f45c819ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:02.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.491+0000 7f45c7e00640 1 -- 172.21.15.43:0/4058506668 >> 172.21.15.43:0/4058506668 conn(0x7f45c8076b70 msgr2=0x7f45c810c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:02.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.491+0000 7f45c7e00640 1 -- 172.21.15.43:0/4058506668 shutdown_connections 2024-03-26T08:34:02.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.491+0000 7f45c7e00640 1 -- 172.21.15.43:0/4058506668 wait complete. 2024-03-26T08:34:02.503 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T08:34:02.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:02 smithi183 ceph-mon[35818]: pgmap v456: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:34:02.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:02 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3577482801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:34:02.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:02 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/4115895584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:34:02.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.731+0000 7ff28be00640 1 -- 172.21.15.43:0/478726768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c0756e0 msgr2=0x7ff28c073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:02.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.731+0000 7ff28be00640 1 --2- 172.21.15.43:0/478726768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c0756e0 0x7ff28c073b30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff27c009930 tx=0x7ff27c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:34:02.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.731+0000 7ff28be00640 1 -- 172.21.15.43:0/478726768 shutdown_connections 2024-03-26T08:34:02.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.731+0000 7ff28be00640 1 --2- 172.21.15.43:0/478726768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff28c10db70 0x7ff28c10ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:02.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.731+0000 7ff28be00640 1 --2- 172.21.15.43:0/478726768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff28c074100 0x7ff28c074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:02.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.731+0000 7ff28be00640 1 --2- 172.21.15.43:0/478726768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c0756e0 0x7ff28c073b30 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:02.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.731+0000 7ff28be00640 1 -- 172.21.15.43:0/478726768 >> 172.21.15.43:0/478726768 conn(0x7ff28c0fcb10 msgr2=0x7ff28c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:02.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.731+0000 7ff28be00640 1 -- 172.21.15.43:0/478726768 shutdown_connections 2024-03-26T08:34:02.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.731+0000 7ff28be00640 1 -- 172.21.15.43:0/478726768 wait complete. 2024-03-26T08:34:02.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.732+0000 7ff28be00640 1 Processor -- start 2024-03-26T08:34:02.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.732+0000 7ff28be00640 1 -- start start 2024-03-26T08:34:02.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.732+0000 7ff28be00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff28c074100 0x7ff28c19dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:02.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.733+0000 7ff28be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff28c0756e0 0x7ff28c19e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:02.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.733+0000 7ff28be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c10db70 0x7ff28c1a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:02.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.733+0000 7ff28be00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c1136a0 con 0x7ff28c10db70 2024-03-26T08:34:02.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.733+0000 7ff28be00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c113520 con 0x7ff28c074100 2024-03-26T08:34:02.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.733+0000 7ff28be00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c113820 con 0x7ff28c0756e0 2024-03-26T08:34:02.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.733+0000 7ff28b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c10db70 0x7ff28c1a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:02.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.733+0000 7ff28b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c10db70 0x7ff28c1a52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55642/0 (socket says 172.21.15.43:55642) 2024-03-26T08:34:02.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.733+0000 7ff28b400640 1 -- 172.21.15.43:0/674 learned_addr learned my addr 172.21.15.43:0/674 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:34:02.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.733+0000 7ff28aa00640 1 --2- 172.21.15.43:0/674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff28c074100 0x7ff28c19dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:02.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.733+0000 7ff282000640 1 --2- 172.21.15.43:0/674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff28c0756e0 0x7ff28c19e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:02.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.733+0000 7ff28b400640 1 -- 172.21.15.43:0/674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff28c0756e0 msgr2=0x7ff28c19e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:02.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.733+0000 7ff28b400640 1 --2- 172.21.15.43:0/674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff28c0756e0 0x7ff28c19e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:02.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.733+0000 7ff28b400640 1 -- 172.21.15.43:0/674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff28c074100 msgr2=0x7ff28c19dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:02.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.733+0000 7ff28b400640 1 --2- 172.21.15.43:0/674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff28c074100 0x7ff28c19dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:02.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.733+0000 7ff28b400640 1 -- 172.21.15.43:0/674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28c1a5940 con 0x7ff28c10db70 2024-03-26T08:34:02.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.734+0000 7ff28b400640 1 --2- 172.21.15.43:0/674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c10db70 0x7ff28c1a52a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff28400cce0 tx=0x7ff28400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:02.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.734+0000 7ff283e00640 1 -- 172.21.15.43:0/674 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28400f450 con 0x7ff28c10db70 2024-03-26T08:34:02.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.734+0000 7ff283e00640 1 -- 172.21.15.43:0/674 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff28400faa0 con 0x7ff28c10db70 2024-03-26T08:34:02.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.734+0000 7ff283e00640 1 -- 172.21.15.43:0/674 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff284018b70 con 0x7ff28c10db70 2024-03-26T08:34:02.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.734+0000 7ff28be00640 1 -- 172.21.15.43:0/674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff28c1a5c30 con 0x7ff28c10db70 2024-03-26T08:34:02.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.734+0000 7ff28be00640 1 -- 172.21.15.43:0/674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff28c1a6180 con 0x7ff28c10db70 2024-03-26T08:34:02.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.736+0000 7ff28be00640 1 -- 172.21.15.43:0/674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff28c103d40 con 0x7ff28c10db70 2024-03-26T08:34:02.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.737+0000 7ff283e00640 1 -- 172.21.15.43:0/674 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ff284018d10 con 0x7ff28c10db70 2024-03-26T08:34:02.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.738+0000 7ff283e00640 1 --2- 172.21.15.43:0/674 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff2700a2880 0x7ff2700a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:02.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.739+0000 7ff283e00640 1 -- 172.21.15.43:0/674 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7ff2840cf330 con 0x7ff28c10db70 2024-03-26T08:34:02.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.739+0000 7ff28aa00640 1 --2- 172.21.15.43:0/674 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff2700a2880 0x7ff2700a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:02.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.740+0000 7ff28aa00640 1 --2- 172.21.15.43:0/674 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff2700a2880 0x7ff2700a4d40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff27c009900 tx=0x7ff27c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:02.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.743+0000 7ff283e00640 1 -- 172.21.15.43:0/674 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff284014030 con 0x7ff28c10db70 2024-03-26T08:34:02.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.943+0000 7ff28be00640 1 -- 172.21.15.43:0/674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff28c0622a0 con 0x7ff28c10db70 2024-03-26T08:34:02.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.944+0000 7ff283e00640 1 -- 172.21.15.43:0/674 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff28409bff0 con 0x7ff28c10db70 2024-03-26T08:34:02.936 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-26T08:34:02.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.946+0000 7ff28be00640 1 -- 172.21.15.43:0/674 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff2700a2880 msgr2=0x7ff2700a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:02.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.946+0000 7ff28be00640 1 --2- 172.21.15.43:0/674 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff2700a2880 0x7ff2700a4d40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff27c009900 tx=0x7ff27c005fd0 comp rx=0 tx=0).stop 2024-03-26T08:34:02.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.946+0000 7ff28be00640 1 -- 172.21.15.43:0/674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c10db70 msgr2=0x7ff28c1a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:02.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.946+0000 7ff28be00640 1 --2- 172.21.15.43:0/674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c10db70 0x7ff28c1a52a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff28400cce0 tx=0x7ff28400a620 comp rx=0 tx=0).stop 2024-03-26T08:34:02.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.946+0000 7ff28be00640 1 -- 172.21.15.43:0/674 shutdown_connections 2024-03-26T08:34:02.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.946+0000 7ff28be00640 1 --2- 172.21.15.43:0/674 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ff2700a2880 0x7ff2700a4d40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:02.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.946+0000 7ff28be00640 1 --2- 172.21.15.43:0/674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff28c10db70 0x7ff28c1a52a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:02.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.946+0000 7ff28be00640 1 --2- 172.21.15.43:0/674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff28c0756e0 0x7ff28c19e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:02.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.946+0000 7ff28be00640 1 --2- 172.21.15.43:0/674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff28c074100 0x7ff28c19dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:02.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.946+0000 7ff28be00640 1 -- 172.21.15.43:0/674 >> 172.21.15.43:0/674 conn(0x7ff28c0fcb10 msgr2=0x7ff28c100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:02.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.947+0000 7ff28be00640 1 -- 172.21.15.43:0/674 shutdown_connections 2024-03-26T08:34:02.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:02.947+0000 7ff28be00640 1 -- 172.21.15.43:0/674 wait complete. 2024-03-26T08:34:02.959 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T08:34:03.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.192+0000 7f88ba000640 1 -- 172.21.15.43:0/3771199239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88b4105470 msgr2=0x7f88b410bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:03.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.192+0000 7f88ba000640 1 --2- 172.21.15.43:0/3771199239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88b4105470 0x7f88b410bd00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f88ac009ca0 tx=0x7f88ac02c0f0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.192+0000 7f88ba000640 1 -- 172.21.15.43:0/3771199239 shutdown_connections 2024-03-26T08:34:03.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.192+0000 7f88ba000640 1 --2- 172.21.15.43:0/3771199239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88b4105470 0x7f88b410bd00 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.192+0000 7f88ba000640 1 --2- 172.21.15.43:0/3771199239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88b4104ab0 0x7f88b4104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.192+0000 7f88ba000640 1 --2- 172.21.15.43:0/3771199239 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88b41038c0 0x7f88b4103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.192+0000 7f88ba000640 1 -- 172.21.15.43:0/3771199239 >> 172.21.15.43:0/3771199239 conn(0x7f88b40fee50 msgr2=0x7f88b4101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:03.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.192+0000 7f88ba000640 1 -- 172.21.15.43:0/3771199239 shutdown_connections 2024-03-26T08:34:03.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.192+0000 7f88ba000640 1 -- 172.21.15.43:0/3771199239 wait complete. 2024-03-26T08:34:03.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.193+0000 7f88ba000640 1 Processor -- start 2024-03-26T08:34:03.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.193+0000 7f88ba000640 1 -- start start 2024-03-26T08:34:03.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.193+0000 7f88ba000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88b41038c0 0x7f88b4199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:03.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.193+0000 7f88ba000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88b4104ab0 0x7f88b4199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:03.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.193+0000 7f88ba000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88b4105470 0x7f88b41a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:03.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.193+0000 7f88ba000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b410f3f0 con 0x7f88b4104ab0 2024-03-26T08:34:03.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.193+0000 7f88ba000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b410f270 con 0x7f88b41038c0 2024-03-26T08:34:03.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.193+0000 7f88ba000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b410f570 con 0x7f88b4105470 2024-03-26T08:34:03.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.194+0000 7f88b2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88b4104ab0 0x7f88b4199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:03.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.194+0000 7f88b2a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88b4104ab0 0x7f88b4199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55656/0 (socket says 172.21.15.43:55656) 2024-03-26T08:34:03.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.194+0000 7f88b2a00640 1 -- 172.21.15.43:0/1625692072 learned_addr learned my addr 172.21.15.43:0/1625692072 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:34:03.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.194+0000 7f88b3400640 1 --2- 172.21.15.43:0/1625692072 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88b41038c0 0x7f88b4199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:03.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.194+0000 7f88b2a00640 1 -- 172.21.15.43:0/1625692072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88b4105470 msgr2=0x7f88b41a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:03.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.194+0000 7f88b3e00640 1 --2- 172.21.15.43:0/1625692072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88b4105470 0x7f88b41a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:03.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.194+0000 7f88b2a00640 1 --2- 172.21.15.43:0/1625692072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88b4105470 0x7f88b41a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.194+0000 7f88b2a00640 1 -- 172.21.15.43:0/1625692072 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88b41038c0 msgr2=0x7f88b4199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:03.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.194+0000 7f88b2a00640 1 --2- 172.21.15.43:0/1625692072 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88b41038c0 0x7f88b4199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.194+0000 7f88b2a00640 1 -- 172.21.15.43:0/1625692072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b41a16c0 con 0x7f88b4104ab0 2024-03-26T08:34:03.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.194+0000 7f88b3e00640 1 --2- 172.21.15.43:0/1625692072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88b4105470 0x7f88b41a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:34:03.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.195+0000 7f88b2a00640 1 --2- 172.21.15.43:0/1625692072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88b4104ab0 0x7f88b4199fe0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f889c00d770 tx=0x7f889c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:03.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.195+0000 7f88abe00640 1 -- 172.21.15.43:0/1625692072 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c017070 con 0x7f88b4104ab0 2024-03-26T08:34:03.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.195+0000 7f88abe00640 1 -- 172.21.15.43:0/1625692072 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f889c0054d0 con 0x7f88b4104ab0 2024-03-26T08:34:03.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.195+0000 7f88ba000640 1 -- 172.21.15.43:0/1625692072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b41a19b0 con 0x7f88b4104ab0 2024-03-26T08:34:03.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.195+0000 7f88abe00640 1 -- 172.21.15.43:0/1625692072 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c004cf0 con 0x7f88b4104ab0 2024-03-26T08:34:03.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.195+0000 7f88ba000640 1 -- 172.21.15.43:0/1625692072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b41a1f00 con 0x7f88b4104ab0 2024-03-26T08:34:03.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.196+0000 7f88ba000640 1 -- 172.21.15.43:0/1625692072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b410ad00 con 0x7f88b4104ab0 2024-03-26T08:34:03.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.198+0000 7f88abe00640 1 -- 172.21.15.43:0/1625692072 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f889c004e90 con 0x7f88b4104ab0 2024-03-26T08:34:03.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.201+0000 7f88abe00640 1 --2- 172.21.15.43:0/1625692072 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f88980a2880 0x7f88980a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:03.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.201+0000 7f88abe00640 1 -- 172.21.15.43:0/1625692072 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f889c0ce850 con 0x7f88b4104ab0 2024-03-26T08:34:03.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.201+0000 7f88b3400640 1 --2- 172.21.15.43:0/1625692072 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f88980a2880 0x7f88980a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:03.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.202+0000 7f88b3400640 1 --2- 172.21.15.43:0/1625692072 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f88980a2880 0x7f88980a4d40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f88a4005d00 tx=0x7f88a4005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:03.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.205+0000 7f88abe00640 1 -- 172.21.15.43:0/1625692072 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f889c09b510 con 0x7f88b4104ab0 2024-03-26T08:34:03.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:03 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/4058506668' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:34:03.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:03 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:34:03.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.412+0000 7f88ba000640 1 -- 172.21.15.43:0/1625692072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f88b40622a0 con 0x7f88b4104ab0 2024-03-26T08:34:03.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.413+0000 7f88abe00640 1 -- 172.21.15.43:0/1625692072 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7f889c09ae10 con 0x7f88b4104ab0 2024-03-26T08:34:03.405 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-26T08:34:03.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.416+0000 7f88ba000640 1 -- 172.21.15.43:0/1625692072 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f88980a2880 msgr2=0x7f88980a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:03.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.416+0000 7f88ba000640 1 --2- 172.21.15.43:0/1625692072 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f88980a2880 0x7f88980a4d40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f88a4005d00 tx=0x7f88a4005c50 comp rx=0 tx=0).stop 2024-03-26T08:34:03.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.416+0000 7f88ba000640 1 -- 172.21.15.43:0/1625692072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88b4104ab0 msgr2=0x7f88b4199fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:03.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.416+0000 7f88ba000640 1 --2- 172.21.15.43:0/1625692072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88b4104ab0 0x7f88b4199fe0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f889c00d770 tx=0x7f889c00dc30 comp rx=0 tx=0).stop 2024-03-26T08:34:03.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.416+0000 7f88ba000640 1 -- 172.21.15.43:0/1625692072 shutdown_connections 2024-03-26T08:34:03.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.416+0000 7f88ba000640 1 --2- 172.21.15.43:0/1625692072 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f88980a2880 0x7f88980a4d40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.416+0000 7f88ba000640 1 --2- 172.21.15.43:0/1625692072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88b4105470 0x7f88b41a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.416+0000 7f88ba000640 1 --2- 172.21.15.43:0/1625692072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88b4104ab0 0x7f88b4199fe0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.416+0000 7f88ba000640 1 --2- 172.21.15.43:0/1625692072 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88b41038c0 0x7f88b4199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.416+0000 7f88ba000640 1 -- 172.21.15.43:0/1625692072 >> 172.21.15.43:0/1625692072 conn(0x7f88b40fee50 msgr2=0x7f88b4107ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:03.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.416+0000 7f88ba000640 1 -- 172.21.15.43:0/1625692072 shutdown_connections 2024-03-26T08:34:03.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.416+0000 7f88ba000640 1 -- 172.21.15.43:0/1625692072 wait complete. 2024-03-26T08:34:03.428 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:34:03.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:03 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/4058506668' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:34:03.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:03 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:34:03.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:03 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/4058506668' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:34:03.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:03 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:34:03.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.662+0000 7f1be1000640 1 -- 172.21.15.43:0/2166770467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bdc100900 msgr2=0x7f1bdc100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:03.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.662+0000 7f1be1000640 1 --2- 172.21.15.43:0/2166770467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bdc100900 0x7f1bdc100d20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009900 tx=0x7f1bcc02bc30 comp rx=0 tx=0).stop 2024-03-26T08:34:03.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.662+0000 7f1be1000640 1 -- 172.21.15.43:0/2166770467 shutdown_connections 2024-03-26T08:34:03.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.662+0000 7f1be1000640 1 --2- 172.21.15.43:0/2166770467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bdc0ffd10 0x7f1bdc1019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.662+0000 7f1be1000640 1 --2- 172.21.15.43:0/2166770467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bdc0ff350 0x7f1bdc0ff7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.662+0000 7f1be1000640 1 --2- 172.21.15.43:0/2166770467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bdc100900 0x7f1bdc100d20 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.662+0000 7f1be1000640 1 -- 172.21.15.43:0/2166770467 >> 172.21.15.43:0/2166770467 conn(0x7f1bdc0faea0 msgr2=0x7f1bdc0fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:03.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.662+0000 7f1be1000640 1 -- 172.21.15.43:0/2166770467 shutdown_connections 2024-03-26T08:34:03.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.662+0000 7f1be1000640 1 -- 172.21.15.43:0/2166770467 wait complete. 2024-03-26T08:34:03.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.663+0000 7f1be1000640 1 Processor -- start 2024-03-26T08:34:03.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.664+0000 7f1be1000640 1 -- start start 2024-03-26T08:34:03.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.664+0000 7f1be1000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bdc0ff350 0x7f1bdc199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:03.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.664+0000 7f1be1000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bdc0ffd10 0x7f1bdc19a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:03.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.664+0000 7f1be1000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bdc100900 0x7f1bdc1a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:03.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.664+0000 7f1be1000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bdc10f450 con 0x7f1bdc0ffd10 2024-03-26T08:34:03.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.664+0000 7f1be1000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bdc10f2d0 con 0x7f1bdc0ff350 2024-03-26T08:34:03.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.664+0000 7f1be1000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bdc10f5d0 con 0x7f1bdc100900 2024-03-26T08:34:03.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.664+0000 7f1bd9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bdc0ffd10 0x7f1bdc19a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:03.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.664+0000 7f1bd9400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bdc0ffd10 0x7f1bdc19a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55678/0 (socket says 172.21.15.43:55678) 2024-03-26T08:34:03.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.664+0000 7f1bd9400640 1 -- 172.21.15.43:0/658387730 learned_addr learned my addr 172.21.15.43:0/658387730 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:34:03.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.665+0000 7f1bd9e00640 1 --2- 172.21.15.43:0/658387730 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bdc0ff350 0x7f1bdc199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:03.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.665+0000 7f1bda800640 1 --2- 172.21.15.43:0/658387730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bdc100900 0x7f1bdc1a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:03.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.665+0000 7f1bd9400640 1 -- 172.21.15.43:0/658387730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bdc100900 msgr2=0x7f1bdc1a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:03.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.665+0000 7f1bd9400640 1 --2- 172.21.15.43:0/658387730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bdc100900 0x7f1bdc1a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.665+0000 7f1bd9400640 1 -- 172.21.15.43:0/658387730 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bdc0ff350 msgr2=0x7f1bdc199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:03.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.665+0000 7f1bd9400640 1 --2- 172.21.15.43:0/658387730 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bdc0ff350 0x7f1bdc199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.665+0000 7f1bd9400640 1 -- 172.21.15.43:0/658387730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bdc1a1720 con 0x7f1bdc0ffd10 2024-03-26T08:34:03.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.665+0000 7f1bda800640 1 --2- 172.21.15.43:0/658387730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bdc100900 0x7f1bdc1a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:34:03.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.665+0000 7f1bd9400640 1 --2- 172.21.15.43:0/658387730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bdc0ffd10 0x7f1bdc19a040 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1bc800d770 tx=0x7f1bc800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:03.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.665+0000 7f1bd2a00640 1 -- 172.21.15.43:0/658387730 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8017070 con 0x7f1bdc0ffd10 2024-03-26T08:34:03.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.665+0000 7f1bd2a00640 1 -- 172.21.15.43:0/658387730 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1bc8009c30 con 0x7f1bdc0ffd10 2024-03-26T08:34:03.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.665+0000 7f1bd2a00640 1 -- 172.21.15.43:0/658387730 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8003d00 con 0x7f1bdc0ffd10 2024-03-26T08:34:03.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.665+0000 7f1be1000640 1 -- 172.21.15.43:0/658387730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bdc1a1a10 con 0x7f1bdc0ffd10 2024-03-26T08:34:03.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.666+0000 7f1be1000640 1 -- 172.21.15.43:0/658387730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bdc1a1f10 con 0x7f1bdc0ffd10 2024-03-26T08:34:03.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.667+0000 7f1be1000640 1 -- 172.21.15.43:0/658387730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bdc10bbb0 con 0x7f1bdc0ffd10 2024-03-26T08:34:03.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.669+0000 7f1bd2a00640 1 -- 172.21.15.43:0/658387730 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1bc8003ea0 con 0x7f1bdc0ffd10 2024-03-26T08:34:03.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.671+0000 7f1bd2a00640 1 --2- 172.21.15.43:0/658387730 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1bc00a2880 0x7f1bc00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:03.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.671+0000 7f1bd2a00640 1 -- 172.21.15.43:0/658387730 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f1bc80ceb90 con 0x7f1bdc0ffd10 2024-03-26T08:34:03.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.671+0000 7f1bd9e00640 1 --2- 172.21.15.43:0/658387730 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1bc00a2880 0x7f1bc00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:03.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.672+0000 7f1bd9e00640 1 --2- 172.21.15.43:0/658387730 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1bc00a2880 0x7f1bc00a4d40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc006010 tx=0x7f1bcc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:03.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.675+0000 7f1bd2a00640 1 -- 172.21.15.43:0/658387730 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1bc809b850 con 0x7f1bdc0ffd10 2024-03-26T08:34:03.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.922+0000 7f1be1000640 1 -- 172.21.15.43:0/658387730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1bdc062340 con 0x7f1bdc0ffd10 2024-03-26T08:34:03.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.923+0000 7f1bd2a00640 1 -- 172.21.15.43:0/658387730 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11331 (secure 0 0 0) 0x7f1bc8009de0 con 0x7f1bdc0ffd10 2024-03-26T08:34:03.916 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:34:03.916 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1452206807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1452206807},{"type":"v1","addr":"172.21.15.43:6835","nonce":1452206807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3151775467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3151775467},{"type":"v1","addr":"172.21.15.43:6837","nonce":3151775467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/1284655682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":1284655682},{"type":"v1","addr":"172.21.15.43:6839","nonce":1284655682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3081085787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3081085787},{"type":"v1","addr":"172.21.15.43:6841","nonce":3081085787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1842295306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1842295306},{"type":"v1","addr":"172.21.15.183:6835","nonce":1842295306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2267781666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2267781666},{"type":"v1","addr":"172.21.15.183:6837","nonce":2267781666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3572429948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3572429948},{"type":"v1","addr":"172.21.15.183:6839","nonce":3572429948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34448,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/1108327179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108327179},{"type":"v1","addr":"172.21.15.52:6833","nonce":1108327179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34487,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/742057410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":742057410},{"type":"v1","addr":"172.21.15.52:6835","nonce":742057410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:31:21.819689+0000","modified":"2024-03-26T08:31:48.080233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34517":{"gid":34517,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/748239918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":748239918},{"type":"v1","addr":"172.21.15.52:6837","nonce":748239918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34559":{"gid":34559,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.52:6839/2551965101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2551965101},{"type":"v1","addr":"172.21.15.52:6839","nonce":2551965101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34559,"qdb_cluster":[34559]},"id":1}]} 2024-03-26T08:34:03.916 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 21 2024-03-26T08:34:03.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.925+0000 7f1be1000640 1 -- 172.21.15.43:0/658387730 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1bc00a2880 msgr2=0x7f1bc00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:03.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.925+0000 7f1be1000640 1 --2- 172.21.15.43:0/658387730 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1bc00a2880 0x7f1bc00a4d40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc006010 tx=0x7f1bcc000bc0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.925+0000 7f1be1000640 1 -- 172.21.15.43:0/658387730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bdc0ffd10 msgr2=0x7f1bdc19a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:03.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.925+0000 7f1be1000640 1 --2- 172.21.15.43:0/658387730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bdc0ffd10 0x7f1bdc19a040 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1bc800d770 tx=0x7f1bc800dc30 comp rx=0 tx=0).stop 2024-03-26T08:34:03.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.925+0000 7f1be1000640 1 -- 172.21.15.43:0/658387730 shutdown_connections 2024-03-26T08:34:03.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.925+0000 7f1be1000640 1 --2- 172.21.15.43:0/658387730 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1bc00a2880 0x7f1bc00a4d40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.925+0000 7f1be1000640 1 --2- 172.21.15.43:0/658387730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bdc100900 0x7f1bdc1a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.925+0000 7f1be1000640 1 --2- 172.21.15.43:0/658387730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bdc0ffd10 0x7f1bdc19a040 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.925+0000 7f1be1000640 1 --2- 172.21.15.43:0/658387730 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bdc0ff350 0x7f1bdc199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.925+0000 7f1be1000640 1 -- 172.21.15.43:0/658387730 >> 172.21.15.43:0/658387730 conn(0x7f1bdc0faea0 msgr2=0x7f1bdc0fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:03.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.926+0000 7f1be1000640 1 -- 172.21.15.43:0/658387730 shutdown_connections 2024-03-26T08:34:03.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:03.926+0000 7f1be1000640 1 -- 172.21.15.43:0/658387730 wait complete. 2024-03-26T08:34:03.937 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:34:04.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.170+0000 7f3d47200640 1 -- 172.21.15.43:0/4171056668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d481048e0 msgr2=0x7f3d48104d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:04.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.170+0000 7f3d47200640 1 --2- 172.21.15.43:0/4171056668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d481048e0 0x7f3d48104d60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3d34009900 tx=0x7f3d3402bc40 comp rx=0 tx=0).stop 2024-03-26T08:34:04.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.170+0000 7f3d47200640 1 -- 172.21.15.43:0/4171056668 shutdown_connections 2024-03-26T08:34:04.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.170+0000 7f3d47200640 1 --2- 172.21.15.43:0/4171056668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d481052a0 0x7f3d4810bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:04.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.170+0000 7f3d47200640 1 --2- 172.21.15.43:0/4171056668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d481048e0 0x7f3d48104d60 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:04.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.170+0000 7f3d47200640 1 --2- 172.21.15.43:0/4171056668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d481036f0 0x7f3d48103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:04.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.170+0000 7f3d47200640 1 -- 172.21.15.43:0/4171056668 >> 172.21.15.43:0/4171056668 conn(0x7f3d480fec80 msgr2=0x7f3d481010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:04.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.170+0000 7f3d47200640 1 -- 172.21.15.43:0/4171056668 shutdown_connections 2024-03-26T08:34:04.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.170+0000 7f3d47200640 1 -- 172.21.15.43:0/4171056668 wait complete. 2024-03-26T08:34:04.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.172+0000 7f3d47200640 1 Processor -- start 2024-03-26T08:34:04.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.172+0000 7f3d47200640 1 -- start start 2024-03-26T08:34:04.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.172+0000 7f3d47200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d481036f0 0x7f3d48079450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:04.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.172+0000 7f3d47200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d481048e0 0x7f3d48079990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:04.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.172+0000 7f3d47200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d481052a0 0x7f3d48075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:04.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.172+0000 7f3d47200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4810f250 con 0x7f3d481048e0 2024-03-26T08:34:04.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.172+0000 7f3d47200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4810f0d0 con 0x7f3d481052a0 2024-03-26T08:34:04.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.172+0000 7f3d47200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4810f3d0 con 0x7f3d481036f0 2024-03-26T08:34:04.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.173+0000 7f3d45400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d481048e0 0x7f3d48079990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:04.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.173+0000 7f3d45400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d481048e0 0x7f3d48079990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36808/0 (socket says 172.21.15.43:36808) 2024-03-26T08:34:04.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.173+0000 7f3d45400640 1 -- 172.21.15.43:0/2542195244 learned_addr learned my addr 172.21.15.43:0/2542195244 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:34:04.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.173+0000 7f3d46800640 1 --2- 172.21.15.43:0/2542195244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d481052a0 0x7f3d48075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:04.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.173+0000 7f3d45e00640 1 --2- 172.21.15.43:0/2542195244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d481036f0 0x7f3d48079450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:04.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.173+0000 7f3d45400640 1 -- 172.21.15.43:0/2542195244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d481036f0 msgr2=0x7f3d48079450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:04.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.173+0000 7f3d45400640 1 --2- 172.21.15.43:0/2542195244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d481036f0 0x7f3d48079450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:04.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.173+0000 7f3d45400640 1 -- 172.21.15.43:0/2542195244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d481052a0 msgr2=0x7f3d48075fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:04.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.173+0000 7f3d45400640 1 --2- 172.21.15.43:0/2542195244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d481052a0 0x7f3d48075fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:04.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.173+0000 7f3d45400640 1 -- 172.21.15.43:0/2542195244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d48076880 con 0x7f3d481048e0 2024-03-26T08:34:04.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.174+0000 7f3d45400640 1 --2- 172.21.15.43:0/2542195244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d481048e0 0x7f3d48079990 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3d34000c00 tx=0x7f3d340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:04.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.174+0000 7f3d3ea00640 1 -- 172.21.15.43:0/2542195244 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d34043020 con 0x7f3d481048e0 2024-03-26T08:34:04.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.174+0000 7f3d3ea00640 1 -- 172.21.15.43:0/2542195244 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3d340054c0 con 0x7f3d481048e0 2024-03-26T08:34:04.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.174+0000 7f3d3ea00640 1 -- 172.21.15.43:0/2542195244 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d34003d80 con 0x7f3d481048e0 2024-03-26T08:34:04.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.174+0000 7f3d47200640 1 -- 172.21.15.43:0/2542195244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d481a5c80 con 0x7f3d481048e0 2024-03-26T08:34:04.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.174+0000 7f3d47200640 1 -- 172.21.15.43:0/2542195244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d481a6030 con 0x7f3d481048e0 2024-03-26T08:34:04.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.175+0000 7f3d47200640 1 -- 172.21.15.43:0/2542195244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d48068800 con 0x7f3d481048e0 2024-03-26T08:34:04.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.177+0000 7f3d3ea00640 1 -- 172.21.15.43:0/2542195244 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f3d34046020 con 0x7f3d481048e0 2024-03-26T08:34:04.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.179+0000 7f3d3ea00640 1 --2- 172.21.15.43:0/2542195244 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3d2c0a2880 0x7f3d2c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:04.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.179+0000 7f3d3ea00640 1 -- 172.21.15.43:0/2542195244 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f3d340f12a0 con 0x7f3d481048e0 2024-03-26T08:34:04.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.179+0000 7f3d45e00640 1 --2- 172.21.15.43:0/2542195244 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3d2c0a2880 0x7f3d2c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:04.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.180+0000 7f3d45e00640 1 --2- 172.21.15.43:0/2542195244 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3d2c0a2880 0x7f3d2c0a4d40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3d380058c0 tx=0x7f3d38005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:04.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.183+0000 7f3d3ea00640 1 -- 172.21.15.43:0/2542195244 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3d3404d020 con 0x7f3d481048e0 2024-03-26T08:34:04.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.390+0000 7f3d47200640 1 -- 172.21.15.43:0/2542195244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d480622a0 con 0x7f3d481048e0 2024-03-26T08:34:04.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.394+0000 7f3d3ea00640 1 -- 172.21.15.43:0/2542195244 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24670 (secure 0 0 0) 0x7f3d340be080 con 0x7f3d481048e0 2024-03-26T08:34:04.387 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:34:04.387 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":94,"fsid":"88759420-eb49-11ee-95d0-87774f69a715","created":"2024-03-26T08:20:27.817107+0000","modified":"2024-03-26T08:32:38.256877+0000","last_up_change":"2024-03-26T08:30:10.520976+0000","last_in_change":"2024-03-26T08:29:46.394360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:24:54.852427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:31:19.924902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:31:21.323881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T08:31:24.951015+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"28670673-9193-4fa3-9cc0-c1a5a6b27284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6803","nonce":936601146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6805","nonce":936601146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6809","nonce":936601146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6807","nonce":936601146}]},"public_addr":"172.21.15.43:6803/936601146","cluster_addr":"172.21.15.43:6805/936601146","heartbeat_back_addr":"172.21.15.43:6809/936601146","heartbeat_front_addr":"172.21.15.43:6807/936601146","state":["exists","up"]},{"osd":1,"uuid":"fa8ad6b6-277a-42b8-b0f8-b36fc219de2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6801","nonce":3268412328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6803","nonce":3268412328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6807","nonce":3268412328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6805","nonce":3268412328}]},"public_addr":"172.21.15.52:6801/3268412328","cluster_addr":"172.21.15.52:6803/3268412328","heartbeat_back_addr":"172.21.15.52:6807/3268412328","heartbeat_front_addr":"172.21.15.52:6805/3268412328","state":["exists","up"]},{"osd":2,"uuid":"ec0039c3-f9bd-419a-81de-fa4418359808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6801","nonce":2103180908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6803","nonce":2103180908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6807","nonce":2103180908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6805","nonce":2103180908}]},"public_addr":"172.21.15.183:6801/2103180908","cluster_addr":"172.21.15.183:6803/2103180908","heartbeat_back_addr":"172.21.15.183:6807/2103180908","heartbeat_front_addr":"172.21.15.183:6805/2103180908","state":["exists","up"]},{"osd":3,"uuid":"e7dcbf00-6586-4664-a69e-3cb0df8dfd30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6811","nonce":3722846033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6813","nonce":3722846033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6817","nonce":3722846033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6815","nonce":3722846033}]},"public_addr":"172.21.15.43:6811/3722846033","cluster_addr":"172.21.15.43:6813/3722846033","heartbeat_back_addr":"172.21.15.43:6817/3722846033","heartbeat_front_addr":"172.21.15.43:6815/3722846033","state":["exists","up"]},{"osd":4,"uuid":"db089a49-5e98-432d-9687-cdc044114c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6809","nonce":4028436523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6811","nonce":4028436523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6815","nonce":4028436523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6813","nonce":4028436523}]},"public_addr":"172.21.15.52:6809/4028436523","cluster_addr":"172.21.15.52:6811/4028436523","heartbeat_back_addr":"172.21.15.52:6815/4028436523","heartbeat_front_addr":"172.21.15.52:6813/4028436523","state":["exists","up"]},{"osd":5,"uuid":"556ac804-02ea-423b-a599-55baf5be4ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6809","nonce":787132252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6811","nonce":787132252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6815","nonce":787132252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6813","nonce":787132252}]},"public_addr":"172.21.15.183:6809/787132252","cluster_addr":"172.21.15.183:6811/787132252","heartbeat_back_addr":"172.21.15.183:6815/787132252","heartbeat_front_addr":"172.21.15.183:6813/787132252","state":["exists","up"]},{"osd":6,"uuid":"3de2a49d-6cef-4580-9d8a-f90cdfcecd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6819","nonce":4206861158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6821","nonce":4206861158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6825","nonce":4206861158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6823","nonce":4206861158}]},"public_addr":"172.21.15.43:6819/4206861158","cluster_addr":"172.21.15.43:6821/4206861158","heartbeat_back_addr":"172.21.15.43:6825/4206861158","heartbeat_front_addr":"172.21.15.43:6823/4206861158","state":["exists","up"]},{"osd":7,"uuid":"a3b832f7-7d7d-4a62-b679-db6da98275d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6817","nonce":3452064146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6819","nonce":3452064146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6823","nonce":3452064146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6821","nonce":3452064146}]},"public_addr":"172.21.15.52:6817/3452064146","cluster_addr":"172.21.15.52:6819/3452064146","heartbeat_back_addr":"172.21.15.52:6823/3452064146","heartbeat_front_addr":"172.21.15.52:6821/3452064146","state":["exists","up"]},{"osd":8,"uuid":"ddc6b07f-d703-414d-a59a-02316376b13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6817","nonce":1023131725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6819","nonce":1023131725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6823","nonce":1023131725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6821","nonce":1023131725}]},"public_addr":"172.21.15.183:6817/1023131725","cluster_addr":"172.21.15.183:6819/1023131725","heartbeat_back_addr":"172.21.15.183:6823/1023131725","heartbeat_front_addr":"172.21.15.183:6821/1023131725","state":["exists","up"]},{"osd":9,"uuid":"ac16bb0e-6232-45c8-b16a-4d878874c588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6827","nonce":3935195657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6829","nonce":3935195657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6833","nonce":3935195657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6831","nonce":3935195657}]},"public_addr":"172.21.15.43:6827/3935195657","cluster_addr":"172.21.15.43:6829/3935195657","heartbeat_back_addr":"172.21.15.43:6833/3935195657","heartbeat_front_addr":"172.21.15.43:6831/3935195657","state":["exists","up"]},{"osd":10,"uuid":"fd839188-f398-4cff-aa7b-36522ff1a8a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6825","nonce":377628320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6827","nonce":377628320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6831","nonce":377628320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6829","nonce":377628320}]},"public_addr":"172.21.15.52:6825/377628320","cluster_addr":"172.21.15.52:6827/377628320","heartbeat_back_addr":"172.21.15.52:6831/377628320","heartbeat_front_addr":"172.21.15.52:6829/377628320","state":["exists","up"]},{"osd":11,"uuid":"a40282a0-0c64-49ee-b64e-6f3f8119e827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6825","nonce":1373086433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6827","nonce":1373086433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6831","nonce":1373086433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6829","nonce":1373086433}]},"public_addr":"172.21.15.183:6825/1373086433","cluster_addr":"172.21.15.183:6827/1373086433","heartbeat_back_addr":"172.21.15.183:6831/1373086433","heartbeat_front_addr":"172.21.15.183:6829/1373086433","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:23:43.478905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:17.573532+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:50.286610+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:25:27.407663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:03.081294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:36.057416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:12.062013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:47.032271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:21.023201+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:57.364739+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:29:34.039995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:30:08.827577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6801/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/4240775295":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/510203140":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/4251957687":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:6801/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1885244665":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/298320867":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1301092625":"2024-03-27T08:21:36.149092+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:34:04.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.397+0000 7f3d47200640 1 -- 172.21.15.43:0/2542195244 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3d2c0a2880 msgr2=0x7f3d2c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:04.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.397+0000 7f3d47200640 1 --2- 172.21.15.43:0/2542195244 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3d2c0a2880 0x7f3d2c0a4d40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3d380058c0 tx=0x7f3d38005850 comp rx=0 tx=0).stop 2024-03-26T08:34:04.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.397+0000 7f3d47200640 1 -- 172.21.15.43:0/2542195244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d481048e0 msgr2=0x7f3d48079990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:04.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.397+0000 7f3d47200640 1 --2- 172.21.15.43:0/2542195244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d481048e0 0x7f3d48079990 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3d34000c00 tx=0x7f3d340042f0 comp rx=0 tx=0).stop 2024-03-26T08:34:04.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.397+0000 7f3d47200640 1 -- 172.21.15.43:0/2542195244 shutdown_connections 2024-03-26T08:34:04.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.397+0000 7f3d47200640 1 --2- 172.21.15.43:0/2542195244 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3d2c0a2880 0x7f3d2c0a4d40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:04.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.397+0000 7f3d47200640 1 --2- 172.21.15.43:0/2542195244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d481052a0 0x7f3d48075fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:04.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.397+0000 7f3d47200640 1 --2- 172.21.15.43:0/2542195244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d481048e0 0x7f3d48079990 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:04.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.397+0000 7f3d47200640 1 --2- 172.21.15.43:0/2542195244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d481036f0 0x7f3d48079450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:04.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.397+0000 7f3d47200640 1 -- 172.21.15.43:0/2542195244 >> 172.21.15.43:0/2542195244 conn(0x7f3d480fec80 msgr2=0x7f3d48107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:04.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.397+0000 7f3d47200640 1 -- 172.21.15.43:0/2542195244 shutdown_connections 2024-03-26T08:34:04.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.397+0000 7f3d47200640 1 -- 172.21.15.43:0/2542195244 wait complete. 2024-03-26T08:34:04.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:04 smithi043 ceph-mon[31929]: pgmap v457: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:04.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:04 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1625692072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T08:34:04.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:04 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/658387730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:34:04.409 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T08:24:54.852427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T08:31:19.924902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T08:31:21.323881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T08:31:24.951015+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T08:34:04.410 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T08:34:04.474 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:04 smithi052 ceph-mon[35071]: pgmap v457: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:04.474 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:04 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1625692072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T08:34:04.474 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:04 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/658387730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:34:04.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:04 smithi183 ceph-mon[35818]: pgmap v457: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:04.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:04 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1625692072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T08:34:04.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:04 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/658387730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:34:04.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.643+0000 7f11fbc00640 1 -- 172.21.15.43:0/1287952920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11f4107cc0 msgr2=0x7f11f41080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:04.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.643+0000 7f11fbc00640 1 --2- 172.21.15.43:0/1287952920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11f4107cc0 0x7f11f41080e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009900 tx=0x7f11e802bc30 comp rx=0 tx=0).stop 2024-03-26T08:34:04.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.644+0000 7f11fbc00640 1 -- 172.21.15.43:0/1287952920 shutdown_connections 2024-03-26T08:34:04.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.644+0000 7f11fbc00640 1 --2- 172.21.15.43:0/1287952920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11f4109870 0x7f11f4110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:04.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.644+0000 7f11fbc00640 1 --2- 172.21.15.43:0/1287952920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11f4108eb0 0x7f11f4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:04.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.644+0000 7f11fbc00640 1 --2- 172.21.15.43:0/1287952920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11f4107cc0 0x7f11f41080e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:04.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.644+0000 7f11fbc00640 1 -- 172.21.15.43:0/1287952920 >> 172.21.15.43:0/1287952920 conn(0x7f11f4076c60 msgr2=0x7f11f40790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:04.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.644+0000 7f11fbc00640 1 -- 172.21.15.43:0/1287952920 shutdown_connections 2024-03-26T08:34:04.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.644+0000 7f11fbc00640 1 -- 172.21.15.43:0/1287952920 wait complete. 2024-03-26T08:34:04.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.645+0000 7f11fbc00640 1 Processor -- start 2024-03-26T08:34:04.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.645+0000 7f11fbc00640 1 -- start start 2024-03-26T08:34:04.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.645+0000 7f11fbc00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11f4107cc0 0x7f11f419de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:04.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.645+0000 7f11fbc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11f4108eb0 0x7f11f419e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:04.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.645+0000 7f11fbc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11f4109870 0x7f11f41a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:04.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.645+0000 7f11fbc00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f4113810 con 0x7f11f4109870 2024-03-26T08:34:04.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.645+0000 7f11fbc00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f4113690 con 0x7f11f4107cc0 2024-03-26T08:34:04.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.645+0000 7f11fbc00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f4113990 con 0x7f11f4108eb0 2024-03-26T08:34:04.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.646+0000 7f11f9c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11f4109870 0x7f11f41a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:04.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.646+0000 7f11f9c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11f4109870 0x7f11f41a5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36826/0 (socket says 172.21.15.43:36826) 2024-03-26T08:34:04.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.646+0000 7f11f9c00640 1 -- 172.21.15.43:0/3767876085 learned_addr learned my addr 172.21.15.43:0/3767876085 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:34:04.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.646+0000 7f11f9200640 1 --2- 172.21.15.43:0/3767876085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11f4107cc0 0x7f11f419de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:04.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.646+0000 7f11f9c00640 1 -- 172.21.15.43:0/3767876085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11f4108eb0 msgr2=0x7f11f419e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:04.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.646+0000 7f11f9c00640 1 --2- 172.21.15.43:0/3767876085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11f4108eb0 0x7f11f419e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:04.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.646+0000 7f11f9c00640 1 -- 172.21.15.43:0/3767876085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11f4107cc0 msgr2=0x7f11f419de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:04.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.646+0000 7f11f9c00640 1 --2- 172.21.15.43:0/3767876085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11f4107cc0 0x7f11f419de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:04.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.646+0000 7f11f9c00640 1 -- 172.21.15.43:0/3767876085 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f41a5ab0 con 0x7f11f4109870 2024-03-26T08:34:04.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.646+0000 7f11f9c00640 1 --2- 172.21.15.43:0/3767876085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11f4109870 0x7f11f41a5440 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f11ec009fd0 tx=0x7f11ec00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:04.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.647+0000 7f11f1600640 1 -- 172.21.15.43:0/3767876085 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec017070 con 0x7f11f4109870 2024-03-26T08:34:04.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.647+0000 7f11f1600640 1 -- 172.21.15.43:0/3767876085 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f11ec003e20 con 0x7f11f4109870 2024-03-26T08:34:04.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.647+0000 7f11f1600640 1 -- 172.21.15.43:0/3767876085 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec010460 con 0x7f11f4109870 2024-03-26T08:34:04.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.647+0000 7f11fbc00640 1 -- 172.21.15.43:0/3767876085 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f41a5da0 con 0x7f11f4109870 2024-03-26T08:34:04.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.647+0000 7f11fbc00640 1 -- 172.21.15.43:0/3767876085 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f41a6270 con 0x7f11f4109870 2024-03-26T08:34:04.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.648+0000 7f11fbc00640 1 -- 172.21.15.43:0/3767876085 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f410efe0 con 0x7f11f4109870 2024-03-26T08:34:04.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.650+0000 7f11f1600640 1 -- 172.21.15.43:0/3767876085 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f11ec003970 con 0x7f11f4109870 2024-03-26T08:34:04.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.652+0000 7f11f1600640 1 --2- 172.21.15.43:0/3767876085 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f11dc0a2800 0x7f11dc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:04.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.652+0000 7f11f1600640 1 -- 172.21.15.43:0/3767876085 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f11ec0ced40 con 0x7f11f4109870 2024-03-26T08:34:04.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.652+0000 7f11f9200640 1 --2- 172.21.15.43:0/3767876085 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f11dc0a2800 0x7f11dc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:04.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.653+0000 7f11f9200640 1 --2- 172.21.15.43:0/3767876085 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f11dc0a2800 0x7f11dc0a4cc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f11e802c240 tx=0x7f11e8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:04.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.655+0000 7f11f1600640 1 -- 172.21.15.43:0/3767876085 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f11ec09ba00 con 0x7f11f4109870 2024-03-26T08:34:04.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.858+0000 7f11fbc00640 1 -- 172.21.15.43:0/3767876085 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f11f402d2c0 con 0x7f11f4109870 2024-03-26T08:34:04.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.859+0000 7f11f1600640 1 -- 172.21.15.43:0/3767876085 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f11ec09b300 con 0x7f11f4109870 2024-03-26T08:34:04.852 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-26T08:34:04.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.862+0000 7f11fbc00640 1 -- 172.21.15.43:0/3767876085 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f11dc0a2800 msgr2=0x7f11dc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:04.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.862+0000 7f11fbc00640 1 --2- 172.21.15.43:0/3767876085 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f11dc0a2800 0x7f11dc0a4cc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f11e802c240 tx=0x7f11e8005fd0 comp rx=0 tx=0).stop 2024-03-26T08:34:04.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.862+0000 7f11fbc00640 1 -- 172.21.15.43:0/3767876085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11f4109870 msgr2=0x7f11f41a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:04.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.862+0000 7f11fbc00640 1 --2- 172.21.15.43:0/3767876085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11f4109870 0x7f11f41a5440 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f11ec009fd0 tx=0x7f11ec00b260 comp rx=0 tx=0).stop 2024-03-26T08:34:04.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.862+0000 7f11fbc00640 1 -- 172.21.15.43:0/3767876085 shutdown_connections 2024-03-26T08:34:04.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.862+0000 7f11fbc00640 1 --2- 172.21.15.43:0/3767876085 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f11dc0a2800 0x7f11dc0a4cc0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:04.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.862+0000 7f11fbc00640 1 --2- 172.21.15.43:0/3767876085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11f4109870 0x7f11f41a5440 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:04.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.862+0000 7f11fbc00640 1 --2- 172.21.15.43:0/3767876085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11f4108eb0 0x7f11f419e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:04.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.862+0000 7f11fbc00640 1 --2- 172.21.15.43:0/3767876085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11f4107cc0 0x7f11f419de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:04.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.862+0000 7f11fbc00640 1 -- 172.21.15.43:0/3767876085 >> 172.21.15.43:0/3767876085 conn(0x7f11f4076c60 msgr2=0x7f11f410c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:04.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.862+0000 7f11fbc00640 1 -- 172.21.15.43:0/3767876085 shutdown_connections 2024-03-26T08:34:04.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:04.862+0000 7f11fbc00640 1 -- 172.21.15.43:0/3767876085 wait complete. 2024-03-26T08:34:04.875 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T08:34:05.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.116+0000 7fe8ff000640 1 -- 172.21.15.43:0/2872684345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8f8103150 msgr2=0x7fe8f81099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:05.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.116+0000 7fe8ff000640 1 --2- 172.21.15.43:0/2872684345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8f8103150 0x7fe8f81099e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0009860 tx=0x7fe8f002bb70 comp rx=0 tx=0).stop 2024-03-26T08:34:05.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.116+0000 7fe8ff000640 1 -- 172.21.15.43:0/2872684345 shutdown_connections 2024-03-26T08:34:05.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.116+0000 7fe8ff000640 1 --2- 172.21.15.43:0/2872684345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8f8103150 0x7fe8f81099e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:05.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.116+0000 7fe8ff000640 1 --2- 172.21.15.43:0/2872684345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8f8102790 0x7fe8f8102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:05.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.116+0000 7fe8ff000640 1 --2- 172.21.15.43:0/2872684345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8f81015a0 0x7fe8f81019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:05.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.116+0000 7fe8ff000640 1 -- 172.21.15.43:0/2872684345 >> 172.21.15.43:0/2872684345 conn(0x7fe8f80fcb10 msgr2=0x7fe8f80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:05.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.117+0000 7fe8ff000640 1 -- 172.21.15.43:0/2872684345 shutdown_connections 2024-03-26T08:34:05.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.117+0000 7fe8ff000640 1 -- 172.21.15.43:0/2872684345 wait complete. 2024-03-26T08:34:05.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.117+0000 7fe8ff000640 1 Processor -- start 2024-03-26T08:34:05.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.118+0000 7fe8ff000640 1 -- start start 2024-03-26T08:34:05.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.118+0000 7fe8ff000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8f81015a0 0x7fe8f8197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:05.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.118+0000 7fe8ff000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8f8102790 0x7fe8f8197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:05.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.118+0000 7fe8ff000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8f8103150 0x7fe8f819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:05.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.118+0000 7fe8ff000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f810d0e0 con 0x7fe8f8103150 2024-03-26T08:34:05.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.118+0000 7fe8ff000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f810cf60 con 0x7fe8f8102790 2024-03-26T08:34:05.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.118+0000 7fe8ff000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f810d260 con 0x7fe8f81015a0 2024-03-26T08:34:05.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.118+0000 7fe8fe600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8f8103150 0x7fe8f819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:05.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.118+0000 7fe8fe600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8f8103150 0x7fe8f819ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36848/0 (socket says 172.21.15.43:36848) 2024-03-26T08:34:05.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.118+0000 7fe8fe600640 1 -- 172.21.15.43:0/1673138309 learned_addr learned my addr 172.21.15.43:0/1673138309 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:34:05.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.118+0000 7fe8fdc00640 1 --2- 172.21.15.43:0/1673138309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8f81015a0 0x7fe8f8197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:05.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.119+0000 7fe8fd200640 1 --2- 172.21.15.43:0/1673138309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8f8102790 0x7fe8f8197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:05.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.119+0000 7fe8fe600640 1 -- 172.21.15.43:0/1673138309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8f81015a0 msgr2=0x7fe8f8197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:05.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.119+0000 7fe8fe600640 1 --2- 172.21.15.43:0/1673138309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8f81015a0 0x7fe8f8197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:05.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.119+0000 7fe8fe600640 1 -- 172.21.15.43:0/1673138309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8f8102790 msgr2=0x7fe8f8197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:05.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.119+0000 7fe8fe600640 1 --2- 172.21.15.43:0/1673138309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8f8102790 0x7fe8f8197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:05.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.119+0000 7fe8fe600640 1 -- 172.21.15.43:0/1673138309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f819f350 con 0x7fe8f8103150 2024-03-26T08:34:05.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.119+0000 7fe8fe600640 1 --2- 172.21.15.43:0/1673138309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8f8103150 0x7fe8f819ece0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe8f002c180 tx=0x7fe8f0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:05.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.119+0000 7fe8f6000640 1 -- 172.21.15.43:0/1673138309 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0041020 con 0x7fe8f8103150 2024-03-26T08:34:05.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.119+0000 7fe8f6000640 1 -- 172.21.15.43:0/1673138309 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe8f002c9c0 con 0x7fe8f8103150 2024-03-26T08:34:05.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.119+0000 7fe8ff000640 1 -- 172.21.15.43:0/1673138309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f819f5e0 con 0x7fe8f8103150 2024-03-26T08:34:05.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.119+0000 7fe8f6000640 1 -- 172.21.15.43:0/1673138309 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0039040 con 0x7fe8f8103150 2024-03-26T08:34:05.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.119+0000 7fe8ff000640 1 -- 172.21.15.43:0/1673138309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f819fad0 con 0x7fe8f8103150 2024-03-26T08:34:05.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.121+0000 7fe8ff000640 1 -- 172.21.15.43:0/1673138309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8f81088c0 con 0x7fe8f8103150 2024-03-26T08:34:05.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.123+0000 7fe8f6000640 1 -- 172.21.15.43:0/1673138309 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fe8f0044020 con 0x7fe8f8103150 2024-03-26T08:34:05.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.124+0000 7fe8f6000640 1 --2- 172.21.15.43:0/1673138309 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fe8e00a2800 0x7fe8e00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:05.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.124+0000 7fe8f6000640 1 -- 172.21.15.43:0/1673138309 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7fe8f00f09c0 con 0x7fe8f8103150 2024-03-26T08:34:05.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.124+0000 7fe8fdc00640 1 --2- 172.21.15.43:0/1673138309 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fe8e00a2800 0x7fe8e00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:05.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.125+0000 7fe8fdc00640 1 --2- 172.21.15.43:0/1673138309 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fe8e00a2800 0x7fe8e00a4cc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe8f81025f0 tx=0x7fe8ec0095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:05.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.128+0000 7fe8f6000640 1 -- 172.21.15.43:0/1673138309 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe8f004b020 con 0x7fe8f8103150 2024-03-26T08:34:05.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:05 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2542195244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:34:05.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:05 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3767876085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:34:05.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.337+0000 7fe8ff000640 1 -- 172.21.15.43:0/1673138309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe8f80622a0 con 0x7fe8f8103150 2024-03-26T08:34:05.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.338+0000 7fe8f6000640 1 -- 172.21.15.43:0/1673138309 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe8f00bd680 con 0x7fe8f8103150 2024-03-26T08:34:05.331 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-26T08:34:05.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.341+0000 7fe8ff000640 1 -- 172.21.15.43:0/1673138309 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fe8e00a2800 msgr2=0x7fe8e00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:05.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.341+0000 7fe8ff000640 1 --2- 172.21.15.43:0/1673138309 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fe8e00a2800 0x7fe8e00a4cc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe8f81025f0 tx=0x7fe8ec0095c0 comp rx=0 tx=0).stop 2024-03-26T08:34:05.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.341+0000 7fe8ff000640 1 -- 172.21.15.43:0/1673138309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8f8103150 msgr2=0x7fe8f819ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:05.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.341+0000 7fe8ff000640 1 --2- 172.21.15.43:0/1673138309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8f8103150 0x7fe8f819ece0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe8f002c180 tx=0x7fe8f0004270 comp rx=0 tx=0).stop 2024-03-26T08:34:05.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.341+0000 7fe8ff000640 1 -- 172.21.15.43:0/1673138309 shutdown_connections 2024-03-26T08:34:05.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.341+0000 7fe8ff000640 1 --2- 172.21.15.43:0/1673138309 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fe8e00a2800 0x7fe8e00a4cc0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:05.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.341+0000 7fe8ff000640 1 --2- 172.21.15.43:0/1673138309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8f8103150 0x7fe8f819ece0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:05.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.341+0000 7fe8ff000640 1 --2- 172.21.15.43:0/1673138309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8f8102790 0x7fe8f8197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:05.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.341+0000 7fe8ff000640 1 --2- 172.21.15.43:0/1673138309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8f81015a0 0x7fe8f8197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:05.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.341+0000 7fe8ff000640 1 -- 172.21.15.43:0/1673138309 >> 172.21.15.43:0/1673138309 conn(0x7fe8f80fcb10 msgr2=0x7fe8f8105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:05.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.342+0000 7fe8ff000640 1 -- 172.21.15.43:0/1673138309 shutdown_connections 2024-03-26T08:34:05.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.342+0000 7fe8ff000640 1 -- 172.21.15.43:0/1673138309 wait complete. 2024-03-26T08:34:05.354 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T08:34:05.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:05 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2542195244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:34:05.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:05 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3767876085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:34:05.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:05 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2542195244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:34:05.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:05 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3767876085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:34:05.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.590+0000 7fb813e00640 1 -- 172.21.15.43:0/2730040625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb80c104a70 msgr2=0x7fb80c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:05.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.590+0000 7fb813e00640 1 --2- 172.21.15.43:0/2730040625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb80c104a70 0x7fb80c106e60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8009900 tx=0x7fb7f802bc40 comp rx=0 tx=0).stop 2024-03-26T08:34:05.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.590+0000 7fb813e00640 1 -- 172.21.15.43:0/2730040625 shutdown_connections 2024-03-26T08:34:05.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.590+0000 7fb813e00640 1 --2- 172.21.15.43:0/2730040625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb80c1073a0 0x7fb80c1097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:05.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.590+0000 7fb813e00640 1 --2- 172.21.15.43:0/2730040625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb80c104a70 0x7fb80c106e60 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:05.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.590+0000 7fb813e00640 1 --2- 172.21.15.43:0/2730040625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb80c102140 0x7fb80c104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:05.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.590+0000 7fb813e00640 1 -- 172.21.15.43:0/2730040625 >> 172.21.15.43:0/2730040625 conn(0x7fb80c0fbb30 msgr2=0x7fb80c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:05.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.590+0000 7fb813e00640 1 -- 172.21.15.43:0/2730040625 shutdown_connections 2024-03-26T08:34:05.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.590+0000 7fb813e00640 1 -- 172.21.15.43:0/2730040625 wait complete. 2024-03-26T08:34:05.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.591+0000 7fb813e00640 1 Processor -- start 2024-03-26T08:34:05.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.591+0000 7fb813e00640 1 -- start start 2024-03-26T08:34:05.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.592+0000 7fb813e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb80c102140 0x7fb80c101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:05.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.592+0000 7fb813e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb80c104a70 0x7fb80c1000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:05.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.592+0000 7fb813e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb80c1073a0 0x7fb80c100630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:05.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.592+0000 7fb813e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c10d1f0 con 0x7fb80c104a70 2024-03-26T08:34:05.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.592+0000 7fb813e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c10d070 con 0x7fb80c102140 2024-03-26T08:34:05.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.592+0000 7fb813e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c10d370 con 0x7fb80c1073a0 2024-03-26T08:34:05.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.592+0000 7fb810a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb80c104a70 0x7fb80c1000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:05.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.592+0000 7fb810a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb80c104a70 0x7fb80c1000f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36860/0 (socket says 172.21.15.43:36860) 2024-03-26T08:34:05.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.592+0000 7fb810a00640 1 -- 172.21.15.43:0/1031179500 learned_addr learned my addr 172.21.15.43:0/1031179500 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:34:05.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.592+0000 7fb811400640 1 --2- 172.21.15.43:0/1031179500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb80c102140 0x7fb80c101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:05.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.593+0000 7fb811e00640 1 --2- 172.21.15.43:0/1031179500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb80c1073a0 0x7fb80c100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:05.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.593+0000 7fb810a00640 1 -- 172.21.15.43:0/1031179500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb80c1073a0 msgr2=0x7fb80c100630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:05.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.593+0000 7fb810a00640 1 --2- 172.21.15.43:0/1031179500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb80c1073a0 0x7fb80c100630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:05.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.593+0000 7fb810a00640 1 -- 172.21.15.43:0/1031179500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb80c102140 msgr2=0x7fb80c101ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:05.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.593+0000 7fb810a00640 1 --2- 172.21.15.43:0/1031179500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb80c102140 0x7fb80c101ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:05.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.593+0000 7fb810a00640 1 -- 172.21.15.43:0/1031179500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80c100ec0 con 0x7fb80c104a70 2024-03-26T08:34:05.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.593+0000 7fb810a00640 1 --2- 172.21.15.43:0/1031179500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb80c104a70 0x7fb80c1000f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8036040 tx=0x7fb7f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:05.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.594+0000 7fb80a000640 1 -- 172.21.15.43:0/1031179500 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8043020 con 0x7fb80c104a70 2024-03-26T08:34:05.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.594+0000 7fb80a000640 1 -- 172.21.15.43:0/1031179500 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb7f8004aa0 con 0x7fb80c104a70 2024-03-26T08:34:05.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.594+0000 7fb813e00640 1 -- 172.21.15.43:0/1031179500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb80c1a1a10 con 0x7fb80c104a70 2024-03-26T08:34:05.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.594+0000 7fb80a000640 1 -- 172.21.15.43:0/1031179500 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8003ce0 con 0x7fb80c104a70 2024-03-26T08:34:05.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.594+0000 7fb813e00640 1 -- 172.21.15.43:0/1031179500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb80c1a1f60 con 0x7fb80c104a70 2024-03-26T08:34:05.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.595+0000 7fb813e00640 1 -- 172.21.15.43:0/1031179500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb80c0fd620 con 0x7fb80c104a70 2024-03-26T08:34:05.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.597+0000 7fb80a000640 1 -- 172.21.15.43:0/1031179500 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb7f8046020 con 0x7fb80c104a70 2024-03-26T08:34:05.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.598+0000 7fb80a000640 1 --2- 172.21.15.43:0/1031179500 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb7f40a2800 0x7fb7f40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:05.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.599+0000 7fb80a000640 1 -- 172.21.15.43:0/1031179500 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7fb7f80f11c0 con 0x7fb80c104a70 2024-03-26T08:34:05.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.599+0000 7fb811400640 1 --2- 172.21.15.43:0/1031179500 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb7f40a2800 0x7fb7f40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:05.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.600+0000 7fb811400640 1 --2- 172.21.15.43:0/1031179500 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb7f40a2800 0x7fb7f40a4cc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb80000abe0 tx=0x7fb80000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:05.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.603+0000 7fb80a000640 1 -- 172.21.15.43:0/1031179500 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb7f804d020 con 0x7fb80c104a70 2024-03-26T08:34:05.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.805+0000 7fb813e00640 1 -- 172.21.15.43:0/1031179500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb80c0622a0 con 0x7fb80c104a70 2024-03-26T08:34:05.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.806+0000 7fb80a000640 1 -- 172.21.15.43:0/1031179500 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb7f80bdfa0 con 0x7fb80c104a70 2024-03-26T08:34:05.798 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-26T08:34:05.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.808+0000 7fb813e00640 1 -- 172.21.15.43:0/1031179500 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb7f40a2800 msgr2=0x7fb7f40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:05.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.808+0000 7fb813e00640 1 --2- 172.21.15.43:0/1031179500 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb7f40a2800 0x7fb7f40a4cc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb80000abe0 tx=0x7fb80000a2d0 comp rx=0 tx=0).stop 2024-03-26T08:34:05.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.808+0000 7fb813e00640 1 -- 172.21.15.43:0/1031179500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb80c104a70 msgr2=0x7fb80c1000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:05.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.808+0000 7fb813e00640 1 --2- 172.21.15.43:0/1031179500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb80c104a70 0x7fb80c1000f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8036040 tx=0x7fb7f80042f0 comp rx=0 tx=0).stop 2024-03-26T08:34:05.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.809+0000 7fb813e00640 1 -- 172.21.15.43:0/1031179500 shutdown_connections 2024-03-26T08:34:05.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.809+0000 7fb813e00640 1 --2- 172.21.15.43:0/1031179500 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb7f40a2800 0x7fb7f40a4cc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:05.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.809+0000 7fb813e00640 1 --2- 172.21.15.43:0/1031179500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb80c1073a0 0x7fb80c100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:05.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.809+0000 7fb813e00640 1 --2- 172.21.15.43:0/1031179500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb80c104a70 0x7fb80c1000f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:05.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.809+0000 7fb813e00640 1 --2- 172.21.15.43:0/1031179500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb80c102140 0x7fb80c101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:05.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.809+0000 7fb813e00640 1 -- 172.21.15.43:0/1031179500 >> 172.21.15.43:0/1031179500 conn(0x7fb80c0fbb30 msgr2=0x7fb80c102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:05.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.809+0000 7fb813e00640 1 -- 172.21.15.43:0/1031179500 shutdown_connections 2024-03-26T08:34:05.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:05.809+0000 7fb813e00640 1 -- 172.21.15.43:0/1031179500 wait complete. 2024-03-26T08:34:05.821 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T08:34:06.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.024+0000 7fd8f9c00640 1 -- 172.21.15.43:0/33092146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f4105370 msgr2=0x7fd8f410bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.024+0000 7fd8f9c00640 1 --2- 172.21.15.43:0/33092146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f4105370 0x7fd8f410bc00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec009900 tx=0x7fd8ec02bc30 comp rx=0 tx=0).stop 2024-03-26T08:34:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.024+0000 7fd8f9c00640 1 -- 172.21.15.43:0/33092146 shutdown_connections 2024-03-26T08:34:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.024+0000 7fd8f9c00640 1 --2- 172.21.15.43:0/33092146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f4105370 0x7fd8f410bc00 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.024+0000 7fd8f9c00640 1 --2- 172.21.15.43:0/33092146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8f41049b0 0x7fd8f4104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.024+0000 7fd8f9c00640 1 --2- 172.21.15.43:0/33092146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8f41037c0 0x7fd8f4103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.024+0000 7fd8f9c00640 1 -- 172.21.15.43:0/33092146 >> 172.21.15.43:0/33092146 conn(0x7fd8f40fed50 msgr2=0x7fd8f4101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.024+0000 7fd8f9c00640 1 -- 172.21.15.43:0/33092146 shutdown_connections 2024-03-26T08:34:06.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.024+0000 7fd8f9c00640 1 -- 172.21.15.43:0/33092146 wait complete. 2024-03-26T08:34:06.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.026+0000 7fd8f9c00640 1 Processor -- start 2024-03-26T08:34:06.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.026+0000 7fd8f9c00640 1 -- start start 2024-03-26T08:34:06.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.026+0000 7fd8f9c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f41037c0 0x7fd8f4199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:06.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.026+0000 7fd8f9c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8f41049b0 0x7fd8f4199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:06.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.026+0000 7fd8f9c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8f4105370 0x7fd8f41a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:06.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.026+0000 7fd8f9c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f410f340 con 0x7fd8f41037c0 2024-03-26T08:34:06.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.026+0000 7fd8f9c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f410f1c0 con 0x7fd8f4105370 2024-03-26T08:34:06.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.026+0000 7fd8f9c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f410f4c0 con 0x7fd8f41049b0 2024-03-26T08:34:06.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.027+0000 7fd8f3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f41037c0 0x7fd8f4199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:06.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.027+0000 7fd8f3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f41037c0 0x7fd8f4199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36884/0 (socket says 172.21.15.43:36884) 2024-03-26T08:34:06.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.027+0000 7fd8f3e00640 1 -- 172.21.15.43:0/1755752333 learned_addr learned my addr 172.21.15.43:0/1755752333 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:34:06.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.027+0000 7fd8f3e00640 1 -- 172.21.15.43:0/1755752333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8f41049b0 msgr2=0x7fd8f4199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:06.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.027+0000 7fd8f3e00640 1 --2- 172.21.15.43:0/1755752333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8f41049b0 0x7fd8f4199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.027+0000 7fd8f3e00640 1 -- 172.21.15.43:0/1755752333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8f4105370 msgr2=0x7fd8f41a0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:06.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.027+0000 7fd8f3e00640 1 --2- 172.21.15.43:0/1755752333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8f4105370 0x7fd8f41a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.027+0000 7fd8f3e00640 1 -- 172.21.15.43:0/1755752333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f41a1580 con 0x7fd8f41037c0 2024-03-26T08:34:06.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.027+0000 7fd8f3e00640 1 --2- 172.21.15.43:0/1755752333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f41037c0 0x7fd8f4199960 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400ebc0 tx=0x7fd8e400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:06.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.028+0000 7fd8f0c00640 1 -- 172.21.15.43:0/1755752333 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4003b70 con 0x7fd8f41037c0 2024-03-26T08:34:06.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.028+0000 7fd8f0c00640 1 -- 172.21.15.43:0/1755752333 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd8e4003d10 con 0x7fd8f41037c0 2024-03-26T08:34:06.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.028+0000 7fd8f0c00640 1 -- 172.21.15.43:0/1755752333 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4019840 con 0x7fd8f41037c0 2024-03-26T08:34:06.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.028+0000 7fd8f9c00640 1 -- 172.21.15.43:0/1755752333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f41a1870 con 0x7fd8f41037c0 2024-03-26T08:34:06.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.028+0000 7fd8f9c00640 1 -- 172.21.15.43:0/1755752333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f41a1dc0 con 0x7fd8f41037c0 2024-03-26T08:34:06.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.029+0000 7fd8f9c00640 1 -- 172.21.15.43:0/1755752333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f410aae0 con 0x7fd8f41037c0 2024-03-26T08:34:06.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.031+0000 7fd8f0c00640 1 -- 172.21.15.43:0/1755752333 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fd8e40199e0 con 0x7fd8f41037c0 2024-03-26T08:34:06.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.034+0000 7fd8f0c00640 1 --2- 172.21.15.43:0/1755752333 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fd8d80a2800 0x7fd8d80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:06.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.034+0000 7fd8f0c00640 1 -- 172.21.15.43:0/1755752333 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7fd8e40cf8c0 con 0x7fd8f41037c0 2024-03-26T08:34:06.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.034+0000 7fd8f3400640 1 --2- 172.21.15.43:0/1755752333 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fd8d80a2800 0x7fd8d80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:06.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.035+0000 7fd8f3400640 1 --2- 172.21.15.43:0/1755752333 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fd8d80a2800 0x7fd8d80a4cc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc005fd0 tx=0x7fd8dc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:06.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.035+0000 7fd8f0c00640 1 -- 172.21.15.43:0/1755752333 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd8e409c670 con 0x7fd8f41037c0 2024-03-26T08:34:06.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.241+0000 7fd8f9c00640 1 -- 172.21.15.43:0/1755752333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd8f402d320 con 0x7fd8f41037c0 2024-03-26T08:34:06.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.242+0000 7fd8f0c00640 1 -- 172.21.15.43:0/1755752333 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd8e409bf70 con 0x7fd8f41037c0 2024-03-26T08:34:06.234 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-26T08:34:06.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.244+0000 7fd8f9c00640 1 -- 172.21.15.43:0/1755752333 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fd8d80a2800 msgr2=0x7fd8d80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:06.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.244+0000 7fd8f9c00640 1 --2- 172.21.15.43:0/1755752333 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fd8d80a2800 0x7fd8d80a4cc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc005fd0 tx=0x7fd8dc005850 comp rx=0 tx=0).stop 2024-03-26T08:34:06.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.244+0000 7fd8f9c00640 1 -- 172.21.15.43:0/1755752333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f41037c0 msgr2=0x7fd8f4199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:06.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.244+0000 7fd8f9c00640 1 --2- 172.21.15.43:0/1755752333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f41037c0 0x7fd8f4199960 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400ebc0 tx=0x7fd8e400c6a0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.244+0000 7fd8f9c00640 1 -- 172.21.15.43:0/1755752333 shutdown_connections 2024-03-26T08:34:06.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.244+0000 7fd8f9c00640 1 --2- 172.21.15.43:0/1755752333 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fd8d80a2800 0x7fd8d80a4cc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.244+0000 7fd8f9c00640 1 --2- 172.21.15.43:0/1755752333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8f4105370 0x7fd8f41a0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.244+0000 7fd8f9c00640 1 --2- 172.21.15.43:0/1755752333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8f41049b0 0x7fd8f4199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.244+0000 7fd8f9c00640 1 --2- 172.21.15.43:0/1755752333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8f41037c0 0x7fd8f4199960 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.244+0000 7fd8f9c00640 1 -- 172.21.15.43:0/1755752333 >> 172.21.15.43:0/1755752333 conn(0x7fd8f40fed50 msgr2=0x7fd8f4107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:06.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.245+0000 7fd8f9c00640 1 -- 172.21.15.43:0/1755752333 shutdown_connections 2024-03-26T08:34:06.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.245+0000 7fd8f9c00640 1 -- 172.21.15.43:0/1755752333 wait complete. 2024-03-26T08:34:06.256 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:34:06.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:06 smithi043 ceph-mon[31929]: pgmap v458: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:06.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:06 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1673138309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:34:06.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:06 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1031179500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:34:06.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.465+0000 7faf12c00640 1 -- 172.21.15.43:0/2329270599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf0c101500 msgr2=0x7faf0c101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:06.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.465+0000 7faf12c00640 1 --2- 172.21.15.43:0/2329270599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf0c101500 0x7faf0c101920 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7faf00009900 tx=0x7faf0002bc30 comp rx=0 tx=0).stop 2024-03-26T08:34:06.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.465+0000 7faf12c00640 1 -- 172.21.15.43:0/2329270599 shutdown_connections 2024-03-26T08:34:06.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.465+0000 7faf12c00640 1 --2- 172.21.15.43:0/2329270599 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf0c1030b0 0x7faf0c109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.465+0000 7faf12c00640 1 --2- 172.21.15.43:0/2329270599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf0c1026f0 0x7faf0c102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.465+0000 7faf12c00640 1 --2- 172.21.15.43:0/2329270599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf0c101500 0x7faf0c101920 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.465+0000 7faf12c00640 1 -- 172.21.15.43:0/2329270599 >> 172.21.15.43:0/2329270599 conn(0x7faf0c0fca70 msgr2=0x7faf0c0feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:06.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.465+0000 7faf12c00640 1 -- 172.21.15.43:0/2329270599 shutdown_connections 2024-03-26T08:34:06.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.466+0000 7faf12c00640 1 -- 172.21.15.43:0/2329270599 wait complete. 2024-03-26T08:34:06.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.466+0000 7faf12c00640 1 Processor -- start 2024-03-26T08:34:06.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.467+0000 7faf12c00640 1 -- start start 2024-03-26T08:34:06.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.467+0000 7faf12c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf0c101500 0x7faf0c199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:06.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.467+0000 7faf12c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf0c1026f0 0x7faf0c199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:06.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.467+0000 7faf12c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf0c1030b0 0x7faf0c1a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:06.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.467+0000 7faf12c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c10d090 con 0x7faf0c1026f0 2024-03-26T08:34:06.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.467+0000 7faf12c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c10cf10 con 0x7faf0c1030b0 2024-03-26T08:34:06.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.467+0000 7faf12c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c10d210 con 0x7faf0c101500 2024-03-26T08:34:06.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.467+0000 7faf10e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf0c1026f0 0x7faf0c199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:06.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.467+0000 7faf10e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf0c1026f0 0x7faf0c199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36900/0 (socket says 172.21.15.43:36900) 2024-03-26T08:34:06.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.467+0000 7faf10e00640 1 -- 172.21.15.43:0/1987716018 learned_addr learned my addr 172.21.15.43:0/1987716018 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:34:06.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.468+0000 7faf11800640 1 --2- 172.21.15.43:0/1987716018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf0c101500 0x7faf0c199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:06.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.468+0000 7faf12200640 1 --2- 172.21.15.43:0/1987716018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf0c1030b0 0x7faf0c1a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:06.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.468+0000 7faf10e00640 1 -- 172.21.15.43:0/1987716018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf0c101500 msgr2=0x7faf0c199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:06.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.468+0000 7faf10e00640 1 --2- 172.21.15.43:0/1987716018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf0c101500 0x7faf0c199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.468+0000 7faf10e00640 1 -- 172.21.15.43:0/1987716018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf0c1030b0 msgr2=0x7faf0c1a0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:06.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.468+0000 7faf10e00640 1 --2- 172.21.15.43:0/1987716018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf0c1030b0 0x7faf0c1a0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.468+0000 7faf10e00640 1 -- 172.21.15.43:0/1987716018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf0c1a1580 con 0x7faf0c1026f0 2024-03-26T08:34:06.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.468+0000 7faf10e00640 1 --2- 172.21.15.43:0/1987716018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf0c1026f0 0x7faf0c199ea0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7faef800e9f0 tx=0x7faef800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:06.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.468+0000 7faf0a000640 1 -- 172.21.15.43:0/1987716018 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef800cd80 con 0x7faf0c1026f0 2024-03-26T08:34:06.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.468+0000 7faf0a000640 1 -- 172.21.15.43:0/1987716018 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faef80054d0 con 0x7faf0c1026f0 2024-03-26T08:34:06.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.468+0000 7faf12c00640 1 -- 172.21.15.43:0/1987716018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf0c1a1870 con 0x7faf0c1026f0 2024-03-26T08:34:06.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.468+0000 7faf0a000640 1 -- 172.21.15.43:0/1987716018 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef8010490 con 0x7faf0c1026f0 2024-03-26T08:34:06.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.468+0000 7faf12c00640 1 -- 172.21.15.43:0/1987716018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf0c1a1d70 con 0x7faf0c1026f0 2024-03-26T08:34:06.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.470+0000 7faf12c00640 1 -- 172.21.15.43:0/1987716018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf0c1a2030 con 0x7faf0c1026f0 2024-03-26T08:34:06.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.472+0000 7faf0a000640 1 -- 172.21.15.43:0/1987716018 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7faef8005020 con 0x7faf0c1026f0 2024-03-26T08:34:06.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.473+0000 7faf0a000640 1 --2- 172.21.15.43:0/1987716018 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7faef40a2800 0x7faef40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:06.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.474+0000 7faf0a000640 1 -- 172.21.15.43:0/1987716018 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7faef80cf320 con 0x7faf0c1026f0 2024-03-26T08:34:06.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.474+0000 7faf11800640 1 --2- 172.21.15.43:0/1987716018 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7faef40a2800 0x7faef40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:06.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.475+0000 7faf11800640 1 --2- 172.21.15.43:0/1987716018 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7faef40a2800 0x7faef40a4cc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7faf0002c240 tx=0x7faf00005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:06.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.478+0000 7faf0a000640 1 -- 172.21.15.43:0/1987716018 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faef809bfe0 con 0x7faf0c1026f0 2024-03-26T08:34:06.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:06 smithi183 ceph-mon[35818]: pgmap v458: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:06.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:06 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1673138309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:34:06.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:06 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1031179500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:34:06.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:06 smithi052 ceph-mon[35071]: pgmap v458: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:06.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:06 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1673138309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:34:06.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:06 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1031179500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:34:06.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.723+0000 7faf12c00640 1 -- 172.21.15.43:0/1987716018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf0c1a2200 con 0x7faf0c1026f0 2024-03-26T08:34:06.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.725+0000 7faf0a000640 1 -- 172.21.15.43:0/1987716018 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11331 (secure 0 0 0) 0x7faef809b8e0 con 0x7faf0c1026f0 2024-03-26T08:34:06.718 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:34:06.718 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1452206807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1452206807},{"type":"v1","addr":"172.21.15.43:6835","nonce":1452206807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3151775467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3151775467},{"type":"v1","addr":"172.21.15.43:6837","nonce":3151775467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/1284655682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":1284655682},{"type":"v1","addr":"172.21.15.43:6839","nonce":1284655682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3081085787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3081085787},{"type":"v1","addr":"172.21.15.43:6841","nonce":3081085787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1842295306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1842295306},{"type":"v1","addr":"172.21.15.183:6835","nonce":1842295306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2267781666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2267781666},{"type":"v1","addr":"172.21.15.183:6837","nonce":2267781666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3572429948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3572429948},{"type":"v1","addr":"172.21.15.183:6839","nonce":3572429948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34448,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/1108327179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108327179},{"type":"v1","addr":"172.21.15.52:6833","nonce":1108327179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34487,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/742057410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":742057410},{"type":"v1","addr":"172.21.15.52:6835","nonce":742057410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:31:21.819689+0000","modified":"2024-03-26T08:31:48.080233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34517":{"gid":34517,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/748239918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":748239918},{"type":"v1","addr":"172.21.15.52:6837","nonce":748239918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34559":{"gid":34559,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.52:6839/2551965101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2551965101},{"type":"v1","addr":"172.21.15.52:6839","nonce":2551965101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34559,"qdb_cluster":[34559]},"id":1}]} 2024-03-26T08:34:06.718 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 21 2024-03-26T08:34:06.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.727+0000 7faf12c00640 1 -- 172.21.15.43:0/1987716018 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7faef40a2800 msgr2=0x7faef40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:06.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.728+0000 7faf12c00640 1 --2- 172.21.15.43:0/1987716018 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7faef40a2800 0x7faef40a4cc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7faf0002c240 tx=0x7faf00005fd0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.728+0000 7faf12c00640 1 -- 172.21.15.43:0/1987716018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf0c1026f0 msgr2=0x7faf0c199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:06.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.728+0000 7faf12c00640 1 --2- 172.21.15.43:0/1987716018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf0c1026f0 0x7faf0c199ea0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7faef800e9f0 tx=0x7faef800edf0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.728+0000 7faf12c00640 1 -- 172.21.15.43:0/1987716018 shutdown_connections 2024-03-26T08:34:06.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.728+0000 7faf12c00640 1 --2- 172.21.15.43:0/1987716018 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7faef40a2800 0x7faef40a4cc0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.728+0000 7faf12c00640 1 --2- 172.21.15.43:0/1987716018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf0c1030b0 0x7faf0c1a0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.728+0000 7faf12c00640 1 --2- 172.21.15.43:0/1987716018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf0c1026f0 0x7faf0c199ea0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.728+0000 7faf12c00640 1 --2- 172.21.15.43:0/1987716018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf0c101500 0x7faf0c199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.728+0000 7faf12c00640 1 -- 172.21.15.43:0/1987716018 >> 172.21.15.43:0/1987716018 conn(0x7faf0c0fca70 msgr2=0x7faf0c0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:06.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.728+0000 7faf12c00640 1 -- 172.21.15.43:0/1987716018 shutdown_connections 2024-03-26T08:34:06.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.728+0000 7faf12c00640 1 -- 172.21.15.43:0/1987716018 wait complete. 2024-03-26T08:34:06.739 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:34:06.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.942+0000 7fcb90c00640 1 -- 172.21.15.43:0/3713621483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb8c104ad0 msgr2=0x7fcb8c104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:06.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.942+0000 7fcb90c00640 1 --2- 172.21.15.43:0/3713621483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb8c104ad0 0x7fcb8c104f50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcb74009900 tx=0x7fcb7402bc30 comp rx=0 tx=0).stop 2024-03-26T08:34:06.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.942+0000 7fcb90c00640 1 -- 172.21.15.43:0/3713621483 shutdown_connections 2024-03-26T08:34:06.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.942+0000 7fcb90c00640 1 --2- 172.21.15.43:0/3713621483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb8c105490 0x7fcb8c10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.942+0000 7fcb90c00640 1 --2- 172.21.15.43:0/3713621483 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb8c104ad0 0x7fcb8c104f50 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.942+0000 7fcb90c00640 1 --2- 172.21.15.43:0/3713621483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb8c1038e0 0x7fcb8c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.942+0000 7fcb90c00640 1 -- 172.21.15.43:0/3713621483 >> 172.21.15.43:0/3713621483 conn(0x7fcb8c0fee50 msgr2=0x7fcb8c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:06.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.942+0000 7fcb90c00640 1 -- 172.21.15.43:0/3713621483 shutdown_connections 2024-03-26T08:34:06.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.942+0000 7fcb90c00640 1 -- 172.21.15.43:0/3713621483 wait complete. 2024-03-26T08:34:06.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.943+0000 7fcb90c00640 1 Processor -- start 2024-03-26T08:34:06.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.943+0000 7fcb90c00640 1 -- start start 2024-03-26T08:34:06.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.944+0000 7fcb90c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb8c1038e0 0x7fcb8c199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:06.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.944+0000 7fcb90c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb8c104ad0 0x7fcb8c19a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:06.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.944+0000 7fcb90c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb8c105490 0x7fcb8c1a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:06.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.944+0000 7fcb90c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c10f410 con 0x7fcb8c105490 2024-03-26T08:34:06.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.944+0000 7fcb90c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c10f290 con 0x7fcb8c104ad0 2024-03-26T08:34:06.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.944+0000 7fcb90c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c10f590 con 0x7fcb8c1038e0 2024-03-26T08:34:06.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.944+0000 7fcb8a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb8c105490 0x7fcb8c1a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:06.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.944+0000 7fcb8a800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb8c105490 0x7fcb8c1a10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36928/0 (socket says 172.21.15.43:36928) 2024-03-26T08:34:06.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.944+0000 7fcb8a800640 1 -- 172.21.15.43:0/4092666061 learned_addr learned my addr 172.21.15.43:0/4092666061 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:34:06.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.944+0000 7fcb89e00640 1 --2- 172.21.15.43:0/4092666061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb8c1038e0 0x7fcb8c199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:06.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.944+0000 7fcb83e00640 1 --2- 172.21.15.43:0/4092666061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb8c104ad0 0x7fcb8c19a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:06.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.945+0000 7fcb8a800640 1 -- 172.21.15.43:0/4092666061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb8c1038e0 msgr2=0x7fcb8c199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:06.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.945+0000 7fcb8a800640 1 --2- 172.21.15.43:0/4092666061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb8c1038e0 0x7fcb8c199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.945+0000 7fcb8a800640 1 -- 172.21.15.43:0/4092666061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb8c104ad0 msgr2=0x7fcb8c19a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:06.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.945+0000 7fcb8a800640 1 --2- 172.21.15.43:0/4092666061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb8c104ad0 0x7fcb8c19a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.945+0000 7fcb8a800640 1 -- 172.21.15.43:0/4092666061 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8c1a1730 con 0x7fcb8c105490 2024-03-26T08:34:06.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.945+0000 7fcb8a800640 1 --2- 172.21.15.43:0/4092666061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb8c105490 0x7fcb8c1a10c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400eaa0 tx=0x7fcb8400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:06.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.945+0000 7fcb82a00640 1 -- 172.21.15.43:0/4092666061 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8400cdb0 con 0x7fcb8c105490 2024-03-26T08:34:06.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.945+0000 7fcb82a00640 1 -- 172.21.15.43:0/4092666061 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcb84005020 con 0x7fcb8c105490 2024-03-26T08:34:06.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.945+0000 7fcb82a00640 1 -- 172.21.15.43:0/4092666061 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb840116d0 con 0x7fcb8c105490 2024-03-26T08:34:06.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.945+0000 7fcb90c00640 1 -- 172.21.15.43:0/4092666061 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8c1a1a20 con 0x7fcb8c105490 2024-03-26T08:34:06.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.945+0000 7fcb90c00640 1 -- 172.21.15.43:0/4092666061 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb8c1a1f70 con 0x7fcb8c105490 2024-03-26T08:34:06.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.947+0000 7fcb90c00640 1 -- 172.21.15.43:0/4092666061 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb8c10ac70 con 0x7fcb8c105490 2024-03-26T08:34:06.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.949+0000 7fcb82a00640 1 -- 172.21.15.43:0/4092666061 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fcb840051d0 con 0x7fcb8c105490 2024-03-26T08:34:06.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.950+0000 7fcb82a00640 1 --2- 172.21.15.43:0/4092666061 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fcb700a2880 0x7fcb700a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:06.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.950+0000 7fcb82a00640 1 -- 172.21.15.43:0/4092666061 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7fcb840cf060 con 0x7fcb8c105490 2024-03-26T08:34:06.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.950+0000 7fcb89e00640 1 --2- 172.21.15.43:0/4092666061 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fcb700a2880 0x7fcb700a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:06.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.951+0000 7fcb89e00640 1 --2- 172.21.15.43:0/4092666061 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fcb700a2880 0x7fcb700a4d40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c005fd0 tx=0x7fcb7c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:06.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:06.955+0000 7fcb82a00640 1 -- 172.21.15.43:0/4092666061 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcb84016030 con 0x7fcb8c105490 2024-03-26T08:34:07.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.160+0000 7fcb90c00640 1 -- 172.21.15.43:0/4092666061 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb8c062340 con 0x7fcb8c105490 2024-03-26T08:34:07.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.163+0000 7fcb82a00640 1 -- 172.21.15.43:0/4092666061 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24670 (secure 0 0 0) 0x7fcb8409bd20 con 0x7fcb8c105490 2024-03-26T08:34:07.156 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:34:07.156 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":94,"fsid":"88759420-eb49-11ee-95d0-87774f69a715","created":"2024-03-26T08:20:27.817107+0000","modified":"2024-03-26T08:32:38.256877+0000","last_up_change":"2024-03-26T08:30:10.520976+0000","last_in_change":"2024-03-26T08:29:46.394360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:24:54.852427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:31:19.924902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:31:21.323881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T08:31:24.951015+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"28670673-9193-4fa3-9cc0-c1a5a6b27284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6803","nonce":936601146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6805","nonce":936601146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6809","nonce":936601146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6807","nonce":936601146}]},"public_addr":"172.21.15.43:6803/936601146","cluster_addr":"172.21.15.43:6805/936601146","heartbeat_back_addr":"172.21.15.43:6809/936601146","heartbeat_front_addr":"172.21.15.43:6807/936601146","state":["exists","up"]},{"osd":1,"uuid":"fa8ad6b6-277a-42b8-b0f8-b36fc219de2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6801","nonce":3268412328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6803","nonce":3268412328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6807","nonce":3268412328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6805","nonce":3268412328}]},"public_addr":"172.21.15.52:6801/3268412328","cluster_addr":"172.21.15.52:6803/3268412328","heartbeat_back_addr":"172.21.15.52:6807/3268412328","heartbeat_front_addr":"172.21.15.52:6805/3268412328","state":["exists","up"]},{"osd":2,"uuid":"ec0039c3-f9bd-419a-81de-fa4418359808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6801","nonce":2103180908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6803","nonce":2103180908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6807","nonce":2103180908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6805","nonce":2103180908}]},"public_addr":"172.21.15.183:6801/2103180908","cluster_addr":"172.21.15.183:6803/2103180908","heartbeat_back_addr":"172.21.15.183:6807/2103180908","heartbeat_front_addr":"172.21.15.183:6805/2103180908","state":["exists","up"]},{"osd":3,"uuid":"e7dcbf00-6586-4664-a69e-3cb0df8dfd30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6811","nonce":3722846033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6813","nonce":3722846033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6817","nonce":3722846033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6815","nonce":3722846033}]},"public_addr":"172.21.15.43:6811/3722846033","cluster_addr":"172.21.15.43:6813/3722846033","heartbeat_back_addr":"172.21.15.43:6817/3722846033","heartbeat_front_addr":"172.21.15.43:6815/3722846033","state":["exists","up"]},{"osd":4,"uuid":"db089a49-5e98-432d-9687-cdc044114c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6809","nonce":4028436523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6811","nonce":4028436523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6815","nonce":4028436523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6813","nonce":4028436523}]},"public_addr":"172.21.15.52:6809/4028436523","cluster_addr":"172.21.15.52:6811/4028436523","heartbeat_back_addr":"172.21.15.52:6815/4028436523","heartbeat_front_addr":"172.21.15.52:6813/4028436523","state":["exists","up"]},{"osd":5,"uuid":"556ac804-02ea-423b-a599-55baf5be4ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6809","nonce":787132252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6811","nonce":787132252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6815","nonce":787132252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6813","nonce":787132252}]},"public_addr":"172.21.15.183:6809/787132252","cluster_addr":"172.21.15.183:6811/787132252","heartbeat_back_addr":"172.21.15.183:6815/787132252","heartbeat_front_addr":"172.21.15.183:6813/787132252","state":["exists","up"]},{"osd":6,"uuid":"3de2a49d-6cef-4580-9d8a-f90cdfcecd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6819","nonce":4206861158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6821","nonce":4206861158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6825","nonce":4206861158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6823","nonce":4206861158}]},"public_addr":"172.21.15.43:6819/4206861158","cluster_addr":"172.21.15.43:6821/4206861158","heartbeat_back_addr":"172.21.15.43:6825/4206861158","heartbeat_front_addr":"172.21.15.43:6823/4206861158","state":["exists","up"]},{"osd":7,"uuid":"a3b832f7-7d7d-4a62-b679-db6da98275d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6817","nonce":3452064146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6819","nonce":3452064146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6823","nonce":3452064146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6821","nonce":3452064146}]},"public_addr":"172.21.15.52:6817/3452064146","cluster_addr":"172.21.15.52:6819/3452064146","heartbeat_back_addr":"172.21.15.52:6823/3452064146","heartbeat_front_addr":"172.21.15.52:6821/3452064146","state":["exists","up"]},{"osd":8,"uuid":"ddc6b07f-d703-414d-a59a-02316376b13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6817","nonce":1023131725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6819","nonce":1023131725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6823","nonce":1023131725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6821","nonce":1023131725}]},"public_addr":"172.21.15.183:6817/1023131725","cluster_addr":"172.21.15.183:6819/1023131725","heartbeat_back_addr":"172.21.15.183:6823/1023131725","heartbeat_front_addr":"172.21.15.183:6821/1023131725","state":["exists","up"]},{"osd":9,"uuid":"ac16bb0e-6232-45c8-b16a-4d878874c588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6827","nonce":3935195657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6829","nonce":3935195657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6833","nonce":3935195657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6831","nonce":3935195657}]},"public_addr":"172.21.15.43:6827/3935195657","cluster_addr":"172.21.15.43:6829/3935195657","heartbeat_back_addr":"172.21.15.43:6833/3935195657","heartbeat_front_addr":"172.21.15.43:6831/3935195657","state":["exists","up"]},{"osd":10,"uuid":"fd839188-f398-4cff-aa7b-36522ff1a8a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6825","nonce":377628320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6827","nonce":377628320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6831","nonce":377628320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6829","nonce":377628320}]},"public_addr":"172.21.15.52:6825/377628320","cluster_addr":"172.21.15.52:6827/377628320","heartbeat_back_addr":"172.21.15.52:6831/377628320","heartbeat_front_addr":"172.21.15.52:6829/377628320","state":["exists","up"]},{"osd":11,"uuid":"a40282a0-0c64-49ee-b64e-6f3f8119e827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6825","nonce":1373086433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6827","nonce":1373086433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6831","nonce":1373086433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6829","nonce":1373086433}]},"public_addr":"172.21.15.183:6825/1373086433","cluster_addr":"172.21.15.183:6827/1373086433","heartbeat_back_addr":"172.21.15.183:6831/1373086433","heartbeat_front_addr":"172.21.15.183:6829/1373086433","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:23:43.478905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:17.573532+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:50.286610+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:25:27.407663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:03.081294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:36.057416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:12.062013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:47.032271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:21.023201+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:57.364739+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:29:34.039995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:30:08.827577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6801/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/4240775295":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/510203140":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/4251957687":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:6801/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1885244665":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/298320867":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1301092625":"2024-03-27T08:21:36.149092+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:34:07.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.165+0000 7fcb90c00640 1 -- 172.21.15.43:0/4092666061 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fcb700a2880 msgr2=0x7fcb700a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:07.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.165+0000 7fcb90c00640 1 --2- 172.21.15.43:0/4092666061 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fcb700a2880 0x7fcb700a4d40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c005fd0 tx=0x7fcb7c005850 comp rx=0 tx=0).stop 2024-03-26T08:34:07.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.165+0000 7fcb90c00640 1 -- 172.21.15.43:0/4092666061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb8c105490 msgr2=0x7fcb8c1a10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:07.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.165+0000 7fcb90c00640 1 --2- 172.21.15.43:0/4092666061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb8c105490 0x7fcb8c1a10c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400eaa0 tx=0x7fcb8400ef60 comp rx=0 tx=0).stop 2024-03-26T08:34:07.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.166+0000 7fcb90c00640 1 -- 172.21.15.43:0/4092666061 shutdown_connections 2024-03-26T08:34:07.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.166+0000 7fcb90c00640 1 --2- 172.21.15.43:0/4092666061 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fcb700a2880 0x7fcb700a4d40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.166+0000 7fcb90c00640 1 --2- 172.21.15.43:0/4092666061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb8c105490 0x7fcb8c1a10c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.166+0000 7fcb90c00640 1 --2- 172.21.15.43:0/4092666061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb8c104ad0 0x7fcb8c19a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.166+0000 7fcb90c00640 1 --2- 172.21.15.43:0/4092666061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb8c1038e0 0x7fcb8c199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.166+0000 7fcb90c00640 1 -- 172.21.15.43:0/4092666061 >> 172.21.15.43:0/4092666061 conn(0x7fcb8c0fee50 msgr2=0x7fcb8c107e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:07.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.166+0000 7fcb90c00640 1 -- 172.21.15.43:0/4092666061 shutdown_connections 2024-03-26T08:34:07.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.166+0000 7fcb90c00640 1 -- 172.21.15.43:0/4092666061 wait complete. 2024-03-26T08:34:07.178 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-26T08:31:21.819689+0000', 'modified': '2024-03-26T08:31:48.080233+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34559}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34517': {'gid': 34517, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.52:6837/748239918', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6836', 'nonce': 748239918}, {'type': 'v1', 'addr': '172.21.15.52:6837', 'nonce': 748239918}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34559': {'gid': 34559, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.52:6839/2551965101', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6838', 'nonce': 2551965101}, {'type': 'v1', 'addr': '172.21.15.52:6839', 'nonce': 2551965101}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34559, 'qdb_cluster': [34559]} 2024-03-26T08:34:07.178 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-26T08:34:07.178 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-26T08:34:07.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.404+0000 7f716ca00640 1 -- 172.21.15.43:0/3557420091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71681037a0 msgr2=0x7f7168103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:07.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.404+0000 7f716ca00640 1 --2- 172.21.15.43:0/3557420091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71681037a0 0x7f7168103bc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7158009900 tx=0x7f715802bc50 comp rx=0 tx=0).stop 2024-03-26T08:34:07.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.404+0000 7f716ca00640 1 -- 172.21.15.43:0/3557420091 shutdown_connections 2024-03-26T08:34:07.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.404+0000 7f716ca00640 1 --2- 172.21.15.43:0/3557420091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7168105350 0x7f716810bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.404+0000 7f716ca00640 1 --2- 172.21.15.43:0/3557420091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7168104990 0x7f7168104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.404+0000 7f716ca00640 1 --2- 172.21.15.43:0/3557420091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71681037a0 0x7f7168103bc0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.404+0000 7f716ca00640 1 -- 172.21.15.43:0/3557420091 >> 172.21.15.43:0/3557420091 conn(0x7f71680fed10 msgr2=0x7f7168101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:07.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.404+0000 7f716ca00640 1 -- 172.21.15.43:0/3557420091 shutdown_connections 2024-03-26T08:34:07.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.404+0000 7f716ca00640 1 -- 172.21.15.43:0/3557420091 wait complete. 2024-03-26T08:34:07.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.405+0000 7f716ca00640 1 Processor -- start 2024-03-26T08:34:07.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.405+0000 7f716ca00640 1 -- start start 2024-03-26T08:34:07.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.406+0000 7f716ca00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71681037a0 0x7f7168199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:07.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.406+0000 7f716ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7168104990 0x7f7168199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:07.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.406+0000 7f716ca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7168105350 0x7f71681a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:07.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.406+0000 7f716ca00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716810f340 con 0x7f7168104990 2024-03-26T08:34:07.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.406+0000 7f716ca00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716810f1c0 con 0x7f71681037a0 2024-03-26T08:34:07.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.406+0000 7f716ca00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716810f4c0 con 0x7f7168105350 2024-03-26T08:34:07.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.406+0000 7f7166a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7168104990 0x7f7168199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:07.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.406+0000 7f7166a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7168104990 0x7f7168199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36942/0 (socket says 172.21.15.43:36942) 2024-03-26T08:34:07.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.406+0000 7f7166a00640 1 -- 172.21.15.43:0/3718326876 learned_addr learned my addr 172.21.15.43:0/3718326876 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:34:07.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.406+0000 7f7167400640 1 --2- 172.21.15.43:0/3718326876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71681037a0 0x7f7168199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:07.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.406+0000 7f7167e00640 1 --2- 172.21.15.43:0/3718326876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7168105350 0x7f71681a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:07.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.407+0000 7f7166a00640 1 -- 172.21.15.43:0/3718326876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7168105350 msgr2=0x7f71681a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:07.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.407+0000 7f7166a00640 1 --2- 172.21.15.43:0/3718326876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7168105350 0x7f71681a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.407+0000 7f7166a00640 1 -- 172.21.15.43:0/3718326876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71681037a0 msgr2=0x7f7168199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:07.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.407+0000 7f7166a00640 1 --2- 172.21.15.43:0/3718326876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71681037a0 0x7f7168199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.407+0000 7f7166a00640 1 -- 172.21.15.43:0/3718326876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71681a1690 con 0x7f7168104990 2024-03-26T08:34:07.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.407+0000 7f7166a00640 1 --2- 172.21.15.43:0/3718326876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7168104990 0x7f7168199fb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f715400b450 tx=0x7f715400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:07.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.407+0000 7f715fe00640 1 -- 172.21.15.43:0/3718326876 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7154015020 con 0x7f7168104990 2024-03-26T08:34:07.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.407+0000 7f715fe00640 1 -- 172.21.15.43:0/3718326876 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7154011d40 con 0x7f7168104990 2024-03-26T08:34:07.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.407+0000 7f716ca00640 1 -- 172.21.15.43:0/3718326876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71681a1980 con 0x7f7168104990 2024-03-26T08:34:07.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.407+0000 7f715fe00640 1 -- 172.21.15.43:0/3718326876 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715400f340 con 0x7f7168104990 2024-03-26T08:34:07.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.407+0000 7f716ca00640 1 -- 172.21.15.43:0/3718326876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71681a1f50 con 0x7f7168104990 2024-03-26T08:34:07.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:07 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1755752333' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T08:34:07.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:07 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1987716018' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:34:07.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.412+0000 7f7170d9e740 1 Processor -- start 2024-03-26T08:34:07.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.412+0000 7f7170d9e740 1 -- start start 2024-03-26T08:34:07.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.412+0000 7f7170d9e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8de41ad80 0x55d8de41b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:07.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.412+0000 7f7170d9e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8de41b770 0x55d8de420270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:07.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.412+0000 7f7170d9e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8de4208e0 0x55d8de422d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:07.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.412+0000 7f7170d9e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8de37fc90 con 0x55d8de41b770 2024-03-26T08:34:07.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.412+0000 7f7170d9e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8de328980 con 0x55d8de4208e0 2024-03-26T08:34:07.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.412+0000 7f7170d9e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8de37bd30 con 0x55d8de41ad80 2024-03-26T08:34:07.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.412+0000 7f715fe00640 1 -- 172.21.15.43:0/3718326876 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f7154018020 con 0x7f7168104990 2024-03-26T08:34:07.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.412+0000 7f7167400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8de41b770 0x55d8de420270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:07.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.413+0000 7f7167400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8de41b770 0x55d8de420270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36956/0 (socket says 172.21.15.43:36956) 2024-03-26T08:34:07.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.413+0000 7f7167400640 1 -- 172.21.15.43:0/3945733790 learned_addr learned my addr 172.21.15.43:0/3945733790 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:34:07.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.413+0000 7f7167e00640 1 --2- 172.21.15.43:0/3945733790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8de41ad80 0x55d8de41b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:07.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.413+0000 7f7167400640 1 -- 172.21.15.43:0/3945733790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8de41ad80 msgr2=0x55d8de41b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:34:07.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.413+0000 7f7167400640 1 --2- 172.21.15.43:0/3945733790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8de41ad80 0x55d8de41b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.413+0000 7f7167400640 1 -- 172.21.15.43:0/3945733790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8de4208e0 msgr2=0x55d8de422d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:34:07.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.413+0000 7f7167400640 1 --2- 172.21.15.43:0/3945733790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8de4208e0 0x55d8de422d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.413+0000 7f7167400640 1 -- 172.21.15.43:0/3945733790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8de340510 con 0x55d8de41b770 2024-03-26T08:34:07.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.413+0000 7f7167400640 1 --2- 172.21.15.43:0/3945733790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8de41b770 0x55d8de420270 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7158005940 tx=0x7f7158005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ccec03e0024a14e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:07.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.414+0000 7f7153e00640 1 -- 172.21.15.43:0/3945733790 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7158044020 con 0x55d8de41b770 2024-03-26T08:34:07.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.414+0000 7f7153e00640 1 -- 172.21.15.43:0/3945733790 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7158005040 con 0x55d8de41b770 2024-03-26T08:34:07.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.414+0000 7f7153e00640 1 -- 172.21.15.43:0/3945733790 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7158004920 con 0x55d8de41b770 2024-03-26T08:34:07.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.414+0000 7f715fe00640 1 --2- 172.21.15.43:0/3718326876 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f714c0a2880 0x7f714c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:07.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.414+0000 7f715fe00640 1 -- 172.21.15.43:0/3718326876 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f71540ce980 con 0x7f7168104990 2024-03-26T08:34:07.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.414+0000 7f7167e00640 1 --2- 172.21.15.43:0/3718326876 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f714c0a2880 0x7f714c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:07.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.414+0000 7f7170d9e740 1 -- 172.21.15.43:0/3945733790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8de41b770 msgr2=0x55d8de420270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:07.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.414+0000 7f7170d9e740 1 --2- 172.21.15.43:0/3945733790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8de41b770 0x55d8de420270 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7158005940 tx=0x7f7158005dd0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.414+0000 7f7170d9e740 1 -- 172.21.15.43:0/3945733790 shutdown_connections 2024-03-26T08:34:07.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.414+0000 7f7170d9e740 1 --2- 172.21.15.43:0/3945733790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8de4208e0 0x55d8de422d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.414+0000 7f7170d9e740 1 --2- 172.21.15.43:0/3945733790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8de41b770 0x55d8de420270 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.414+0000 7f7170d9e740 1 --2- 172.21.15.43:0/3945733790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8de41ad80 0x55d8de41b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.414+0000 7f7170d9e740 1 -- 172.21.15.43:0/3945733790 >> 172.21.15.43:0/3945733790 conn(0x55d8de4162f0 msgr2=0x55d8de4166f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:07.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.415+0000 7f7170d9e740 1 -- 172.21.15.43:0/3945733790 shutdown_connections 2024-03-26T08:34:07.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.415+0000 7f7170d9e740 1 -- 172.21.15.43:0/3945733790 wait complete. 2024-03-26T08:34:07.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.415+0000 7f7167e00640 1 --2- 172.21.15.43:0/3718326876 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f714c0a2880 0x7f714c0a4d40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55d8de41bfd0 tx=0x7f71600097f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:07.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.415+0000 7f7170d9e740 1 -- start start 2024-03-26T08:34:07.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.415+0000 7f7170d9e740 1 Processor -- start 2024-03-26T08:34:07.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.416+0000 7f7170d9e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8de41ad80 0x55d8de425290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:07.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.416+0000 7f7170d9e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8de41b770 0x55d8de4257d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:07.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.416+0000 7f7170d9e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8de4208e0 0x55d8de42c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:07.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.416+0000 7f7170d9e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8de423830 con 0x55d8de41ad80 2024-03-26T08:34:07.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.416+0000 7f7170d9e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8de33bdd0 con 0x55d8de41b770 2024-03-26T08:34:07.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.416+0000 7f7170d9e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8de4239b0 con 0x55d8de4208e0 2024-03-26T08:34:07.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.416+0000 7f7170d9e740 10 client.0 _ll_register_callbacks cb 0x55d8de093ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:34:07.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.416+0000 7f7167400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8de41ad80 0x55d8de425290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:07.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.416+0000 7f7167400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8de41ad80 0x55d8de425290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36968/0 (socket says 172.21.15.43:36968) 2024-03-26T08:34:07.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.416+0000 7f7167400640 1 -- 172.21.15.43:0/1430993536 learned_addr learned my addr 172.21.15.43:0/1430993536 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:34:07.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.416+0000 7f7166a00640 1 --2- 172.21.15.43:0/1430993536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8de41b770 0x55d8de4257d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:07.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.416+0000 7f7167400640 1 -- 172.21.15.43:0/1430993536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8de4208e0 msgr2=0x55d8de42c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:34:07.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.416+0000 7f7167e00640 1 --2- 172.21.15.43:0/1430993536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8de4208e0 0x55d8de42c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:07.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.416+0000 7f7167400640 1 --2- 172.21.15.43:0/1430993536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8de4208e0 0x55d8de42c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.416+0000 7f7167400640 1 -- 172.21.15.43:0/1430993536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8de41b770 msgr2=0x55d8de4257d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:34:07.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.416+0000 7f7167400640 1 --2- 172.21.15.43:0/1430993536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8de41b770 0x55d8de4257d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.416+0000 7f7167400640 1 -- 172.21.15.43:0/1430993536 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8de0942d0 con 0x55d8de41ad80 2024-03-26T08:34:07.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.417+0000 7f7166a00640 1 --2- 172.21.15.43:0/1430993536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8de41b770 0x55d8de4257d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:34:07.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.417+0000 7f7167400640 1 --2- 172.21.15.43:0/1430993536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8de41ad80 0x55d8de425290 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7158036040 tx=0x7f7158004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238e0905f5ad5581 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:07.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.417+0000 7f7150c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-26T08:34:07.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.417+0000 7f7150c00640 1 -- 172.21.15.43:0/1430993536 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7158050070 con 0x55d8de41ad80 2024-03-26T08:34:07.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.417+0000 7f7150c00640 1 -- 172.21.15.43:0/1430993536 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f7158004d70 con 0x55d8de41ad80 2024-03-26T08:34:07.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.417+0000 7f7170d9e740 1 -- 172.21.15.43:0/1430993536 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7158005040 con 0x55d8de41ad80 2024-03-26T08:34:07.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.417+0000 7f7150c00640 1 -- 172.21.15.43:0/1430993536 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f715805b050 con 0x55d8de41ad80 2024-03-26T08:34:07.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.417+0000 7f7170d9e740 10 client.15390 fetch_fsmap learned FSMap version 21 2024-03-26T08:34:07.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.417+0000 7f7170d9e740 1 -- 172.21.15.43:0/1430993536 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55d8de15e6b0 con 0x55d8de41ad80 2024-03-26T08:34:07.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.418+0000 7f7150c00640 1 -- 172.21.15.43:0/1430993536 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7158005040 con 0x55d8de41ad80 2024-03-26T08:34:07.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.418+0000 7f7150c00640 1 -- 172.21.15.43:0/1430993536 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f71580494e0 con 0x55d8de41ad80 2024-03-26T08:34:07.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.418+0000 7f7170d9e740 10 client.15390 fetch_fsmap finished waiting for FSMap version 21 2024-03-26T08:34:07.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.418+0000 7f7170d9e740 10 client.15390 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T08:34:07.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.418+0000 7f7170d9e740 20 client.15390 populate_metadata read hostname 'smithi043' 2024-03-26T08:34:07.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.418+0000 7f7170d9e740 1 --2- 172.21.15.43:0/1430993536 >> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] conn(0x55d8de431510 0x55d8de435990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:07.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.418+0000 7f7170d9e740 4 client.15390 mds_command: new command op to 34559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:34:07.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.418+0000 7f7170d9e740 1 -- 172.21.15.43:0/1430993536 --> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8de321520 con 0x55d8de431510 2024-03-26T08:34:07.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.419+0000 7f7166a00640 1 --2- 172.21.15.43:0/1430993536 >> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] conn(0x55d8de431510 0x55d8de435990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:07.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.420+0000 7f7166a00640 1 --2- 172.21.15.43:0/1430993536 >> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] conn(0x55d8de431510 0x55d8de435990 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1e47382b328e511 server_cookie=68acfd4ec7a217ce in_seq=0 out_seq=0 2024-03-26T08:34:07.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.420+0000 7f7150c00640 10 client.15390 ms_handle_connect on v2:172.21.15.52:6838/2551965101 2024-03-26T08:34:07.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.423+0000 7f7150c00640 1 -- 172.21.15.43:0/1430993536 <== mds.0 v2:172.21.15.52:6838/2551965101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d8de321520 con 0x55d8de431510 2024-03-26T08:34:07.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.423+0000 7f7150c00640 10 client.15390 handle_command_reply: tid=0 2024-03-26T08:34:07.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.423+0000 7f7170d9e740 1 client.15390 shutdown 2024-03-26T08:34:07.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.423+0000 7f7170d9e740 1 -- 172.21.15.43:0/1430993536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8de41ad80 msgr2=0x55d8de425290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:07.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.423+0000 7f7170d9e740 1 --2- 172.21.15.43:0/1430993536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8de41ad80 0x55d8de425290 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7158036040 tx=0x7f7158004970 comp rx=0 tx=0).stop 2024-03-26T08:34:07.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.423+0000 7f7170d9e740 1 -- 172.21.15.43:0/1430993536 shutdown_connections 2024-03-26T08:34:07.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.423+0000 7f7170d9e740 1 --2- 172.21.15.43:0/1430993536 >> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] conn(0x55d8de431510 0x55d8de435990 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.423+0000 7f7170d9e740 1 --2- 172.21.15.43:0/1430993536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8de4208e0 0x55d8de42c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.423+0000 7f7170d9e740 1 --2- 172.21.15.43:0/1430993536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8de41b770 0x55d8de4257d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.423+0000 7f7170d9e740 1 --2- 172.21.15.43:0/1430993536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8de41ad80 0x55d8de425290 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.423+0000 7f7150c00640 0 client.15390 ms_handle_reset on v2:172.21.15.52:6838/2551965101 2024-03-26T08:34:07.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.423+0000 7f7170d9e740 1 -- 172.21.15.43:0/1430993536 >> 172.21.15.43:0/1430993536 conn(0x55d8de4162f0 msgr2=0x55d8de4154f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:07.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.423+0000 7f7170d9e740 1 -- 172.21.15.43:0/1430993536 shutdown_connections 2024-03-26T08:34:07.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.423+0000 7f7170d9e740 1 -- 172.21.15.43:0/1430993536 wait complete. 2024-03-26T08:34:07.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.423+0000 7f7170d9e740 20 client.15390 trim_cache size 0 max 16384 2024-03-26T08:34:07.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.441+0000 7f7170d9e740 1 Processor -- start 2024-03-26T08:34:07.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.441+0000 7f7170d9e740 1 -- start start 2024-03-26T08:34:07.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.442+0000 7f7170d9e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8de41ad80 0x55d8de4166d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:07.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.442+0000 7f7170d9e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8de41b770 0x55d8de42c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.442+0000 7f7170d9e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8de4208e0 0x55d8de414c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.442+0000 7f7170d9e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8de430d30 con 0x55d8de41b770 2024-03-26T08:34:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.442+0000 7f7170d9e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8de431390 con 0x55d8de41ad80 2024-03-26T08:34:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.442+0000 7f7170d9e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8de4319f0 con 0x55d8de4208e0 2024-03-26T08:34:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.442+0000 7f7166a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8de41b770 0x55d8de42c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.442+0000 7f7166a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8de41b770 0x55d8de42c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36974/0 (socket says 172.21.15.43:36974) 2024-03-26T08:34:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.442+0000 7f7166a00640 1 -- 172.21.15.43:0/536474424 learned_addr learned my addr 172.21.15.43:0/536474424 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:34:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.442+0000 7f7167400640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8de41ad80 0x55d8de4166d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.443+0000 7f7167e00640 1 --2- 172.21.15.43:0/536474424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8de4208e0 0x55d8de414c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.443+0000 7f7166a00640 1 -- 172.21.15.43:0/536474424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8de4208e0 msgr2=0x55d8de414c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:34:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.443+0000 7f7166a00640 1 --2- 172.21.15.43:0/536474424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8de4208e0 0x55d8de414c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.443+0000 7f7166a00640 1 -- 172.21.15.43:0/536474424 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8de41ad80 msgr2=0x55d8de4166d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:34:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.443+0000 7f7166a00640 1 --2- 172.21.15.43:0/536474424 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8de41ad80 0x55d8de4166d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.443+0000 7f7166a00640 1 -- 172.21.15.43:0/536474424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8de408170 con 0x55d8de41b770 2024-03-26T08:34:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.443+0000 7f7167400640 1 --2- 172.21.15.43:0/536474424 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8de41ad80 0x55d8de4166d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:34:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.443+0000 7f7166a00640 1 --2- 172.21.15.43:0/536474424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8de41b770 0x55d8de42c7d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f715409ac40 tx=0x7f71540108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4abc1ae6dc254f08 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.443+0000 7f7153400640 1 -- 172.21.15.43:0/536474424 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715409f090 con 0x55d8de41b770 2024-03-26T08:34:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.443+0000 7f7153400640 1 -- 172.21.15.43:0/536474424 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f71540a36e0 con 0x55d8de41b770 2024-03-26T08:34:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.443+0000 7f7153400640 1 -- 172.21.15.43:0/536474424 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715400eb70 con 0x55d8de41b770 2024-03-26T08:34:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.444+0000 7f7170d9e740 1 -- 172.21.15.43:0/536474424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8de41b770 msgr2=0x55d8de42c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.444+0000 7f7170d9e740 1 --2- 172.21.15.43:0/536474424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8de41b770 0x55d8de42c7d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f715409ac40 tx=0x7f71540108e0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.444+0000 7f7170d9e740 1 -- 172.21.15.43:0/536474424 shutdown_connections 2024-03-26T08:34:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.444+0000 7f7170d9e740 1 --2- 172.21.15.43:0/536474424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8de4208e0 0x55d8de414c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.444+0000 7f7170d9e740 1 --2- 172.21.15.43:0/536474424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8de41b770 0x55d8de42c7d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.444+0000 7f7170d9e740 1 --2- 172.21.15.43:0/536474424 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8de41ad80 0x55d8de4166d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.444+0000 7f7170d9e740 1 -- 172.21.15.43:0/536474424 >> 172.21.15.43:0/536474424 conn(0x55d8de4162f0 msgr2=0x55d8de423d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:07.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.444+0000 7f7170d9e740 1 -- 172.21.15.43:0/536474424 shutdown_connections 2024-03-26T08:34:07.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.444+0000 7f7170d9e740 1 -- 172.21.15.43:0/536474424 wait complete. 2024-03-26T08:34:07.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.445+0000 7f7170d9e740 1 -- start start 2024-03-26T08:34:07.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.445+0000 7f7170d9e740 1 Processor -- start 2024-03-26T08:34:07.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.445+0000 7f7170d9e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8de41ad80 0x55d8de428060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:07.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.445+0000 7f7170d9e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8de41b770 0x55d8de4285a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:07.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.445+0000 7f7170d9e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8de4208e0 0x55d8de42b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:07.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.445+0000 7f7170d9e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8de4326b0 con 0x55d8de4208e0 2024-03-26T08:34:07.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.445+0000 7f7170d9e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8de432050 con 0x55d8de41ad80 2024-03-26T08:34:07.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.445+0000 7f7170d9e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8de432d10 con 0x55d8de41b770 2024-03-26T08:34:07.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.445+0000 7f7170d9e740 10 client.0 _ll_register_callbacks cb 0x7f7158009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:34:07.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.446+0000 7f7167e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8de4208e0 0x55d8de42b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:07.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.446+0000 7f7167e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8de4208e0 0x55d8de42b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36990/0 (socket says 172.21.15.43:36990) 2024-03-26T08:34:07.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.446+0000 7f7167e00640 1 -- 172.21.15.43:0/467911475 learned_addr learned my addr 172.21.15.43:0/467911475 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:34:07.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.446+0000 7f7167400640 1 --2- 172.21.15.43:0/467911475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8de41ad80 0x55d8de428060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:07.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.446+0000 7f7166a00640 1 --2- 172.21.15.43:0/467911475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8de41b770 0x55d8de4285a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:07.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.446+0000 7f7167e00640 1 -- 172.21.15.43:0/467911475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8de41b770 msgr2=0x55d8de4285a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:34:07.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.446+0000 7f7167e00640 1 --2- 172.21.15.43:0/467911475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8de41b770 0x55d8de4285a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.446+0000 7f7167e00640 1 -- 172.21.15.43:0/467911475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8de41ad80 msgr2=0x55d8de428060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:34:07.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.446+0000 7f7167e00640 1 --2- 172.21.15.43:0/467911475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8de41ad80 0x55d8de428060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.446+0000 7f7167e00640 1 -- 172.21.15.43:0/467911475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8de42bb60 con 0x55d8de4208e0 2024-03-26T08:34:07.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.447+0000 7f7167e00640 1 --2- 172.21.15.43:0/467911475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8de4208e0 0x55d8de42b570 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7160012bb0 tx=0x7f7160004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc501c516dc5acb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:07.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.447+0000 7f7150c00640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-26T08:34:07.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.447+0000 7f7150c00640 1 -- 172.21.15.43:0/467911475 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716001fcc0 con 0x55d8de4208e0 2024-03-26T08:34:07.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.447+0000 7f7150c00640 1 -- 172.21.15.43:0/467911475 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f7160020430 con 0x55d8de4208e0 2024-03-26T08:34:07.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.447+0000 7f7170d9e740 1 -- 172.21.15.43:0/467911475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71540a36e0 con 0x55d8de4208e0 2024-03-26T08:34:07.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.447+0000 7f7150c00640 1 -- 172.21.15.43:0/467911475 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f716001fe60 con 0x55d8de4208e0 2024-03-26T08:34:07.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.447+0000 7f7170d9e740 10 client.15396 fetch_fsmap learned FSMap version 21 2024-03-26T08:34:07.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.447+0000 7f7170d9e740 1 -- 172.21.15.43:0/467911475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55d8de4954e0 con 0x55d8de4208e0 2024-03-26T08:34:07.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.447+0000 7f7150c00640 1 -- 172.21.15.43:0/467911475 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f71540a36e0 con 0x55d8de4208e0 2024-03-26T08:34:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.447+0000 7f7150c00640 1 -- 172.21.15.43:0/467911475 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f716000f050 con 0x55d8de4208e0 2024-03-26T08:34:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.448+0000 7f7170d9e740 10 client.15396 fetch_fsmap finished waiting for FSMap version 21 2024-03-26T08:34:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.448+0000 7f7170d9e740 10 client.15396 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T08:34:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.448+0000 7f7170d9e740 20 client.15396 populate_metadata read hostname 'smithi043' 2024-03-26T08:34:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.448+0000 7f7170d9e740 1 --2- 172.21.15.43:0/467911475 >> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] conn(0x55d8de499c70 0x55d8de495820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.448+0000 7f7170d9e740 4 client.15396 mds_command: new command op to 34559 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T08:34:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.448+0000 7f7170d9e740 1 -- 172.21.15.43:0/467911475 --> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d8de38d1d0 con 0x55d8de499c70 2024-03-26T08:34:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.448+0000 7f7167400640 1 --2- 172.21.15.43:0/467911475 >> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] conn(0x55d8de499c70 0x55d8de495820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.449+0000 7f7167400640 1 --2- 172.21.15.43:0/467911475 >> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] conn(0x55d8de499c70 0x55d8de495820 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aa440e120b03dcb server_cookie=b8d4184cf5c13809 in_seq=0 out_seq=0 2024-03-26T08:34:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.449+0000 7f7150c00640 10 client.15396 ms_handle_connect on v2:172.21.15.52:6838/2551965101 2024-03-26T08:34:07.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.450+0000 7f7150c00640 1 -- 172.21.15.43:0/467911475 <== mds.0 v2:172.21.15.52:6838/2551965101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55d8de38d1d0 con 0x55d8de499c70 2024-03-26T08:34:07.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.450+0000 7f7150c00640 10 client.15396 handle_command_reply: tid=0 2024-03-26T08:34:07.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.450+0000 7f7170d9e740 1 client.15396 shutdown 2024-03-26T08:34:07.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.450+0000 7f7170d9e740 1 -- 172.21.15.43:0/467911475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8de4208e0 msgr2=0x55d8de42b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:07.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.450+0000 7f7170d9e740 1 --2- 172.21.15.43:0/467911475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8de4208e0 0x55d8de42b570 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7160012bb0 tx=0x7f7160004a40 comp rx=0 tx=0).stop 2024-03-26T08:34:07.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.450+0000 7f7170d9e740 1 -- 172.21.15.43:0/467911475 shutdown_connections 2024-03-26T08:34:07.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.450+0000 7f7170d9e740 1 --2- 172.21.15.43:0/467911475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d8de4208e0 0x55d8de42b570 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.450+0000 7f7170d9e740 1 --2- 172.21.15.43:0/467911475 >> [v2:172.21.15.52:6838/2551965101,v1:172.21.15.52:6839/2551965101] conn(0x55d8de499c70 0x55d8de495820 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.450+0000 7f7170d9e740 1 --2- 172.21.15.43:0/467911475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8de41b770 0x55d8de4285a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.450+0000 7f7170d9e740 1 --2- 172.21.15.43:0/467911475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8de41ad80 0x55d8de428060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.450+0000 7f7170d9e740 1 -- 172.21.15.43:0/467911475 >> 172.21.15.43:0/467911475 conn(0x55d8de4162f0 msgr2=0x55d8de4166d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:07.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.450+0000 7f7150c00640 0 client.15396 ms_handle_reset on v2:172.21.15.52:6838/2551965101 2024-03-26T08:34:07.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.450+0000 7f7170d9e740 1 -- 172.21.15.43:0/467911475 shutdown_connections 2024-03-26T08:34:07.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.450+0000 7f7170d9e740 1 -- 172.21.15.43:0/467911475 wait complete. 2024-03-26T08:34:07.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.450+0000 7f7170d9e740 20 client.15396 trim_cache size 0 max 16384 2024-03-26T08:34:07.447 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-26T08:34:07.447 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "88759420-eb49-11ee-95d0-87774f69a715", 2024-03-26T08:34:07.447 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-03-26T08:34:07.447 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 34559, 2024-03-26T08:34:07.447 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-03-26T08:34:07.447 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-03-26T08:34:07.447 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-03-26T08:34:07.447 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 165.07712175500001, 2024-03-26T08:34:07.447 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 21, 2024-03-26T08:34:07.447 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 94, 2024-03-26T08:34:07.447 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 92, 2024-03-26T08:34:07.447 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 173.25898809700001 2024-03-26T08:34:07.448 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-26T08:34:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.451+0000 7f716ca00640 1 -- 172.21.15.43:0/3718326876 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f714c0a2880 msgr2=0x7f714c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.451+0000 7f716ca00640 1 --2- 172.21.15.43:0/3718326876 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f714c0a2880 0x7f714c0a4d40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55d8de41bfd0 tx=0x7f71600097f0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.451+0000 7f716ca00640 1 -- 172.21.15.43:0/3718326876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7168104990 msgr2=0x7f7168199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.451+0000 7f716ca00640 1 --2- 172.21.15.43:0/3718326876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7168104990 0x7f7168199fb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f715400b450 tx=0x7f715400b910 comp rx=0 tx=0).stop 2024-03-26T08:34:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.451+0000 7f716ca00640 1 -- 172.21.15.43:0/3718326876 shutdown_connections 2024-03-26T08:34:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.451+0000 7f716ca00640 1 --2- 172.21.15.43:0/3718326876 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f714c0a2880 0x7f714c0a4d40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.451+0000 7f716ca00640 1 --2- 172.21.15.43:0/3718326876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7168105350 0x7f71681a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.451+0000 7f716ca00640 1 --2- 172.21.15.43:0/3718326876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7168104990 0x7f7168199fb0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.451+0000 7f716ca00640 1 --2- 172.21.15.43:0/3718326876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71681037a0 0x7f7168199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.451+0000 7f716ca00640 1 -- 172.21.15.43:0/3718326876 >> 172.21.15.43:0/3718326876 conn(0x7f71680fed10 msgr2=0x7f7168107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.451+0000 7f716ca00640 1 -- 172.21.15.43:0/3718326876 shutdown_connections 2024-03-26T08:34:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:07.451+0000 7f716ca00640 1 -- 172.21.15.43:0/3718326876 wait complete. 2024-03-26T08:34:07.461 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-26T08:34:07.472 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-26T08:34:07.472 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-03-26T08:34:07.472 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-03-26T08:34:07.472 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-26T08:34:07.472 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-26T08:34:07.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:34:07.473 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-03-26T08:34:07.523 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-26T08:34:07.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:34:07.524 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-26T08:34:07.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:07 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1755752333' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T08:34:07.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:07 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1987716018' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:34:07.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:07 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1755752333' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T08:34:07.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:07 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1987716018' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:34:07.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:34:07.622 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link delete ceph-brx 2024-03-26T08:34:07.758 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-26T08:34:07.759 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 88759420-eb49-11ee-95d0-87774f69a715 -- ceph fs ls 2024-03-26T08:34:08.217 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:34:08.374 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:08 smithi043 ceph-mon[31929]: pgmap v459: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:34:08.374 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:08 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/4092666061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:34:08.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:08 smithi183 ceph-mon[35818]: pgmap v459: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:34:08.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:08 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/4092666061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:34:08.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:08 smithi052 ceph-mon[35071]: pgmap v459: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:34:08.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:08 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/4092666061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:34:10.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:10 smithi043 ceph-mon[31929]: pgmap v460: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:10.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:10 smithi183 ceph-mon[35818]: pgmap v460: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:10.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:10 smithi052 ceph-mon[35071]: pgmap v460: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:10.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.697+0000 7feb41200700 1 -- 172.21.15.43:0/1691879796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb3c069e40 msgr2=0x7feb3c06a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:10.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.697+0000 7feb41200700 1 --2- 172.21.15.43:0/1691879796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb3c069e40 0x7feb3c06a2a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7feb280099c0 tx=0x7feb28009cd0 comp rx=0 tx=0).stop 2024-03-26T08:34:10.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.698+0000 7feb41200700 1 -- 172.21.15.43:0/1691879796 shutdown_connections 2024-03-26T08:34:10.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.698+0000 7feb41200700 1 --2- 172.21.15.43:0/1691879796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb3c06a7e0 0x7feb3c10f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:10.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.698+0000 7feb41200700 1 --2- 172.21.15.43:0/1691879796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb3c1081e0 0x7feb3c1085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:10.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.698+0000 7feb41200700 1 --2- 172.21.15.43:0/1691879796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb3c069e40 0x7feb3c06a2a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:10.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.698+0000 7feb41200700 1 -- 172.21.15.43:0/1691879796 >> 172.21.15.43:0/1691879796 conn(0x7feb3c077fc0 msgr2=0x7feb3c0783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:10.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.698+0000 7feb41200700 1 -- 172.21.15.43:0/1691879796 shutdown_connections 2024-03-26T08:34:10.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.698+0000 7feb41200700 1 -- 172.21.15.43:0/1691879796 wait complete. 2024-03-26T08:34:10.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.699+0000 7feb41200700 1 Processor -- start 2024-03-26T08:34:10.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.700+0000 7feb41200700 1 -- start start 2024-03-26T08:34:10.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.700+0000 7feb41200700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb3c069e40 0x7feb3c101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:10.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.700+0000 7feb41200700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb3c06a7e0 0x7feb3c1016a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:10.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.700+0000 7feb41200700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb3c1081e0 0x7feb3c101be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:10.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.700+0000 7feb41200700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c112310 con 0x7feb3c069e40 2024-03-26T08:34:10.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.700+0000 7feb41200700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c112190 con 0x7feb3c1081e0 2024-03-26T08:34:10.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.700+0000 7feb41200700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c112490 con 0x7feb3c06a7e0 2024-03-26T08:34:10.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.700+0000 7feb39e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb3c069e40 0x7feb3c101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:10.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.701+0000 7feb39e00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb3c069e40 0x7feb3c101160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37018/0 (socket says 172.21.15.43:37018) 2024-03-26T08:34:10.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.701+0000 7feb39e00700 1 -- 172.21.15.43:0/129253928 learned_addr learned my addr 172.21.15.43:0/129253928 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:34:10.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.701+0000 7feb39400700 1 --2- 172.21.15.43:0/129253928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb3c06a7e0 0x7feb3c1016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:10.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.701+0000 7feb3a800700 1 --2- 172.21.15.43:0/129253928 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb3c1081e0 0x7feb3c101be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:10.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.701+0000 7feb39e00700 1 -- 172.21.15.43:0/129253928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb3c06a7e0 msgr2=0x7feb3c1016a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:10.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.701+0000 7feb39e00700 1 --2- 172.21.15.43:0/129253928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb3c06a7e0 0x7feb3c1016a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:10.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.701+0000 7feb39e00700 1 -- 172.21.15.43:0/129253928 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb3c1081e0 msgr2=0x7feb3c101be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:10.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.701+0000 7feb39e00700 1 --2- 172.21.15.43:0/129253928 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb3c1081e0 0x7feb3c101be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:10.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.701+0000 7feb39e00700 1 -- 172.21.15.43:0/129253928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3c1a6d50 con 0x7feb3c069e40 2024-03-26T08:34:10.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.701+0000 7feb3a800700 1 --2- 172.21.15.43:0/129253928 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb3c1081e0 0x7feb3c101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:34:10.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.702+0000 7feb39e00700 1 --2- 172.21.15.43:0/129253928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb3c069e40 0x7feb3c101160 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0060b0 tx=0x7feb2c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:10.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.702+0000 7feb32a00700 1 -- 172.21.15.43:0/129253928 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c01a070 con 0x7feb3c069e40 2024-03-26T08:34:10.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.702+0000 7feb41200700 1 -- 172.21.15.43:0/129253928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb3c1a6fe0 con 0x7feb3c069e40 2024-03-26T08:34:10.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.702+0000 7feb32a00700 1 -- 172.21.15.43:0/129253928 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7feb2c00ddd0 con 0x7feb3c069e40 2024-03-26T08:34:10.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.702+0000 7feb32a00700 1 -- 172.21.15.43:0/129253928 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c014720 con 0x7feb3c069e40 2024-03-26T08:34:10.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.702+0000 7feb41200700 1 -- 172.21.15.43:0/129253928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb3c1a7490 con 0x7feb3c069e40 2024-03-26T08:34:10.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.703+0000 7feb41200700 1 -- 172.21.15.43:0/129253928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb3c068e50 con 0x7feb3c069e40 2024-03-26T08:34:10.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.706+0000 7feb32a00700 1 -- 172.21.15.43:0/129253928 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7feb2c01d070 con 0x7feb3c069e40 2024-03-26T08:34:10.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.707+0000 7feb32a00700 1 --2- 172.21.15.43:0/129253928 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7feb200a2890 0x7feb200a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:10.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.707+0000 7feb32a00700 1 -- 172.21.15.43:0/129253928 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7feb2c0ceab0 con 0x7feb3c069e40 2024-03-26T08:34:10.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.708+0000 7feb39400700 1 --2- 172.21.15.43:0/129253928 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7feb200a2890 0x7feb200a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:10.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.708+0000 7feb39400700 1 --2- 172.21.15.43:0/129253928 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7feb200a2890 0x7feb200a4d50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7feb3c102bf0 tx=0x7feb28005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:10.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:10.715+0000 7feb32a00700 1 -- 172.21.15.43:0/129253928 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7feb2c010070 con 0x7feb3c069e40 2024-03-26T08:34:10.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:11.002+0000 7feb41200700 1 -- 172.21.15.43:0/129253928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7feb3c02da30 con 0x7feb3c069e40 2024-03-26T08:34:10.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:11.002+0000 7feb32a00700 1 -- 172.21.15.43:0/129253928 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) v1 ==== 53+0+88 (secure 0 0 0) 0x7feb2c09b770 con 0x7feb3c069e40 2024-03-26T08:34:10.995 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-26T08:34:10.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:11.005+0000 7feb41200700 1 -- 172.21.15.43:0/129253928 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7feb200a2890 msgr2=0x7feb200a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:10.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:11.005+0000 7feb41200700 1 --2- 172.21.15.43:0/129253928 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7feb200a2890 0x7feb200a4d50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7feb3c102bf0 tx=0x7feb28005fb0 comp rx=0 tx=0).stop 2024-03-26T08:34:10.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:11.005+0000 7feb41200700 1 -- 172.21.15.43:0/129253928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb3c069e40 msgr2=0x7feb3c101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:10.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:11.005+0000 7feb41200700 1 --2- 172.21.15.43:0/129253928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb3c069e40 0x7feb3c101160 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0060b0 tx=0x7feb2c00d3b0 comp rx=0 tx=0).stop 2024-03-26T08:34:10.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:11.005+0000 7feb41200700 1 -- 172.21.15.43:0/129253928 shutdown_connections 2024-03-26T08:34:10.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:11.005+0000 7feb41200700 1 --2- 172.21.15.43:0/129253928 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7feb200a2890 0x7feb200a4d50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:10.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:11.006+0000 7feb41200700 1 --2- 172.21.15.43:0/129253928 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb3c1081e0 0x7feb3c101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:10.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:11.006+0000 7feb41200700 1 --2- 172.21.15.43:0/129253928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb3c069e40 0x7feb3c101160 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:10.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:11.006+0000 7feb41200700 1 --2- 172.21.15.43:0/129253928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb3c06a7e0 0x7feb3c1016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:10.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:11.006+0000 7feb41200700 1 -- 172.21.15.43:0/129253928 >> 172.21.15.43:0/129253928 conn(0x7feb3c077fc0 msgr2=0x7feb3c0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:10.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:11.006+0000 7feb41200700 1 -- 172.21.15.43:0/129253928 shutdown_connections 2024-03-26T08:34:11.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:11.006+0000 7feb41200700 1 -- 172.21.15.43:0/129253928 wait complete. 2024-03-26T08:34:11.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:11 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:34:11.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:11 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/129253928' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T08:34:11.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:11 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:34:11.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:11 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:34:11.527 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:11 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:34:11.527 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:11 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/129253928' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T08:34:11.527 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:11 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:34:11.527 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:11 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:34:11.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:11 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:34:11.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:11 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/129253928' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T08:34:11.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:11 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:34:11.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:11 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:34:11.853 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-26T08:34:11.853 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-26T08:34:11.853 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-03-26T08:34:11.853 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-26T08:34:11.854 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-26T08:34:11.854 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-26T08:34:11.854 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-26T08:34:11.854 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-26T08:34:11.854 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-26T08:34:11.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:34:11.854 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-03-26T08:34:11.909 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-26T08:34:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-26T08:34:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-03-26T08:34:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-26T08:34:11.909 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-03-26T08:34:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-26T08:34:11.910 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T08:34:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-03-26T08:34:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-03-26T08:34:11.910 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T08:34:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-03-26T08:34:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-03-26T08:34:11.910 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-26T08:34:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-03-26T08:34:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-03-26T08:34:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-26T08:34:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 41771sec preferred_lft 41771sec 2024-03-26T08:34:11.910 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link noprefixroute 2024-03-26T08:34:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-26T08:34:11.911 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T08:34:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-03-26T08:34:11.911 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-03-26T08:34:11.911 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-26T08:34:11.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:34:11.912 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-03-26T08:34:11.912 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-03-26T08:34:11.912 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add name ceph-brx type bridge 2024-03-26T08:34:11.912 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr flush dev ceph-brx 2024-03-26T08:34:11.912 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set ceph-brx up 2024-03-26T08:34:11.912 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-26T08:34:11.912 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-03-26T08:34:12.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:34:12.127 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-26T08:34:12.210 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-03-26T08:34:12.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:34:12.214 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-03-26T08:34:12.258 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-03-26T08:34:12.259 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-26T08:34:12.259 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-26T08:34:12.259 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-26T08:34:12.259 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-26T08:34:12.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:34:12.260 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-03-26T08:34:12.260 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-03-26T08:34:12.260 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-26T08:34:12.260 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-26T08:34:12.260 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-26T08:34:12.260 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-03-26T08:34:12.330 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:12 smithi043 ceph-mon[31929]: pgmap v461: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:34:12.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:34:12.462 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-03-26T08:34:12.511 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:12 smithi183 ceph-mon[35818]: pgmap v461: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:34:12.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:34:12.559 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list-id 2024-03-26T08:34:12.571 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:12 smithi052 ceph-mon[35071]: pgmap v461: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:34:12.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:34:12.612 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-03-26T08:34:12.612 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-03-26T08:34:12.612 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-26T08:34:12.612 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-26T08:34:12.612 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-03-26T08:34:12.752 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-26T08:34:12.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:34:12.752 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-03-26T08:34:12.752 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-03-26T08:34:12.752 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-26T08:34:12.752 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-26T08:34:12.752 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-26T08:34:12.752 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-26T08:34:12.753 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-26T08:34:12.753 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-03-26T08:34:12.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:34:12.991 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-03-26T08:34:12.991 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-03-26T08:34:12.991 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-03-26T08:34:12.991 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set dev brx.0 master ceph-brx 2024-03-26T08:34:12.991 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-03-26T08:34:13.142 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-03-26T08:34:13.142 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-26T08:34:13.142 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-26T08:34:13.181 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-26T08:34:13.181 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-26T08:34:13.181 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-26T08:34:13.263 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-03-26T08:34:13.378 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-03-26T08:34:13.424 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-03-26T08:34:13.424 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-03-26T08:34:13.424 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-03-26T08:34:13.424 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-03-26T08:34:13.424 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-03-26T08:34:13.424 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-03-26T08:34:13.425 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-03-26T08:34:13.425 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-03-26T08:34:13.425 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-03-26T08:34:13.425 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-03-26T08:34:13.425 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-03-26T08:34:13.425 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-03-26T08:34:13.425 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T08:34:13.425 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-03-26T08:34:13.425 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-03-26T08:34:13.425 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-03-26T08:34:13.425 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-03-26T08:34:13.425 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-03-26T08:34:13.425 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-03-26T08:34:13.425 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-03-26T08:34:13.425 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-26T08:34:13.426 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-26T08:34:13.426 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-03-26T08:34:13.426 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-03-26T08:34:13.426 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-03-26T08:34:13.426 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay 2024-03-26T08:34:13.426 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/11ba0c83ec55690d45669d206dc55ba30613139ea80436ae3cdd46b540253e72/merged 2024-03-26T08:34:13.426 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/97fe696f1e34509dc86041c67a396fdc77cda785578e668898e68d28eba150cf/merged 2024-03-26T08:34:13.426 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/0 2024-03-26T08:34:13.426 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T08:34:13.426 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/0a3b315194a3cd2e278c3c92d31fc70810e246de129c4a15e55d867020f0c057/merged 2024-03-26T08:34:13.426 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/46b0401bbe4775455adde9cb89058f287288315f170bda4d9d26af266da5aa21/merged 2024-03-26T08:34:13.426 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/5211fce819420e40b58fbf4975b87b4825d6372f9738c6605df2a392286e3f4b/merged 2024-03-26T08:34:13.426 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/6ad88ba55ea27f30c7b0ec77c84d2e5453e87a62bc39a75368ba826be2b1291d/merged 2024-03-26T08:34:13.426 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/d013144c20b550c06081475d3e46065f3f4d252dabc08af06aca254c47a85b50/merged 2024-03-26T08:34:13.426 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/33029eb686fc515d3063e2ff583bcbd0a8ab9c15101baea9ac8c066ce5d23744/merged 2024-03-26T08:34:13.427 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/165e9b1d609cd90aa0578b0fe457f616fabf6ef44c82e53a0fad1bea35209e21/merged 2024-03-26T08:34:13.427 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/24e73d122c85dd284aec81e66c7543cd2bba3cee3b499ae61f083629a49d29fe/merged 2024-03-26T08:34:13.427 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-03-26T08:34:13.427 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-26T08:34:13.427 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-26T08:34:13.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:34:13.428 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-03-26T08:34:13.512 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-26T08:34:13.512 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/32b40a1c-a81f-4074-a091-7196526b98fa --client_fs=cephfs) 2024-03-26T08:34:13.560 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-03-26T08:34:13.641 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-03-26T08:34:13.686 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-03-26T08:34:13.686 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-03-26T08:34:13.686 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-03-26T08:34:13.687 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-03-26T08:34:13.687 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-03-26T08:34:13.687 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-03-26T08:34:13.687 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-03-26T08:34:13.687 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-03-26T08:34:13.687 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-03-26T08:34:13.687 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-03-26T08:34:13.687 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-03-26T08:34:13.687 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-03-26T08:34:13.687 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T08:34:13.687 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-03-26T08:34:13.687 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-03-26T08:34:13.687 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-03-26T08:34:13.687 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-03-26T08:34:13.688 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-03-26T08:34:13.688 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-03-26T08:34:13.688 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-03-26T08:34:13.688 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-26T08:34:13.688 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-26T08:34:13.688 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-03-26T08:34:13.688 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-03-26T08:34:13.688 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-03-26T08:34:13.688 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay 2024-03-26T08:34:13.688 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/11ba0c83ec55690d45669d206dc55ba30613139ea80436ae3cdd46b540253e72/merged 2024-03-26T08:34:13.688 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/97fe696f1e34509dc86041c67a396fdc77cda785578e668898e68d28eba150cf/merged 2024-03-26T08:34:13.688 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/0 2024-03-26T08:34:13.688 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T08:34:13.688 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/0a3b315194a3cd2e278c3c92d31fc70810e246de129c4a15e55d867020f0c057/merged 2024-03-26T08:34:13.688 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/46b0401bbe4775455adde9cb89058f287288315f170bda4d9d26af266da5aa21/merged 2024-03-26T08:34:13.689 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/5211fce819420e40b58fbf4975b87b4825d6372f9738c6605df2a392286e3f4b/merged 2024-03-26T08:34:13.689 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/6ad88ba55ea27f30c7b0ec77c84d2e5453e87a62bc39a75368ba826be2b1291d/merged 2024-03-26T08:34:13.689 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/d013144c20b550c06081475d3e46065f3f4d252dabc08af06aca254c47a85b50/merged 2024-03-26T08:34:13.689 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/33029eb686fc515d3063e2ff583bcbd0a8ab9c15101baea9ac8c066ce5d23744/merged 2024-03-26T08:34:13.689 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/165e9b1d609cd90aa0578b0fe457f616fabf6ef44c82e53a0fad1bea35209e21/merged 2024-03-26T08:34:13.689 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/24e73d122c85dd284aec81e66c7543cd2bba3cee3b499ae61f083629a49d29fe/merged 2024-03-26T08:34:13.689 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-03-26T08:34:13.689 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-26T08:34:13.689 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-26T08:34:13.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:34:13.690 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-03-26T08:34:13.760 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2024-03-26T08:34:13.767+0000 7f3a0cc44480 -1 init, newargv = 0x55a1a0b864a0 newargc=15 2024-03-26T08:34:13.760 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[83304]: starting ceph client 2024-03-26T08:34:13.778 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[83304]: starting fuse 2024-03-26T08:34:14.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:14 smithi043 ceph-mon[31929]: pgmap v462: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:14.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:14 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:34:14.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:14 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:34:14.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:14 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:34:14.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:14 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:34:14.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:14 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:34:14.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:14 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:34:14.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:14 smithi183 ceph-mon[35818]: pgmap v462: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:14.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:14 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:34:14.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:14 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:34:14.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:14 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:34:14.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:14 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:34:14.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:14 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:34:14.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:14 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:34:14.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:14 smithi052 ceph-mon[35071]: pgmap v462: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:14.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:14 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:34:14.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:14 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:34:14.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:14 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:34:14.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:14 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:34:14.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:14 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:34:14.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:14 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:34:14.872 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-03-26T08:34:14.991 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-03-26T08:34:15.048 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-03-26T08:34:15.048 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-03-26T08:34:15.048 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-03-26T08:34:15.048 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-03-26T08:34:15.049 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-03-26T08:34:15.049 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-03-26T08:34:15.049 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-03-26T08:34:15.049 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-03-26T08:34:15.049 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-03-26T08:34:15.049 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-03-26T08:34:15.049 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-03-26T08:34:15.049 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-03-26T08:34:15.049 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T08:34:15.049 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-03-26T08:34:15.049 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-03-26T08:34:15.049 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-03-26T08:34:15.049 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-03-26T08:34:15.050 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-03-26T08:34:15.050 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-03-26T08:34:15.050 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-03-26T08:34:15.050 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-26T08:34:15.050 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-26T08:34:15.050 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-03-26T08:34:15.050 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-03-26T08:34:15.050 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-03-26T08:34:15.050 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay 2024-03-26T08:34:15.050 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/11ba0c83ec55690d45669d206dc55ba30613139ea80436ae3cdd46b540253e72/merged 2024-03-26T08:34:15.050 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/97fe696f1e34509dc86041c67a396fdc77cda785578e668898e68d28eba150cf/merged 2024-03-26T08:34:15.050 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/0 2024-03-26T08:34:15.050 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T08:34:15.050 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/0a3b315194a3cd2e278c3c92d31fc70810e246de129c4a15e55d867020f0c057/merged 2024-03-26T08:34:15.051 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/46b0401bbe4775455adde9cb89058f287288315f170bda4d9d26af266da5aa21/merged 2024-03-26T08:34:15.051 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/5211fce819420e40b58fbf4975b87b4825d6372f9738c6605df2a392286e3f4b/merged 2024-03-26T08:34:15.051 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/6ad88ba55ea27f30c7b0ec77c84d2e5453e87a62bc39a75368ba826be2b1291d/merged 2024-03-26T08:34:15.051 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/d013144c20b550c06081475d3e46065f3f4d252dabc08af06aca254c47a85b50/merged 2024-03-26T08:34:15.051 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/33029eb686fc515d3063e2ff583bcbd0a8ab9c15101baea9ac8c066ce5d23744/merged 2024-03-26T08:34:15.051 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/165e9b1d609cd90aa0578b0fe457f616fabf6ef44c82e53a0fad1bea35209e21/merged 2024-03-26T08:34:15.051 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/24e73d122c85dd284aec81e66c7543cd2bba3cee3b499ae61f083629a49d29fe/merged 2024-03-26T08:34:15.051 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-03-26T08:34:15.051 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-26T08:34:15.051 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-26T08:34:15.051 INFO:teuthology.orchestra.run.smithi043.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-26T08:34:15.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:34:15.054 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-03-26T08:34:15.097 INFO:teuthology.orchestra.run.smithi043.stdout:118 2024-03-26T08:34:15.098 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [118] 2024-03-26T08:34:15.098 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2024-03-26T08:34:15.098 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2024-03-26T08:34:15.098 DEBUG:teuthology.orchestra.run.smithi043:> import re 2024-03-26T08:34:15.098 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-03-26T08:34:15.098 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2024-03-26T08:34:15.098 DEBUG:teuthology.orchestra.run.smithi043:> 2024-03-26T08:34:15.098 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2024-03-26T08:34:15.098 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-26T08:34:15.098 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2024-03-26T08:34:15.099 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-26T08:34:15.099 DEBUG:teuthology.orchestra.run.smithi043:> 2024-03-26T08:34:15.099 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2024-03-26T08:34:15.099 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2024-03-26T08:34:15.099 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2024-03-26T08:34:15.099 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2024-03-26T08:34:15.099 DEBUG:teuthology.orchestra.run.smithi043:> 2024-03-26T08:34:15.099 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2024-03-26T08:34:15.099 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-26T08:34:15.099 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-26T08:34:15.099 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-26T08:34:15.099 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2024-03-26T08:34:15.099 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2024-03-26T08:34:15.099 DEBUG:teuthology.orchestra.run.smithi043:> return f 2024-03-26T08:34:15.099 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-26T08:34:15.099 DEBUG:teuthology.orchestra.run.smithi043:> 2024-03-26T08:34:15.099 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2024-03-26T08:34:15.099 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-03-26T08:34:15.224 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:15 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:15.224 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:15 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:34:15.224 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:15 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:34:15.323 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.83304.asok 2024-03-26T08:34:15.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:34:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T08:34:15.342 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.83304.asok 2024-03-26T08:34:15.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:34:15.342 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.83304.asok status 2024-03-26T08:34:15.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:15 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:15.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:15 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:34:15.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:15 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:34:15.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:15 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:15.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:15 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:34:15.573 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:15 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:34:15.584 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-26T08:34:15.584 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2024-03-26T08:34:15.584 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "3be9a5c9bc793e11e2800a8c0c696e8b46742033", 2024-03-26T08:34:15.585 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-26T08:34:15.585 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2024-03-26T08:34:15.585 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2024-03-26T08:34:15.585 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-26T08:34:15.585 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "83304", 2024-03-26T08:34:15.585 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/volumes/qa/sv_0/32b40a1c-a81f-4074-a091-7196526b98fa" 2024-03-26T08:34:15.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:34:15.585 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2024-03-26T08:34:15.585 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2024-03-26T08:34:15.585 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 15408, 2024-03-26T08:34:15.586 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2024-03-26T08:34:15.586 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2024-03-26T08:34:15.586 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2024-03-26T08:34:15.586 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 15408 2024-03-26T08:34:15.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:34:15.586 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-03-26T08:34:15.586 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-03-26T08:34:15.586 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-03-26T08:34:15.586 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2194131274 2024-03-26T08:34:15.586 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:34:15.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:34:15.586 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-03-26T08:34:15.586 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-03-26T08:34:15.586 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-03-26T08:34:15.587 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2194131274 2024-03-26T08:34:15.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:34:15.587 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.15408 192.168.0.1:0/2194131274", 2024-03-26T08:34:15.587 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/2194131274", 2024-03-26T08:34:15.587 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 5, 2024-03-26T08:34:15.587 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 21, 2024-03-26T08:34:15.587 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 94, 2024-03-26T08:34:15.587 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2024-03-26T08:34:15.587 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2024-03-26T08:34:15.587 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2024-03-26T08:34:15.587 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-26T08:34:15.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:34:15.600 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2024-03-26T08:34:15.600 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-26T08:34:15.657 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2024-03-26T08:34:15.657 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-26T08:34:15.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:34:15.658 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-26T08:34:15.750 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-26T08:34:15.763 INFO:teuthology.run_tasks:Running task workunit... 2024-03-26T08:34:15.797 INFO:tasks.workunit:Pulling workunits from ref 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T08:34:15.798 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-26T08:34:15.798 INFO:tasks.workunit:timeout=3h 2024-03-26T08:34:15.798 INFO:tasks.workunit:cleanup=True 2024-03-26T08:34:15.798 DEBUG:teuthology.orchestra.run.smithi043:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-26T08:34:15.858 INFO:teuthology.orchestra.run.smithi043.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-26T08:34:15.858 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-26T08:34:15.858 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 76h/118d Inode: 1 Links: 2 2024-03-26T08:34:15.858 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T08:34:15.859 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:fusefs_t:s0 2024-03-26T08:34:15.859 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-03-26 08:31:49.623619538 +0000 2024-03-26T08:34:15.859 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-03-26 08:31:49.623619538 +0000 2024-03-26T08:34:15.859 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-03-26 08:34:15.751818823 +0000 2024-03-26T08:34:15.859 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: - 2024-03-26T08:34:15.859 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-26T08:34:15.859 DEBUG:teuthology.orchestra.run.smithi043:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-26T08:34:15.979 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T08:34:16.039 INFO:tasks.workunit.client.0.smithi043.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-26T08:34:16.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:16 smithi043 ceph-mon[31929]: pgmap v463: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:16.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:16 smithi183 ceph-mon[35818]: pgmap v463: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:16.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:16 smithi052 ceph-mon[35071]: pgmap v463: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:18.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:18 smithi183 ceph-mon[35818]: pgmap v464: 129 pgs: 129 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T08:34:18.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:18 smithi052 ceph-mon[35071]: pgmap v464: 129 pgs: 129 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T08:34:18.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:18 smithi043 ceph-mon[31929]: pgmap v464: 129 pgs: 129 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-26T08:34:20.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:20 smithi043 ceph-mon[31929]: pgmap v465: 129 pgs: 129 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-03-26T08:34:20.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:20 smithi183 ceph-mon[35818]: pgmap v465: 129 pgs: 129 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-03-26T08:34:20.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:20 smithi052 ceph-mon[35071]: pgmap v465: 129 pgs: 129 active+clean; 639 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-03-26T08:34:22.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:22 smithi183 ceph-mon[35818]: pgmap v466: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 938 B/s wr, 3 op/s 2024-03-26T08:34:22.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:22 smithi052 ceph-mon[35071]: pgmap v466: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 938 B/s wr, 3 op/s 2024-03-26T08:34:22.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:22 smithi043 ceph-mon[31929]: pgmap v466: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 938 B/s wr, 3 op/s 2024-03-26T08:34:24.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:24 smithi183 ceph-mon[35818]: pgmap v467: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-26T08:34:24.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:24 smithi052 ceph-mon[35071]: pgmap v467: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-26T08:34:24.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:24 smithi043 ceph-mon[31929]: pgmap v467: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-26T08:34:26.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:26 smithi183 ceph-mon[35818]: pgmap v468: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-26T08:34:26.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:26 smithi052 ceph-mon[35071]: pgmap v468: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-26T08:34:26.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:26 smithi043 ceph-mon[31929]: pgmap v468: 129 pgs: 129 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-26T08:34:28.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:28 smithi183 ceph-mon[35818]: pgmap v469: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-03-26T08:34:28.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:28 smithi052 ceph-mon[35071]: pgmap v469: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-03-26T08:34:28.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:28 smithi043 ceph-mon[31929]: pgmap v469: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-03-26T08:34:30.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:30 smithi183 ceph-mon[35818]: pgmap v470: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-03-26T08:34:30.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:30 smithi052 ceph-mon[35071]: pgmap v470: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-03-26T08:34:30.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:30 smithi043 ceph-mon[31929]: pgmap v470: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-03-26T08:34:32.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:32 smithi183 ceph-mon[35818]: pgmap v471: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-03-26T08:34:32.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:32 smithi052 ceph-mon[35071]: pgmap v471: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-03-26T08:34:32.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:32 smithi043 ceph-mon[31929]: pgmap v471: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-03-26T08:34:34.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:34 smithi183 ceph-mon[35818]: pgmap v472: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-26T08:34:34.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:34 smithi052 ceph-mon[35071]: pgmap v472: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-26T08:34:34.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:34 smithi043 ceph-mon[31929]: pgmap v472: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-26T08:34:36.478 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:36 smithi043 ceph-mon[31929]: pgmap v473: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-26T08:34:36.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:36 smithi183 ceph-mon[35818]: pgmap v473: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-26T08:34:36.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:36 smithi052 ceph-mon[35071]: pgmap v473: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-26T08:34:38.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:38 smithi043 ceph-mon[31929]: pgmap v474: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-26T08:34:38.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:38 smithi183 ceph-mon[35818]: pgmap v474: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-26T08:34:38.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:38 smithi052 ceph-mon[35071]: pgmap v474: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-26T08:34:40.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:40 smithi183 ceph-mon[35818]: pgmap v475: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:40.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:40 smithi052 ceph-mon[35071]: pgmap v475: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:40.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:40 smithi043 ceph-mon[31929]: pgmap v475: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:42.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:42 smithi183 ceph-mon[35818]: pgmap v476: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:34:42.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:42 smithi052 ceph-mon[35071]: pgmap v476: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:34:42.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:42 smithi043 ceph-mon[31929]: pgmap v476: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:34:44.548 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:44 smithi183 ceph-mon[35818]: pgmap v477: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:44.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:44 smithi052 ceph-mon[35071]: pgmap v477: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:44.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:44 smithi043 ceph-mon[31929]: pgmap v477: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:45.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:45 smithi052 ceph-mon[35071]: pgmap v478: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:45.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:45 smithi043 ceph-mon[31929]: pgmap v478: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:45.797 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:45 smithi183 ceph-mon[35818]: pgmap v478: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:47.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:47 smithi183 ceph-mon[35818]: pgmap v479: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:34:47.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:47 smithi052 ceph-mon[35071]: pgmap v479: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:34:47.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:47 smithi043 ceph-mon[31929]: pgmap v479: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:34:49.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:49 smithi183 ceph-mon[35818]: pgmap v480: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:49.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:49 smithi052 ceph-mon[35071]: pgmap v480: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:49.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:49 smithi043 ceph-mon[31929]: pgmap v480: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:51.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:51 smithi183 ceph-mon[35818]: pgmap v481: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:34:51.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:51 smithi052 ceph-mon[35071]: pgmap v481: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:34:51.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:51 smithi043 ceph-mon[31929]: pgmap v481: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:34:53.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:53 smithi183 ceph-mon[35818]: pgmap v482: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:53.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:53 smithi052 ceph-mon[35071]: pgmap v482: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:53.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:53 smithi043 ceph-mon[31929]: pgmap v482: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:55.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:55 smithi183 ceph-mon[35818]: pgmap v483: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:55.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:55 smithi052 ceph-mon[35071]: pgmap v483: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:55.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:55 smithi043 ceph-mon[31929]: pgmap v483: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:57.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:57 smithi183 ceph-mon[35818]: pgmap v484: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:34:57.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:57 smithi052 ceph-mon[35071]: pgmap v484: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:34:57.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:57 smithi043 ceph-mon[31929]: pgmap v484: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:34:59.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:34:59 smithi183 ceph-mon[35818]: pgmap v485: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:59.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:34:59 smithi052 ceph-mon[35071]: pgmap v485: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:34:59.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:34:59 smithi043 ceph-mon[31929]: pgmap v485: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:35:01.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:01 smithi183 ceph-mon[35818]: pgmap v486: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:35:01.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:01 smithi052 ceph-mon[35071]: pgmap v486: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:35:01.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:01 smithi043 ceph-mon[31929]: pgmap v486: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:35:03.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:03 smithi183 ceph-mon[35818]: pgmap v487: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:35:03.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:03 smithi052 ceph-mon[35071]: pgmap v487: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:35:03.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:03 smithi043 ceph-mon[31929]: pgmap v487: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:35:05.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:05 smithi183 ceph-mon[35818]: pgmap v488: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:35:05.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:05 smithi052 ceph-mon[35071]: pgmap v488: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:35:05.900 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:05 smithi043 ceph-mon[31929]: pgmap v488: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:35:07.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:07 smithi183 ceph-mon[35818]: pgmap v489: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:35:07.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:07 smithi052 ceph-mon[35071]: pgmap v489: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:35:07.900 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:07 smithi043 ceph-mon[31929]: pgmap v489: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:35:09.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:09 smithi183 ceph-mon[35818]: pgmap v490: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:35:09.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:09 smithi052 ceph-mon[35071]: pgmap v490: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:35:09.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:09 smithi043 ceph-mon[31929]: pgmap v490: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:35:11.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:11 smithi183 ceph-mon[35818]: pgmap v491: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:35:11.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:11 smithi052 ceph-mon[35071]: pgmap v491: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:35:11.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:11 smithi043 ceph-mon[31929]: pgmap v491: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:35:13.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:13 smithi052 ceph-mon[35071]: pgmap v492: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-26T08:35:13.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:13 smithi043 ceph-mon[31929]: pgmap v492: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-26T08:35:14.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:13 smithi183 ceph-mon[35818]: pgmap v492: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-26T08:35:15.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:15 smithi052 ceph-mon[35071]: pgmap v493: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-26T08:35:15.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:15 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:35:15.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:15 smithi043 ceph-mon[31929]: pgmap v493: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-26T08:35:15.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:15 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:35:16.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:15 smithi183 ceph-mon[35818]: pgmap v493: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-26T08:35:16.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:15 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:35:17.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:17 smithi052 ceph-mon[35071]: pgmap v494: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:35:17.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:17 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:17.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:17 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:35:17.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:17 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:35:17.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:17 smithi043 ceph-mon[31929]: pgmap v494: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:35:17.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:17 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:17.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:17 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:35:17.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:17 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:35:18.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:17 smithi183 ceph-mon[35818]: pgmap v494: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:35:18.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:17 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:18.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:17 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:35:18.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:17 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:35:19.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:19 smithi052 ceph-mon[35071]: pgmap v495: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-26T08:35:19.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:19 smithi043 ceph-mon[31929]: pgmap v495: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-26T08:35:20.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:19 smithi183 ceph-mon[35818]: pgmap v495: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-26T08:35:21.822 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:21 smithi052 ceph-mon[35071]: pgmap v496: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:35:21.900 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:21 smithi043 ceph-mon[31929]: pgmap v496: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:35:22.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:21 smithi183 ceph-mon[35818]: pgmap v496: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:35:23.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:23 smithi043 ceph-mon[31929]: pgmap v497: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-26T08:35:24.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:23 smithi183 ceph-mon[35818]: pgmap v497: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-26T08:35:24.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:23 smithi052 ceph-mon[35071]: pgmap v497: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-26T08:35:25.850 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:25 smithi183 ceph-mon[35818]: pgmap v498: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-26T08:35:25.900 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:25 smithi043 ceph-mon[31929]: pgmap v498: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-26T08:35:26.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:25 smithi052 ceph-mon[35071]: pgmap v498: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-26T08:35:27.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:27 smithi043 ceph-mon[31929]: pgmap v499: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:35:28.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:27 smithi183 ceph-mon[35818]: pgmap v499: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:35:28.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:27 smithi052 ceph-mon[35071]: pgmap v499: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:35:29.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:29 smithi043 ceph-mon[31929]: pgmap v500: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-03-26T08:35:30.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:29 smithi183 ceph-mon[35818]: pgmap v500: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-03-26T08:35:30.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:29 smithi052 ceph-mon[35071]: pgmap v500: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-03-26T08:35:31.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:31 smithi043 ceph-mon[31929]: pgmap v501: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:35:32.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:31 smithi183 ceph-mon[35818]: pgmap v501: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:35:32.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:31 smithi052 ceph-mon[35071]: pgmap v501: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:35:33.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:33 smithi043 ceph-mon[31929]: pgmap v502: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-03-26T08:35:34.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:33 smithi183 ceph-mon[35818]: pgmap v502: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-03-26T08:35:34.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:33 smithi052 ceph-mon[35071]: pgmap v502: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-03-26T08:35:35.874 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:35 smithi052 ceph-mon[35071]: pgmap v503: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-03-26T08:35:35.900 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:35 smithi043 ceph-mon[31929]: pgmap v503: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-03-26T08:35:36.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:35 smithi183 ceph-mon[35818]: pgmap v503: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-03-26T08:35:37.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:37 smithi043 ceph-mon[31929]: pgmap v504: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:35:38.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:37 smithi183 ceph-mon[35818]: pgmap v504: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:35:38.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:37 smithi052 ceph-mon[35071]: pgmap v504: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:35:39.852 INFO:tasks.workunit.client.0.smithi043.stderr:Updating files: 59% (7291/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 99% (12172/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-26T08:35:39.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:39 smithi043 ceph-mon[31929]: pgmap v505: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-26T08:35:40.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:39 smithi183 ceph-mon[35818]: pgmap v505: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-26T08:35:40.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:39 smithi052 ceph-mon[35071]: pgmap v505: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-26T08:35:40.653 INFO:tasks.workunit.client.0.smithi043.stderr:Note: switching to '3be9a5c9bc793e11e2800a8c0c696e8b46742033'. 2024-03-26T08:35:40.653 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-03-26T08:35:40.653 INFO:tasks.workunit.client.0.smithi043.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-26T08:35:40.653 INFO:tasks.workunit.client.0.smithi043.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-26T08:35:40.653 INFO:tasks.workunit.client.0.smithi043.stderr:state without impacting any branches by switching back to a branch. 2024-03-26T08:35:40.653 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-03-26T08:35:40.653 INFO:tasks.workunit.client.0.smithi043.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-26T08:35:40.653 INFO:tasks.workunit.client.0.smithi043.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-26T08:35:40.653 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-03-26T08:35:40.653 INFO:tasks.workunit.client.0.smithi043.stderr: git switch -c 2024-03-26T08:35:40.653 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-03-26T08:35:40.654 INFO:tasks.workunit.client.0.smithi043.stderr:Or undo this operation with: 2024-03-26T08:35:40.654 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-03-26T08:35:40.654 INFO:tasks.workunit.client.0.smithi043.stderr: git switch - 2024-03-26T08:35:40.654 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-03-26T08:35:40.654 INFO:tasks.workunit.client.0.smithi043.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-26T08:35:40.654 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-03-26T08:35:40.654 INFO:tasks.workunit.client.0.smithi043.stderr:HEAD is now at 3be9a5c9bc7 dummy commit #1 2024-03-26T08:35:40.663 DEBUG:teuthology.orchestra.run.smithi043:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-26T08:35:40.735 INFO:tasks.workunit.client.0.smithi043.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-26T08:35:40.744 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-26T08:35:40.744 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-26T08:35:41.712 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-26T08:35:41.858 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-26T08:35:41.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:41 smithi043 ceph-mon[31929]: pgmap v506: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:35:41.952 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-26T08:35:41.958 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-26T08:35:41.958 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-26T08:35:42.045 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-26T08:35:42.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:41 smithi183 ceph-mon[35818]: pgmap v506: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:35:42.055 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-26T08:35:42.055 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-26T08:35:42.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:41 smithi052 ceph-mon[35071]: pgmap v506: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-26T08:35:42.109 INFO:tasks.workunit:Running workunits matching suites/iozone.sh on client.0... 2024-03-26T08:35:42.110 INFO:tasks.workunit:Running workunit suites/iozone.sh... 2024-03-26T08:35:42.111 DEBUG:teuthology.orchestra.run.smithi043:workunit test suites/iozone.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh 2024-03-26T08:35:42.216 INFO:tasks.workunit.client.0.smithi043.stderr:+ iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-03-26T08:35:42.337 INFO:tasks.workunit.client.0.smithi043.stdout: Iozone: Performance Test of File I/O 2024-03-26T08:35:42.337 INFO:tasks.workunit.client.0.smithi043.stdout: Version $Revision: 3.494 $ 2024-03-26T08:35:42.337 INFO:tasks.workunit.client.0.smithi043.stdout: Compiled for 64 bit mode. 2024-03-26T08:35:42.337 INFO:tasks.workunit.client.0.smithi043.stdout: Build: linux-AMD64 2024-03-26T08:35:42.337 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:35:42.337 INFO:tasks.workunit.client.0.smithi043.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-03-26T08:35:42.337 INFO:tasks.workunit.client.0.smithi043.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-03-26T08:35:42.337 INFO:tasks.workunit.client.0.smithi043.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-03-26T08:35:42.337 INFO:tasks.workunit.client.0.smithi043.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-03-26T08:35:42.337 INFO:tasks.workunit.client.0.smithi043.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-03-26T08:35:42.337 INFO:tasks.workunit.client.0.smithi043.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-03-26T08:35:42.338 INFO:tasks.workunit.client.0.smithi043.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-03-26T08:35:42.338 INFO:tasks.workunit.client.0.smithi043.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-03-26T08:35:42.338 INFO:tasks.workunit.client.0.smithi043.stdout: Alexey Skidanov, Sudhir Kumar. 2024-03-26T08:35:42.338 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:35:42.338 INFO:tasks.workunit.client.0.smithi043.stdout: Run began: Tue Mar 26 08:35:42 2024 2024-03-26T08:35:42.338 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:35:42.338 INFO:tasks.workunit.client.0.smithi043.stdout: Include close in write timing 2024-03-26T08:35:42.338 INFO:tasks.workunit.client.0.smithi043.stdout: Include fsync in write timing 2024-03-26T08:35:42.338 INFO:tasks.workunit.client.0.smithi043.stdout: File size set to 1048576 kB 2024-03-26T08:35:42.338 INFO:tasks.workunit.client.0.smithi043.stdout: Record Size 16 kB 2024-03-26T08:35:42.338 INFO:tasks.workunit.client.0.smithi043.stdout: Command line used: iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-03-26T08:35:42.338 INFO:tasks.workunit.client.0.smithi043.stdout: Output is in kBytes/sec 2024-03-26T08:35:42.338 INFO:tasks.workunit.client.0.smithi043.stdout: Time Resolution = 0.000001 seconds. 2024-03-26T08:35:42.338 INFO:tasks.workunit.client.0.smithi043.stdout: Processor cache size set to 1024 kBytes. 2024-03-26T08:35:42.338 INFO:tasks.workunit.client.0.smithi043.stdout: Processor cache line size set to 32 bytes. 2024-03-26T08:35:42.339 INFO:tasks.workunit.client.0.smithi043.stdout: File stride size set to 17 * record size. 2024-03-26T08:35:42.339 INFO:tasks.workunit.client.0.smithi043.stdout: Throughput test with 1 process 2024-03-26T08:35:42.339 INFO:tasks.workunit.client.0.smithi043.stdout: Each process writes a 1048576 kByte file in 16 kByte records 2024-03-26T08:35:43.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:43 smithi043 ceph-mon[31929]: pgmap v507: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:35:44.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:43 smithi183 ceph-mon[35818]: pgmap v507: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:35:44.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:43 smithi052 ceph-mon[35071]: pgmap v507: 129 pgs: 129 active+clean; 654 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-03-26T08:35:45.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:45 smithi043 ceph-mon[31929]: pgmap v508: 129 pgs: 129 active+clean; 8.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 682 KiB/s wr, 1 op/s 2024-03-26T08:35:46.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:45 smithi183 ceph-mon[35818]: pgmap v508: 129 pgs: 129 active+clean; 8.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 682 KiB/s wr, 1 op/s 2024-03-26T08:35:46.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:45 smithi052 ceph-mon[35071]: pgmap v508: 129 pgs: 129 active+clean; 8.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 682 KiB/s wr, 1 op/s 2024-03-26T08:35:47.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:47 smithi043 ceph-mon[31929]: pgmap v509: 129 pgs: 129 active+clean; 37 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-03-26T08:35:48.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:47 smithi183 ceph-mon[35818]: pgmap v509: 129 pgs: 129 active+clean; 37 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-03-26T08:35:48.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:47 smithi052 ceph-mon[35071]: pgmap v509: 129 pgs: 129 active+clean; 37 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 3.0 MiB/s wr, 4 op/s 2024-03-26T08:35:49.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:49 smithi043 ceph-mon[31929]: pgmap v510: 129 pgs: 129 active+clean; 45 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 3.7 MiB/s wr, 3 op/s 2024-03-26T08:35:50.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:49 smithi183 ceph-mon[35818]: pgmap v510: 129 pgs: 129 active+clean; 45 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 3.7 MiB/s wr, 3 op/s 2024-03-26T08:35:50.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:49 smithi052 ceph-mon[35071]: pgmap v510: 129 pgs: 129 active+clean; 45 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 3.7 MiB/s wr, 3 op/s 2024-03-26T08:35:51.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:51 smithi043 ceph-mon[31929]: pgmap v511: 129 pgs: 129 active+clean; 105 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 8.7 MiB/s wr, 6 op/s 2024-03-26T08:35:52.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:51 smithi183 ceph-mon[35818]: pgmap v511: 129 pgs: 129 active+clean; 105 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 8.7 MiB/s wr, 6 op/s 2024-03-26T08:35:52.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:51 smithi052 ceph-mon[35071]: pgmap v511: 129 pgs: 129 active+clean; 105 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 8.7 MiB/s wr, 6 op/s 2024-03-26T08:35:54.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:53 smithi183 ceph-mon[35818]: pgmap v512: 129 pgs: 129 active+clean; 129 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 MiB/s wr, 6 op/s 2024-03-26T08:35:54.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:53 smithi052 ceph-mon[35071]: pgmap v512: 129 pgs: 129 active+clean; 129 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 MiB/s wr, 6 op/s 2024-03-26T08:35:54.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:53 smithi043 ceph-mon[31929]: pgmap v512: 129 pgs: 129 active+clean; 129 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 MiB/s wr, 6 op/s 2024-03-26T08:35:56.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:55 smithi183 ceph-mon[35818]: pgmap v513: 129 pgs: 129 active+clean; 145 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 MiB/s wr, 6 op/s 2024-03-26T08:35:56.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:55 smithi052 ceph-mon[35071]: pgmap v513: 129 pgs: 129 active+clean; 145 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 MiB/s wr, 6 op/s 2024-03-26T08:35:56.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:55 smithi043 ceph-mon[31929]: pgmap v513: 129 pgs: 129 active+clean; 145 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 MiB/s wr, 6 op/s 2024-03-26T08:35:58.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:57 smithi183 ceph-mon[35818]: pgmap v514: 129 pgs: 129 active+clean; 197 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 16 MiB/s wr, 8 op/s 2024-03-26T08:35:58.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:57 smithi052 ceph-mon[35071]: pgmap v514: 129 pgs: 129 active+clean; 197 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 16 MiB/s wr, 8 op/s 2024-03-26T08:35:58.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:57 smithi043 ceph-mon[31929]: pgmap v514: 129 pgs: 129 active+clean; 197 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 16 MiB/s wr, 8 op/s 2024-03-26T08:36:00.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:35:59 smithi183 ceph-mon[35818]: pgmap v515: 129 pgs: 129 active+clean; 201 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 14 MiB/s wr, 5 op/s 2024-03-26T08:36:00.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:35:59 smithi052 ceph-mon[35071]: pgmap v515: 129 pgs: 129 active+clean; 201 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 14 MiB/s wr, 5 op/s 2024-03-26T08:36:00.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:35:59 smithi043 ceph-mon[31929]: pgmap v515: 129 pgs: 129 active+clean; 201 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 14 MiB/s wr, 5 op/s 2024-03-26T08:36:02.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:01 smithi183 ceph-mon[35818]: pgmap v516: 129 pgs: 129 active+clean; 277 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 19 MiB/s wr, 8 op/s 2024-03-26T08:36:02.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:01 smithi052 ceph-mon[35071]: pgmap v516: 129 pgs: 129 active+clean; 277 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 19 MiB/s wr, 8 op/s 2024-03-26T08:36:02.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:01 smithi043 ceph-mon[31929]: pgmap v516: 129 pgs: 129 active+clean; 277 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 19 MiB/s wr, 8 op/s 2024-03-26T08:36:04.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:03 smithi183 ceph-mon[35818]: pgmap v517: 129 pgs: 129 active+clean; 297 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 16 MiB/s wr, 6 op/s 2024-03-26T08:36:04.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:03 smithi052 ceph-mon[35071]: pgmap v517: 129 pgs: 129 active+clean; 297 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 16 MiB/s wr, 6 op/s 2024-03-26T08:36:04.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:03 smithi043 ceph-mon[31929]: pgmap v517: 129 pgs: 129 active+clean; 297 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 16 MiB/s wr, 6 op/s 2024-03-26T08:36:06.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:05 smithi183 ceph-mon[35818]: pgmap v518: 129 pgs: 129 active+clean; 333 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 17 MiB/s wr, 6 op/s 2024-03-26T08:36:06.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:05 smithi052 ceph-mon[35071]: pgmap v518: 129 pgs: 129 active+clean; 333 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 17 MiB/s wr, 6 op/s 2024-03-26T08:36:06.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:05 smithi043 ceph-mon[31929]: pgmap v518: 129 pgs: 129 active+clean; 333 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 17 MiB/s wr, 6 op/s 2024-03-26T08:36:08.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:07 smithi183 ceph-mon[35818]: pgmap v519: 129 pgs: 129 active+clean; 397 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 21 MiB/s wr, 8 op/s 2024-03-26T08:36:08.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:07 smithi052 ceph-mon[35071]: pgmap v519: 129 pgs: 129 active+clean; 397 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 21 MiB/s wr, 8 op/s 2024-03-26T08:36:08.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:07 smithi043 ceph-mon[31929]: pgmap v519: 129 pgs: 129 active+clean; 397 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 21 MiB/s wr, 8 op/s 2024-03-26T08:36:10.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:09 smithi183 ceph-mon[35818]: pgmap v520: 129 pgs: 129 active+clean; 405 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 17 MiB/s wr, 6 op/s 2024-03-26T08:36:10.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:09 smithi052 ceph-mon[35071]: pgmap v520: 129 pgs: 129 active+clean; 405 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 17 MiB/s wr, 6 op/s 2024-03-26T08:36:10.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:09 smithi043 ceph-mon[31929]: pgmap v520: 129 pgs: 129 active+clean; 405 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 17 MiB/s wr, 6 op/s 2024-03-26T08:36:12.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:11 smithi183 ceph-mon[35818]: pgmap v521: 129 pgs: 129 active+clean; 445 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 20 MiB/s wr, 8 op/s 2024-03-26T08:36:12.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:11 smithi052 ceph-mon[35071]: pgmap v521: 129 pgs: 129 active+clean; 445 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 20 MiB/s wr, 8 op/s 2024-03-26T08:36:12.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:11 smithi043 ceph-mon[31929]: pgmap v521: 129 pgs: 129 active+clean; 445 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 20 MiB/s wr, 8 op/s 2024-03-26T08:36:14.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:13 smithi183 ceph-mon[35818]: pgmap v522: 129 pgs: 129 active+clean; 477 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 17 MiB/s wr, 6 op/s 2024-03-26T08:36:14.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:13 smithi052 ceph-mon[35071]: pgmap v522: 129 pgs: 129 active+clean; 477 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 17 MiB/s wr, 6 op/s 2024-03-26T08:36:14.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:13 smithi043 ceph-mon[31929]: pgmap v522: 129 pgs: 129 active+clean; 477 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 17 MiB/s wr, 6 op/s 2024-03-26T08:36:16.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:15 smithi183 ceph-mon[35818]: pgmap v523: 129 pgs: 129 active+clean; 517 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-03-26T08:36:16.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:15 smithi052 ceph-mon[35071]: pgmap v523: 129 pgs: 129 active+clean; 517 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-03-26T08:36:16.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:15 smithi043 ceph-mon[31929]: pgmap v523: 129 pgs: 129 active+clean; 517 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-03-26T08:36:18.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:17 smithi183 ceph-mon[35818]: pgmap v524: 129 pgs: 129 active+clean; 581 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 21 MiB/s wr, 7 op/s 2024-03-26T08:36:18.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:17 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:36:18.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:17 smithi052 ceph-mon[35071]: pgmap v524: 129 pgs: 129 active+clean; 581 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 21 MiB/s wr, 7 op/s 2024-03-26T08:36:18.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:17 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:36:18.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:17 smithi043 ceph-mon[31929]: pgmap v524: 129 pgs: 129 active+clean; 581 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 21 MiB/s wr, 7 op/s 2024-03-26T08:36:18.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:17 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:36:19.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:18 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:19.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:18 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:36:19.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:18 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:36:19.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:18 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:19.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:18 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:36:19.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:18 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:36:19.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:18 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:19.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:18 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:36:19.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:18 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:36:20.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:19 smithi183 ceph-mon[35818]: pgmap v525: 129 pgs: 129 active+clean; 597 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 17 MiB/s wr, 6 op/s 2024-03-26T08:36:20.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:19 smithi052 ceph-mon[35071]: pgmap v525: 129 pgs: 129 active+clean; 597 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 17 MiB/s wr, 6 op/s 2024-03-26T08:36:20.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:19 smithi043 ceph-mon[31929]: pgmap v525: 129 pgs: 129 active+clean; 597 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 17 MiB/s wr, 6 op/s 2024-03-26T08:36:22.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:21 smithi183 ceph-mon[35818]: pgmap v526: 129 pgs: 129 active+clean; 673 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 22 MiB/s wr, 8 op/s 2024-03-26T08:36:22.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:21 smithi052 ceph-mon[35071]: pgmap v526: 129 pgs: 129 active+clean; 673 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 22 MiB/s wr, 8 op/s 2024-03-26T08:36:22.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:21 smithi043 ceph-mon[31929]: pgmap v526: 129 pgs: 129 active+clean; 673 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 22 MiB/s wr, 8 op/s 2024-03-26T08:36:24.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:23 smithi183 ceph-mon[35818]: pgmap v527: 129 pgs: 129 active+clean; 709 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 22 MiB/s wr, 7 op/s 2024-03-26T08:36:24.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:23 smithi052 ceph-mon[35071]: pgmap v527: 129 pgs: 129 active+clean; 709 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 22 MiB/s wr, 7 op/s 2024-03-26T08:36:24.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:23 smithi043 ceph-mon[31929]: pgmap v527: 129 pgs: 129 active+clean; 709 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 22 MiB/s wr, 7 op/s 2024-03-26T08:36:26.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:25 smithi183 ceph-mon[35818]: pgmap v528: 129 pgs: 129 active+clean; 737 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 22 MiB/s wr, 7 op/s 2024-03-26T08:36:26.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:25 smithi052 ceph-mon[35071]: pgmap v528: 129 pgs: 129 active+clean; 737 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 22 MiB/s wr, 7 op/s 2024-03-26T08:36:26.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:25 smithi043 ceph-mon[31929]: pgmap v528: 129 pgs: 129 active+clean; 737 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 22 MiB/s wr, 7 op/s 2024-03-26T08:36:28.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:27 smithi183 ceph-mon[35818]: pgmap v529: 129 pgs: 129 active+clean; 785 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 22 MiB/s wr, 8 op/s 2024-03-26T08:36:28.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:27 smithi052 ceph-mon[35071]: pgmap v529: 129 pgs: 129 active+clean; 785 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 22 MiB/s wr, 8 op/s 2024-03-26T08:36:28.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:27 smithi043 ceph-mon[31929]: pgmap v529: 129 pgs: 129 active+clean; 785 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 22 MiB/s wr, 8 op/s 2024-03-26T08:36:30.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:29 smithi183 ceph-mon[35818]: pgmap v530: 129 pgs: 129 active+clean; 797 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-03-26T08:36:30.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:29 smithi052 ceph-mon[35071]: pgmap v530: 129 pgs: 129 active+clean; 797 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-03-26T08:36:30.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:29 smithi043 ceph-mon[31929]: pgmap v530: 129 pgs: 129 active+clean; 797 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-03-26T08:36:32.547 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:32 smithi183 ceph-mon[35818]: pgmap v531: 129 pgs: 129 active+clean; 849 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 21 MiB/s wr, 7 op/s 2024-03-26T08:36:32.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:32 smithi052 ceph-mon[35071]: pgmap v531: 129 pgs: 129 active+clean; 849 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 21 MiB/s wr, 7 op/s 2024-03-26T08:36:32.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:32 smithi043 ceph-mon[31929]: pgmap v531: 129 pgs: 129 active+clean; 849 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 21 MiB/s wr, 7 op/s 2024-03-26T08:36:34.547 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:34 smithi183 ceph-mon[35818]: pgmap v532: 129 pgs: 129 active+clean; 885 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 18 MiB/s wr, 6 op/s 2024-03-26T08:36:34.571 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:34 smithi052 ceph-mon[35071]: pgmap v532: 129 pgs: 129 active+clean; 885 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 18 MiB/s wr, 6 op/s 2024-03-26T08:36:34.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:34 smithi043 ceph-mon[31929]: pgmap v532: 129 pgs: 129 active+clean; 885 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 18 MiB/s wr, 6 op/s 2024-03-26T08:36:36.547 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:36 smithi183 ceph-mon[35818]: pgmap v533: 129 pgs: 129 active+clean; 905 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 16 MiB/s wr, 5 op/s 2024-03-26T08:36:36.571 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:36 smithi052 ceph-mon[35071]: pgmap v533: 129 pgs: 129 active+clean; 905 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 16 MiB/s wr, 5 op/s 2024-03-26T08:36:36.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:36 smithi043 ceph-mon[31929]: pgmap v533: 129 pgs: 129 active+clean; 905 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 16 MiB/s wr, 5 op/s 2024-03-26T08:36:37.081 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:36:37.081 INFO:tasks.workunit.client.0.smithi043.stdout: Children see throughput for 1 initial writers = 19157.84 kB/sec 2024-03-26T08:36:37.081 INFO:tasks.workunit.client.0.smithi043.stdout: Parent sees throughput for 1 initial writers = 19157.18 kB/sec 2024-03-26T08:36:37.081 INFO:tasks.workunit.client.0.smithi043.stdout: Min throughput per process = 19157.84 kB/sec 2024-03-26T08:36:37.081 INFO:tasks.workunit.client.0.smithi043.stdout: Max throughput per process = 19157.84 kB/sec 2024-03-26T08:36:37.082 INFO:tasks.workunit.client.0.smithi043.stdout: Avg throughput per process = 19157.84 kB/sec 2024-03-26T08:36:37.082 INFO:tasks.workunit.client.0.smithi043.stdout: Min xfer = 1048576.00 kB 2024-03-26T08:36:38.547 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:38 smithi183 ceph-mon[35818]: pgmap v534: 129 pgs: 129 active+clean; 985 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 21 MiB/s wr, 7 op/s 2024-03-26T08:36:38.571 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:38 smithi052 ceph-mon[35071]: pgmap v534: 129 pgs: 129 active+clean; 985 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 21 MiB/s wr, 7 op/s 2024-03-26T08:36:38.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:38 smithi043 ceph-mon[31929]: pgmap v534: 129 pgs: 129 active+clean; 985 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 21 MiB/s wr, 7 op/s 2024-03-26T08:36:39.547 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:39 smithi183 ceph-mon[35818]: pgmap v535: 129 pgs: 129 active+clean; 993 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 17 MiB/s wr, 6 op/s 2024-03-26T08:36:39.571 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:39 smithi052 ceph-mon[35071]: pgmap v535: 129 pgs: 129 active+clean; 993 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 17 MiB/s wr, 6 op/s 2024-03-26T08:36:39.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:39 smithi043 ceph-mon[31929]: pgmap v535: 129 pgs: 129 active+clean; 993 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 17 MiB/s wr, 6 op/s 2024-03-26T08:36:41.797 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:41 smithi183 ceph-mon[35818]: pgmap v536: 129 pgs: 129 active+clean; 1017 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 18 MiB/s wr, 7 op/s 2024-03-26T08:36:41.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:41 smithi052 ceph-mon[35071]: pgmap v536: 129 pgs: 129 active+clean; 1017 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 18 MiB/s wr, 7 op/s 2024-03-26T08:36:41.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:41 smithi043 ceph-mon[31929]: pgmap v536: 129 pgs: 129 active+clean; 1017 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 18 MiB/s wr, 7 op/s 2024-03-26T08:36:43.797 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:43 smithi183 ceph-mon[35818]: pgmap v537: 129 pgs: 129 active+clean; 1.0 GiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 15 MiB/s wr, 5 op/s 2024-03-26T08:36:43.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:43 smithi052 ceph-mon[35071]: pgmap v537: 129 pgs: 129 active+clean; 1.0 GiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 15 MiB/s wr, 5 op/s 2024-03-26T08:36:43.900 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:43 smithi043 ceph-mon[31929]: pgmap v537: 129 pgs: 129 active+clean; 1.0 GiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 15 MiB/s wr, 5 op/s 2024-03-26T08:36:45.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:45 smithi052 ceph-mon[35071]: pgmap v538: 129 pgs: 129 active+clean; 1.0 GiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 16 MiB/s wr, 5 op/s 2024-03-26T08:36:45.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:45 smithi043 ceph-mon[31929]: pgmap v538: 129 pgs: 129 active+clean; 1.0 GiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 16 MiB/s wr, 5 op/s 2024-03-26T08:36:46.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:45 smithi183 ceph-mon[35818]: pgmap v538: 129 pgs: 129 active+clean; 1.0 GiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 16 MiB/s wr, 5 op/s 2024-03-26T08:36:47.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:47 smithi052 ceph-mon[35071]: pgmap v539: 129 pgs: 129 active+clean; 1.0 GiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 21 MiB/s wr, 8 op/s 2024-03-26T08:36:47.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:47 smithi043 ceph-mon[31929]: pgmap v539: 129 pgs: 129 active+clean; 1.0 GiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 21 MiB/s wr, 8 op/s 2024-03-26T08:36:48.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:47 smithi183 ceph-mon[35818]: pgmap v539: 129 pgs: 129 active+clean; 1.0 GiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 21 MiB/s wr, 8 op/s 2024-03-26T08:36:49.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:49 smithi052 ceph-mon[35071]: pgmap v540: 129 pgs: 129 active+clean; 1.0 GiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 15 MiB/s wr, 5 op/s 2024-03-26T08:36:49.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:49 smithi043 ceph-mon[31929]: pgmap v540: 129 pgs: 129 active+clean; 1.0 GiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 15 MiB/s wr, 5 op/s 2024-03-26T08:36:50.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:49 smithi183 ceph-mon[35818]: pgmap v540: 129 pgs: 129 active+clean; 1.0 GiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 15 MiB/s wr, 5 op/s 2024-03-26T08:36:51.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:51 smithi043 ceph-mon[31929]: pgmap v541: 129 pgs: 129 active+clean; 1.0 GiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 19 MiB/s wr, 7 op/s 2024-03-26T08:36:52.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:51 smithi183 ceph-mon[35818]: pgmap v541: 129 pgs: 129 active+clean; 1.0 GiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 19 MiB/s wr, 7 op/s 2024-03-26T08:36:52.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:51 smithi052 ceph-mon[35071]: pgmap v541: 129 pgs: 129 active+clean; 1.0 GiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 19 MiB/s wr, 7 op/s 2024-03-26T08:36:53.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:53 smithi043 ceph-mon[31929]: pgmap v542: 129 pgs: 129 active+clean; 1.0 GiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 20 MiB/s wr, 6 op/s 2024-03-26T08:36:54.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:53 smithi183 ceph-mon[35818]: pgmap v542: 129 pgs: 129 active+clean; 1.0 GiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 20 MiB/s wr, 6 op/s 2024-03-26T08:36:54.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:53 smithi052 ceph-mon[35071]: pgmap v542: 129 pgs: 129 active+clean; 1.0 GiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 20 MiB/s wr, 6 op/s 2024-03-26T08:36:55.851 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:55 smithi183 ceph-mon[35818]: pgmap v543: 129 pgs: 129 active+clean; 1.0 GiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 23 MiB/s wr, 7 op/s 2024-03-26T08:36:55.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:55 smithi043 ceph-mon[31929]: pgmap v543: 129 pgs: 129 active+clean; 1.0 GiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 23 MiB/s wr, 7 op/s 2024-03-26T08:36:56.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:55 smithi052 ceph-mon[35071]: pgmap v543: 129 pgs: 129 active+clean; 1.0 GiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 23 MiB/s wr, 7 op/s 2024-03-26T08:36:57.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:57 smithi043 ceph-mon[31929]: pgmap v544: 129 pgs: 129 active+clean; 1.0 GiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 24 MiB/s wr, 8 op/s 2024-03-26T08:36:58.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:57 smithi183 ceph-mon[35818]: pgmap v544: 129 pgs: 129 active+clean; 1.0 GiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 24 MiB/s wr, 8 op/s 2024-03-26T08:36:58.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:57 smithi052 ceph-mon[35071]: pgmap v544: 129 pgs: 129 active+clean; 1.0 GiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 24 MiB/s wr, 8 op/s 2024-03-26T08:36:59.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:36:59 smithi043 ceph-mon[31929]: pgmap v545: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 18 MiB/s wr, 6 op/s 2024-03-26T08:37:00.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:36:59 smithi183 ceph-mon[35818]: pgmap v545: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 18 MiB/s wr, 6 op/s 2024-03-26T08:37:00.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:36:59 smithi052 ceph-mon[35071]: pgmap v545: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 18 MiB/s wr, 6 op/s 2024-03-26T08:37:01.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:01 smithi043 ceph-mon[31929]: pgmap v546: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 21 MiB/s wr, 7 op/s 2024-03-26T08:37:02.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:01 smithi183 ceph-mon[35818]: pgmap v546: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 21 MiB/s wr, 7 op/s 2024-03-26T08:37:02.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:01 smithi052 ceph-mon[35071]: pgmap v546: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 21 MiB/s wr, 7 op/s 2024-03-26T08:37:03.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:03 smithi043 ceph-mon[31929]: pgmap v547: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 21 MiB/s wr, 6 op/s 2024-03-26T08:37:04.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:03 smithi183 ceph-mon[35818]: pgmap v547: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 21 MiB/s wr, 6 op/s 2024-03-26T08:37:04.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:03 smithi052 ceph-mon[35071]: pgmap v547: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 21 MiB/s wr, 6 op/s 2024-03-26T08:37:05.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:05 smithi043 ceph-mon[31929]: pgmap v548: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 22 MiB/s wr, 7 op/s 2024-03-26T08:37:06.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:05 smithi183 ceph-mon[35818]: pgmap v548: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 22 MiB/s wr, 7 op/s 2024-03-26T08:37:06.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:05 smithi052 ceph-mon[35071]: pgmap v548: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 22 MiB/s wr, 7 op/s 2024-03-26T08:37:07.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:07 smithi043 ceph-mon[31929]: pgmap v549: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 24 MiB/s wr, 8 op/s 2024-03-26T08:37:08.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:07 smithi183 ceph-mon[35818]: pgmap v549: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 24 MiB/s wr, 8 op/s 2024-03-26T08:37:08.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:07 smithi052 ceph-mon[35071]: pgmap v549: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 24 MiB/s wr, 8 op/s 2024-03-26T08:37:09.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:09 smithi043 ceph-mon[31929]: pgmap v550: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 20 MiB/s wr, 6 op/s 2024-03-26T08:37:10.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:09 smithi183 ceph-mon[35818]: pgmap v550: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 20 MiB/s wr, 6 op/s 2024-03-26T08:37:10.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:09 smithi052 ceph-mon[35071]: pgmap v550: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 20 MiB/s wr, 6 op/s 2024-03-26T08:37:11.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:11 smithi043 ceph-mon[31929]: pgmap v551: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 24 MiB/s wr, 8 op/s 2024-03-26T08:37:12.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:11 smithi183 ceph-mon[35818]: pgmap v551: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 24 MiB/s wr, 8 op/s 2024-03-26T08:37:12.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:11 smithi052 ceph-mon[35071]: pgmap v551: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 24 MiB/s wr, 8 op/s 2024-03-26T08:37:13.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:13 smithi043 ceph-mon[31929]: pgmap v552: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 23 MiB/s wr, 7 op/s 2024-03-26T08:37:14.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:13 smithi183 ceph-mon[35818]: pgmap v552: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 23 MiB/s wr, 7 op/s 2024-03-26T08:37:14.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:13 smithi052 ceph-mon[35071]: pgmap v552: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 23 MiB/s wr, 7 op/s 2024-03-26T08:37:16.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:15 smithi183 ceph-mon[35818]: pgmap v553: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 22 MiB/s wr, 7 op/s 2024-03-26T08:37:16.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:15 smithi052 ceph-mon[35071]: pgmap v553: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 22 MiB/s wr, 7 op/s 2024-03-26T08:37:16.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:15 smithi043 ceph-mon[31929]: pgmap v553: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 22 MiB/s wr, 7 op/s 2024-03-26T08:37:18.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:17 smithi183 ceph-mon[35818]: pgmap v554: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 23 MiB/s wr, 8 op/s 2024-03-26T08:37:18.073 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:17 smithi052 ceph-mon[35071]: pgmap v554: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 23 MiB/s wr, 8 op/s 2024-03-26T08:37:18.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:17 smithi043 ceph-mon[31929]: pgmap v554: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 23 MiB/s wr, 8 op/s 2024-03-26T08:37:20.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:19 smithi183 ceph-mon[35818]: pgmap v555: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 19 MiB/s wr, 6 op/s 2024-03-26T08:37:20.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:19 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:37:20.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:19 smithi052 ceph-mon[35071]: pgmap v555: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 19 MiB/s wr, 6 op/s 2024-03-26T08:37:20.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:19 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:37:20.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:19 smithi043 ceph-mon[31929]: pgmap v555: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 19 MiB/s wr, 6 op/s 2024-03-26T08:37:20.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:19 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:37:21.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:20 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:37:21.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:20 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:37:21.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:20 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:37:21.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:20 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:37:21.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:20 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:37:21.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:20 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:37:21.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:20 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:37:21.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:20 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:37:21.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:20 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:37:22.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:21 smithi183 ceph-mon[35818]: pgmap v556: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 24 MiB/s wr, 8 op/s 2024-03-26T08:37:22.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:21 smithi052 ceph-mon[35071]: pgmap v556: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 24 MiB/s wr, 8 op/s 2024-03-26T08:37:22.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:21 smithi043 ceph-mon[31929]: pgmap v556: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 24 MiB/s wr, 8 op/s 2024-03-26T08:37:24.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:23 smithi183 ceph-mon[35818]: pgmap v557: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 22 MiB/s wr, 7 op/s 2024-03-26T08:37:24.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:23 smithi052 ceph-mon[35071]: pgmap v557: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 22 MiB/s wr, 7 op/s 2024-03-26T08:37:24.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:23 smithi043 ceph-mon[31929]: pgmap v557: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 22 MiB/s wr, 7 op/s 2024-03-26T08:37:25.336 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:37:25.336 INFO:tasks.workunit.client.0.smithi043.stdout: Children see throughput for 1 rewriters = 22890.47 kB/sec 2024-03-26T08:37:25.337 INFO:tasks.workunit.client.0.smithi043.stdout: Parent sees throughput for 1 rewriters = 22889.67 kB/sec 2024-03-26T08:37:25.337 INFO:tasks.workunit.client.0.smithi043.stdout: Min throughput per process = 22890.47 kB/sec 2024-03-26T08:37:25.337 INFO:tasks.workunit.client.0.smithi043.stdout: Max throughput per process = 22890.47 kB/sec 2024-03-26T08:37:25.337 INFO:tasks.workunit.client.0.smithi043.stdout: Avg throughput per process = 22890.47 kB/sec 2024-03-26T08:37:25.337 INFO:tasks.workunit.client.0.smithi043.stdout: Min xfer = 1048576.00 kB 2024-03-26T08:37:26.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:25 smithi183 ceph-mon[35818]: pgmap v558: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 22 MiB/s wr, 7 op/s 2024-03-26T08:37:26.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:25 smithi052 ceph-mon[35071]: pgmap v558: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 22 MiB/s wr, 7 op/s 2024-03-26T08:37:26.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:25 smithi043 ceph-mon[31929]: pgmap v558: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 22 MiB/s wr, 7 op/s 2024-03-26T08:37:28.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:27 smithi183 ceph-mon[35818]: pgmap v559: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 26 MiB/s wr, 8 op/s 2024-03-26T08:37:28.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:27 smithi052 ceph-mon[35071]: pgmap v559: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 26 MiB/s wr, 8 op/s 2024-03-26T08:37:28.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:27 smithi043 ceph-mon[31929]: pgmap v559: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 26 MiB/s wr, 8 op/s 2024-03-26T08:37:28.515 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:37:28.515 INFO:tasks.workunit.client.0.smithi043.stdout: Children see throughput for 1 readers = 4679077.50 kB/sec 2024-03-26T08:37:28.515 INFO:tasks.workunit.client.0.smithi043.stdout: Parent sees throughput for 1 readers = 4662489.85 kB/sec 2024-03-26T08:37:28.515 INFO:tasks.workunit.client.0.smithi043.stdout: Min throughput per process = 4679077.50 kB/sec 2024-03-26T08:37:28.516 INFO:tasks.workunit.client.0.smithi043.stdout: Max throughput per process = 4679077.50 kB/sec 2024-03-26T08:37:28.516 INFO:tasks.workunit.client.0.smithi043.stdout: Avg throughput per process = 4679077.50 kB/sec 2024-03-26T08:37:28.516 INFO:tasks.workunit.client.0.smithi043.stdout: Min xfer = 1048576.00 kB 2024-03-26T08:37:30.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:29 smithi183 ceph-mon[35818]: pgmap v560: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 20 MiB/s wr, 6 op/s 2024-03-26T08:37:30.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:29 smithi052 ceph-mon[35071]: pgmap v560: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 20 MiB/s wr, 6 op/s 2024-03-26T08:37:30.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:29 smithi043 ceph-mon[31929]: pgmap v560: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 20 MiB/s wr, 6 op/s 2024-03-26T08:37:30.944 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:37:30.945 INFO:tasks.workunit.client.0.smithi043.stdout: Children see throughput for 1 re-readers = 3322568.50 kB/sec 2024-03-26T08:37:30.945 INFO:tasks.workunit.client.0.smithi043.stdout: Parent sees throughput for 1 re-readers = 3314532.96 kB/sec 2024-03-26T08:37:30.945 INFO:tasks.workunit.client.0.smithi043.stdout: Min throughput per process = 3322568.50 kB/sec 2024-03-26T08:37:30.945 INFO:tasks.workunit.client.0.smithi043.stdout: Max throughput per process = 3322568.50 kB/sec 2024-03-26T08:37:30.945 INFO:tasks.workunit.client.0.smithi043.stdout: Avg throughput per process = 3322568.50 kB/sec 2024-03-26T08:37:30.945 INFO:tasks.workunit.client.0.smithi043.stdout: Min xfer = 1048576.00 kB 2024-03-26T08:37:32.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:31 smithi183 ceph-mon[35818]: pgmap v561: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 20 MiB/s wr, 7 op/s 2024-03-26T08:37:32.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:31 smithi052 ceph-mon[35071]: pgmap v561: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 20 MiB/s wr, 7 op/s 2024-03-26T08:37:32.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:31 smithi043 ceph-mon[31929]: pgmap v561: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 20 MiB/s wr, 7 op/s 2024-03-26T08:37:34.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:33 smithi183 ceph-mon[35818]: pgmap v562: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 14 MiB/s wr, 5 op/s 2024-03-26T08:37:34.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:33 smithi052 ceph-mon[35071]: pgmap v562: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 14 MiB/s wr, 5 op/s 2024-03-26T08:37:34.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:33 smithi043 ceph-mon[31929]: pgmap v562: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 14 MiB/s wr, 5 op/s 2024-03-26T08:37:36.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:35 smithi183 ceph-mon[35818]: pgmap v563: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 10 MiB/s wr, 4 op/s 2024-03-26T08:37:36.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:35 smithi052 ceph-mon[35071]: pgmap v563: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 10 MiB/s wr, 4 op/s 2024-03-26T08:37:36.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:35 smithi043 ceph-mon[31929]: pgmap v563: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 10 MiB/s wr, 4 op/s 2024-03-26T08:37:37.421 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:37:37.421 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:37:37.422 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:37:37.422 INFO:tasks.workunit.client.0.smithi043.stdout:iozone test complete. 2024-03-26T08:37:37.426 INFO:tasks.workunit.client.0.smithi043.stderr:+ iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-03-26T08:37:37.518 INFO:tasks.workunit.client.0.smithi043.stdout: Iozone: Performance Test of File I/O 2024-03-26T08:37:37.518 INFO:tasks.workunit.client.0.smithi043.stdout: Version $Revision: 3.494 $ 2024-03-26T08:37:37.518 INFO:tasks.workunit.client.0.smithi043.stdout: Compiled for 64 bit mode. 2024-03-26T08:37:37.518 INFO:tasks.workunit.client.0.smithi043.stdout: Build: linux-AMD64 2024-03-26T08:37:37.518 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:37:37.518 INFO:tasks.workunit.client.0.smithi043.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-03-26T08:37:37.518 INFO:tasks.workunit.client.0.smithi043.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-03-26T08:37:37.519 INFO:tasks.workunit.client.0.smithi043.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-03-26T08:37:37.519 INFO:tasks.workunit.client.0.smithi043.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-03-26T08:37:37.519 INFO:tasks.workunit.client.0.smithi043.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-03-26T08:37:37.519 INFO:tasks.workunit.client.0.smithi043.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-03-26T08:37:37.519 INFO:tasks.workunit.client.0.smithi043.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-03-26T08:37:37.519 INFO:tasks.workunit.client.0.smithi043.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-03-26T08:37:37.519 INFO:tasks.workunit.client.0.smithi043.stdout: Alexey Skidanov, Sudhir Kumar. 2024-03-26T08:37:37.519 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:37:37.519 INFO:tasks.workunit.client.0.smithi043.stdout: Run began: Tue Mar 26 08:37:37 2024 2024-03-26T08:37:37.519 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:37:37.519 INFO:tasks.workunit.client.0.smithi043.stdout: Include close in write timing 2024-03-26T08:37:37.519 INFO:tasks.workunit.client.0.smithi043.stdout: Include fsync in write timing 2024-03-26T08:37:37.519 INFO:tasks.workunit.client.0.smithi043.stdout: File size set to 1048576 kB 2024-03-26T08:37:37.519 INFO:tasks.workunit.client.0.smithi043.stdout: Record Size 1024 kB 2024-03-26T08:37:37.519 INFO:tasks.workunit.client.0.smithi043.stdout: Command line used: iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-03-26T08:37:37.520 INFO:tasks.workunit.client.0.smithi043.stdout: Output is in kBytes/sec 2024-03-26T08:37:37.520 INFO:tasks.workunit.client.0.smithi043.stdout: Time Resolution = 0.000001 seconds. 2024-03-26T08:37:37.520 INFO:tasks.workunit.client.0.smithi043.stdout: Processor cache size set to 1024 kBytes. 2024-03-26T08:37:37.520 INFO:tasks.workunit.client.0.smithi043.stdout: Processor cache line size set to 32 bytes. 2024-03-26T08:37:37.520 INFO:tasks.workunit.client.0.smithi043.stdout: File stride size set to 17 * record size. 2024-03-26T08:37:37.520 INFO:tasks.workunit.client.0.smithi043.stdout: Throughput test with 1 process 2024-03-26T08:37:37.520 INFO:tasks.workunit.client.0.smithi043.stdout: Each process writes a 1048576 kByte file in 1024 kByte records 2024-03-26T08:37:37.797 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:37 smithi183 ceph-mon[35818]: pgmap v564: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 8.3 MiB/s wr, 4 op/s 2024-03-26T08:37:38.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:37 smithi052 ceph-mon[35071]: pgmap v564: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 8.3 MiB/s wr, 4 op/s 2024-03-26T08:37:38.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:37 smithi043 ceph-mon[31929]: pgmap v564: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 8.3 MiB/s wr, 4 op/s 2024-03-26T08:37:40.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:39 smithi183 ceph-mon[35818]: pgmap v565: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2.0 MiB/s wr, 2 op/s 2024-03-26T08:37:40.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:39 smithi052 ceph-mon[35071]: pgmap v565: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2.0 MiB/s wr, 2 op/s 2024-03-26T08:37:40.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:39 smithi043 ceph-mon[31929]: pgmap v565: 129 pgs: 129 active+clean; 1.0 GiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2.0 MiB/s wr, 2 op/s 2024-03-26T08:37:42.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:41 smithi183 ceph-mon[35818]: pgmap v566: 129 pgs: 129 active+clean; 845 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 19 MiB/s wr, 18 op/s 2024-03-26T08:37:42.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:41 smithi052 ceph-mon[35071]: pgmap v566: 129 pgs: 129 active+clean; 845 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 19 MiB/s wr, 18 op/s 2024-03-26T08:37:42.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:41 smithi043 ceph-mon[31929]: pgmap v566: 129 pgs: 129 active+clean; 845 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 19 MiB/s wr, 18 op/s 2024-03-26T08:37:44.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:43 smithi183 ceph-mon[35818]: pgmap v567: 129 pgs: 129 active+clean; 621 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 28 MiB/s wr, 26 op/s 2024-03-26T08:37:44.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:43 smithi052 ceph-mon[35071]: pgmap v567: 129 pgs: 129 active+clean; 621 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 28 MiB/s wr, 26 op/s 2024-03-26T08:37:44.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:43 smithi043 ceph-mon[31929]: pgmap v567: 129 pgs: 129 active+clean; 621 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 28 MiB/s wr, 26 op/s 2024-03-26T08:37:44.250 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:37:44.251 INFO:tasks.workunit.client.0.smithi043.stdout: Children see throughput for 1 initial writers = 156680.62 kB/sec 2024-03-26T08:37:44.251 INFO:tasks.workunit.client.0.smithi043.stdout: Parent sees throughput for 1 initial writers = 156632.09 kB/sec 2024-03-26T08:37:44.251 INFO:tasks.workunit.client.0.smithi043.stdout: Min throughput per process = 156680.62 kB/sec 2024-03-26T08:37:44.251 INFO:tasks.workunit.client.0.smithi043.stdout: Max throughput per process = 156680.62 kB/sec 2024-03-26T08:37:44.251 INFO:tasks.workunit.client.0.smithi043.stdout: Avg throughput per process = 156680.62 kB/sec 2024-03-26T08:37:44.251 INFO:tasks.workunit.client.0.smithi043.stdout: Min xfer = 1048576.00 kB 2024-03-26T08:37:46.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:45 smithi183 ceph-mon[35818]: pgmap v568: 129 pgs: 129 active+clean; 685 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 57 MiB/s wr, 39 op/s 2024-03-26T08:37:46.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:45 smithi052 ceph-mon[35071]: pgmap v568: 129 pgs: 129 active+clean; 685 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 57 MiB/s wr, 39 op/s 2024-03-26T08:37:46.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:45 smithi043 ceph-mon[31929]: pgmap v568: 129 pgs: 129 active+clean; 685 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 57 MiB/s wr, 39 op/s 2024-03-26T08:37:48.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:47 smithi183 ceph-mon[35818]: pgmap v569: 129 pgs: 129 active+clean; 1001 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 83 MiB/s wr, 47 op/s 2024-03-26T08:37:48.075 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:47 smithi052 ceph-mon[35071]: pgmap v569: 129 pgs: 129 active+clean; 1001 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 83 MiB/s wr, 47 op/s 2024-03-26T08:37:48.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:47 smithi043 ceph-mon[31929]: pgmap v569: 129 pgs: 129 active+clean; 1001 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 83 MiB/s wr, 47 op/s 2024-03-26T08:37:49.557 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:37:49.557 INFO:tasks.workunit.client.0.smithi043.stdout: Children see throughput for 1 rewriters = 419762.34 kB/sec 2024-03-26T08:37:49.557 INFO:tasks.workunit.client.0.smithi043.stdout: Parent sees throughput for 1 rewriters = 419614.33 kB/sec 2024-03-26T08:37:49.557 INFO:tasks.workunit.client.0.smithi043.stdout: Min throughput per process = 419762.34 kB/sec 2024-03-26T08:37:49.557 INFO:tasks.workunit.client.0.smithi043.stdout: Max throughput per process = 419762.34 kB/sec 2024-03-26T08:37:49.557 INFO:tasks.workunit.client.0.smithi043.stdout: Avg throughput per process = 419762.34 kB/sec 2024-03-26T08:37:49.557 INFO:tasks.workunit.client.0.smithi043.stdout: Min xfer = 1048576.00 kB 2024-03-26T08:37:50.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:49 smithi183 ceph-mon[35818]: pgmap v570: 129 pgs: 129 active+clean; 1017 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 89 MiB/s wr, 48 op/s 2024-03-26T08:37:50.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:49 smithi052 ceph-mon[35071]: pgmap v570: 129 pgs: 129 active+clean; 1017 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 89 MiB/s wr, 48 op/s 2024-03-26T08:37:50.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:49 smithi043 ceph-mon[31929]: pgmap v570: 129 pgs: 129 active+clean; 1017 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 89 MiB/s wr, 48 op/s 2024-03-26T08:37:52.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:51 smithi183 ceph-mon[35818]: pgmap v571: 129 pgs: 129 active+clean; 1.0 GiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 132 MiB/s wr, 60 op/s 2024-03-26T08:37:52.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:51 smithi052 ceph-mon[35071]: pgmap v571: 129 pgs: 129 active+clean; 1.0 GiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 132 MiB/s wr, 60 op/s 2024-03-26T08:37:52.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:51 smithi043 ceph-mon[31929]: pgmap v571: 129 pgs: 129 active+clean; 1.0 GiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 132 MiB/s wr, 60 op/s 2024-03-26T08:37:52.892 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:37:52.892 INFO:tasks.workunit.client.0.smithi043.stdout: Children see throughput for 1 readers = 5038449.50 kB/sec 2024-03-26T08:37:52.892 INFO:tasks.workunit.client.0.smithi043.stdout: Parent sees throughput for 1 readers = 5016290.20 kB/sec 2024-03-26T08:37:52.892 INFO:tasks.workunit.client.0.smithi043.stdout: Min throughput per process = 5038449.50 kB/sec 2024-03-26T08:37:52.892 INFO:tasks.workunit.client.0.smithi043.stdout: Max throughput per process = 5038449.50 kB/sec 2024-03-26T08:37:52.892 INFO:tasks.workunit.client.0.smithi043.stdout: Avg throughput per process = 5038449.50 kB/sec 2024-03-26T08:37:52.892 INFO:tasks.workunit.client.0.smithi043.stdout: Min xfer = 1048576.00 kB 2024-03-26T08:37:54.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:53 smithi183 ceph-mon[35818]: pgmap v572: 129 pgs: 129 active+clean; 1.0 GiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 137 MiB/s wr, 49 op/s 2024-03-26T08:37:54.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:53 smithi052 ceph-mon[35071]: pgmap v572: 129 pgs: 129 active+clean; 1.0 GiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 137 MiB/s wr, 49 op/s 2024-03-26T08:37:54.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:53 smithi043 ceph-mon[31929]: pgmap v572: 129 pgs: 129 active+clean; 1.0 GiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 137 MiB/s wr, 49 op/s 2024-03-26T08:37:55.290 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:37:55.290 INFO:tasks.workunit.client.0.smithi043.stdout: Children see throughput for 1 re-readers = 3523500.50 kB/sec 2024-03-26T08:37:55.290 INFO:tasks.workunit.client.0.smithi043.stdout: Parent sees throughput for 1 re-readers = 3512783.58 kB/sec 2024-03-26T08:37:55.290 INFO:tasks.workunit.client.0.smithi043.stdout: Min throughput per process = 3523500.50 kB/sec 2024-03-26T08:37:55.290 INFO:tasks.workunit.client.0.smithi043.stdout: Max throughput per process = 3523500.50 kB/sec 2024-03-26T08:37:55.291 INFO:tasks.workunit.client.0.smithi043.stdout: Avg throughput per process = 3523500.50 kB/sec 2024-03-26T08:37:55.291 INFO:tasks.workunit.client.0.smithi043.stdout: Min xfer = 1048576.00 kB 2024-03-26T08:37:56.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:55 smithi183 ceph-mon[35818]: pgmap v573: 129 pgs: 129 active+clean; 1.0 GiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 143 MiB/s wr, 44 op/s 2024-03-26T08:37:56.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:55 smithi052 ceph-mon[35071]: pgmap v573: 129 pgs: 129 active+clean; 1.0 GiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 143 MiB/s wr, 44 op/s 2024-03-26T08:37:56.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:55 smithi043 ceph-mon[31929]: pgmap v573: 129 pgs: 129 active+clean; 1.0 GiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 143 MiB/s wr, 44 op/s 2024-03-26T08:37:58.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:57 smithi183 ceph-mon[35818]: pgmap v574: 129 pgs: 129 active+clean; 1.0 GiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 114 MiB/s wr, 32 op/s 2024-03-26T08:37:58.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:57 smithi052 ceph-mon[35071]: pgmap v574: 129 pgs: 129 active+clean; 1.0 GiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 114 MiB/s wr, 32 op/s 2024-03-26T08:37:58.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:57 smithi043 ceph-mon[31929]: pgmap v574: 129 pgs: 129 active+clean; 1.0 GiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 114 MiB/s wr, 32 op/s 2024-03-26T08:38:00.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:37:59 smithi183 ceph-mon[35818]: pgmap v575: 129 pgs: 129 active+clean; 1.0 GiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 87 MiB/s wr, 23 op/s 2024-03-26T08:38:00.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:37:59 smithi052 ceph-mon[35071]: pgmap v575: 129 pgs: 129 active+clean; 1.0 GiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 87 MiB/s wr, 23 op/s 2024-03-26T08:38:00.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:37:59 smithi043 ceph-mon[31929]: pgmap v575: 129 pgs: 129 active+clean; 1.0 GiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 87 MiB/s wr, 23 op/s 2024-03-26T08:38:01.780 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:38:01.780 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:38:01.780 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:38:01.781 INFO:tasks.workunit.client.0.smithi043.stdout:iozone test complete. 2024-03-26T08:38:01.788 INFO:tasks.workunit.client.0.smithi043.stderr:+ iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-03-26T08:38:01.880 INFO:tasks.workunit.client.0.smithi043.stdout: Iozone: Performance Test of File I/O 2024-03-26T08:38:01.880 INFO:tasks.workunit.client.0.smithi043.stdout: Version $Revision: 3.494 $ 2024-03-26T08:38:01.880 INFO:tasks.workunit.client.0.smithi043.stdout: Compiled for 64 bit mode. 2024-03-26T08:38:01.880 INFO:tasks.workunit.client.0.smithi043.stdout: Build: linux-AMD64 2024-03-26T08:38:01.880 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:38:01.880 INFO:tasks.workunit.client.0.smithi043.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-03-26T08:38:01.880 INFO:tasks.workunit.client.0.smithi043.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-03-26T08:38:01.880 INFO:tasks.workunit.client.0.smithi043.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-03-26T08:38:01.880 INFO:tasks.workunit.client.0.smithi043.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-03-26T08:38:01.881 INFO:tasks.workunit.client.0.smithi043.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-03-26T08:38:01.881 INFO:tasks.workunit.client.0.smithi043.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-03-26T08:38:01.881 INFO:tasks.workunit.client.0.smithi043.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-03-26T08:38:01.881 INFO:tasks.workunit.client.0.smithi043.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-03-26T08:38:01.881 INFO:tasks.workunit.client.0.smithi043.stdout: Alexey Skidanov, Sudhir Kumar. 2024-03-26T08:38:01.881 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:38:01.881 INFO:tasks.workunit.client.0.smithi043.stdout: Run began: Tue Mar 26 08:38:01 2024 2024-03-26T08:38:01.881 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:38:01.881 INFO:tasks.workunit.client.0.smithi043.stdout: Include close in write timing 2024-03-26T08:38:01.881 INFO:tasks.workunit.client.0.smithi043.stdout: Include fsync in write timing 2024-03-26T08:38:01.882 INFO:tasks.workunit.client.0.smithi043.stdout: File size set to 10485760 kB 2024-03-26T08:38:01.882 INFO:tasks.workunit.client.0.smithi043.stdout: Record Size 1024 kB 2024-03-26T08:38:01.882 INFO:tasks.workunit.client.0.smithi043.stdout: Command line used: iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-03-26T08:38:01.882 INFO:tasks.workunit.client.0.smithi043.stdout: Output is in kBytes/sec 2024-03-26T08:38:01.882 INFO:tasks.workunit.client.0.smithi043.stdout: Time Resolution = 0.000001 seconds. 2024-03-26T08:38:01.882 INFO:tasks.workunit.client.0.smithi043.stdout: Processor cache size set to 1024 kBytes. 2024-03-26T08:38:01.882 INFO:tasks.workunit.client.0.smithi043.stdout: Processor cache line size set to 32 bytes. 2024-03-26T08:38:01.882 INFO:tasks.workunit.client.0.smithi043.stdout: File stride size set to 17 * record size. 2024-03-26T08:38:01.882 INFO:tasks.workunit.client.0.smithi043.stdout: Throughput test with 1 process 2024-03-26T08:38:01.883 INFO:tasks.workunit.client.0.smithi043.stdout: Each process writes a 10485760 kByte file in 1024 kByte records 2024-03-26T08:38:02.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:01 smithi052 ceph-mon[35071]: pgmap v576: 129 pgs: 129 active+clean; 1.0 GiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 82 MiB/s wr, 23 op/s 2024-03-26T08:38:02.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:01 smithi043 ceph-mon[31929]: pgmap v576: 129 pgs: 129 active+clean; 1.0 GiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 82 MiB/s wr, 23 op/s 2024-03-26T08:38:02.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:01 smithi183 ceph-mon[35818]: pgmap v576: 129 pgs: 129 active+clean; 1.0 GiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 82 MiB/s wr, 23 op/s 2024-03-26T08:38:04.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:03 smithi052 ceph-mon[35071]: pgmap v577: 129 pgs: 129 active+clean; 1.0 GiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 37 MiB/s wr, 11 op/s 2024-03-26T08:38:04.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:03 smithi043 ceph-mon[31929]: pgmap v577: 129 pgs: 129 active+clean; 1.0 GiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 37 MiB/s wr, 11 op/s 2024-03-26T08:38:04.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:03 smithi183 ceph-mon[35818]: pgmap v577: 129 pgs: 129 active+clean; 1.0 GiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 37 MiB/s wr, 11 op/s 2024-03-26T08:38:06.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:05 smithi052 ceph-mon[35071]: pgmap v578: 129 pgs: 129 active+clean; 1.0 GiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 23 MiB/s wr, 9 op/s 2024-03-26T08:38:06.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:05 smithi043 ceph-mon[31929]: pgmap v578: 129 pgs: 129 active+clean; 1.0 GiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 23 MiB/s wr, 9 op/s 2024-03-26T08:38:06.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:05 smithi183 ceph-mon[35818]: pgmap v578: 129 pgs: 129 active+clean; 1.0 GiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 23 MiB/s wr, 9 op/s 2024-03-26T08:38:08.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:07 smithi052 ceph-mon[35071]: pgmap v579: 129 pgs: 129 active+clean; 809 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 38 MiB/s wr, 28 op/s 2024-03-26T08:38:08.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:07 smithi043 ceph-mon[31929]: pgmap v579: 129 pgs: 129 active+clean; 809 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 38 MiB/s wr, 28 op/s 2024-03-26T08:38:08.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:07 smithi183 ceph-mon[35818]: pgmap v579: 129 pgs: 129 active+clean; 809 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 38 MiB/s wr, 28 op/s 2024-03-26T08:38:10.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:09 smithi043 ceph-mon[31929]: pgmap v580: 129 pgs: 129 active+clean; 745 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 46 MiB/s wr, 32 op/s 2024-03-26T08:38:10.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:09 smithi183 ceph-mon[35818]: pgmap v580: 129 pgs: 129 active+clean; 745 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 46 MiB/s wr, 32 op/s 2024-03-26T08:38:10.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:09 smithi052 ceph-mon[35071]: pgmap v580: 129 pgs: 129 active+clean; 745 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 46 MiB/s wr, 32 op/s 2024-03-26T08:38:12.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:11 smithi043 ceph-mon[31929]: pgmap v581: 129 pgs: 129 active+clean; 945 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 79 MiB/s wr, 46 op/s 2024-03-26T08:38:12.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:11 smithi183 ceph-mon[35818]: pgmap v581: 129 pgs: 129 active+clean; 945 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 79 MiB/s wr, 46 op/s 2024-03-26T08:38:12.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:11 smithi052 ceph-mon[35071]: pgmap v581: 129 pgs: 129 active+clean; 945 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 79 MiB/s wr, 46 op/s 2024-03-26T08:38:14.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:13 smithi043 ceph-mon[31929]: pgmap v582: 129 pgs: 129 active+clean; 1.1 GiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 96 MiB/s wr, 49 op/s 2024-03-26T08:38:14.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:13 smithi183 ceph-mon[35818]: pgmap v582: 129 pgs: 129 active+clean; 1.1 GiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 96 MiB/s wr, 49 op/s 2024-03-26T08:38:14.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:13 smithi052 ceph-mon[35071]: pgmap v582: 129 pgs: 129 active+clean; 1.1 GiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 96 MiB/s wr, 49 op/s 2024-03-26T08:38:16.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:15 smithi043 ceph-mon[31929]: pgmap v583: 129 pgs: 129 active+clean; 1.4 GiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 117 MiB/s wr, 54 op/s 2024-03-26T08:38:16.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:15 smithi183 ceph-mon[35818]: pgmap v583: 129 pgs: 129 active+clean; 1.4 GiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 117 MiB/s wr, 54 op/s 2024-03-26T08:38:16.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:15 smithi052 ceph-mon[35071]: pgmap v583: 129 pgs: 129 active+clean; 1.4 GiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 117 MiB/s wr, 54 op/s 2024-03-26T08:38:18.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:17 smithi043 ceph-mon[31929]: pgmap v584: 129 pgs: 129 active+clean; 1.8 GiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 145 MiB/s wr, 61 op/s 2024-03-26T08:38:18.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:17 smithi183 ceph-mon[35818]: pgmap v584: 129 pgs: 129 active+clean; 1.8 GiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 145 MiB/s wr, 61 op/s 2024-03-26T08:38:18.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:17 smithi052 ceph-mon[35071]: pgmap v584: 129 pgs: 129 active+clean; 1.8 GiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 145 MiB/s wr, 61 op/s 2024-03-26T08:38:20.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:19 smithi043 ceph-mon[31929]: pgmap v585: 129 pgs: 129 active+clean; 1.9 GiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 123 MiB/s wr, 40 op/s 2024-03-26T08:38:20.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:19 smithi183 ceph-mon[35818]: pgmap v585: 129 pgs: 129 active+clean; 1.9 GiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 123 MiB/s wr, 40 op/s 2024-03-26T08:38:20.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:19 smithi052 ceph-mon[35071]: pgmap v585: 129 pgs: 129 active+clean; 1.9 GiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 123 MiB/s wr, 40 op/s 2024-03-26T08:38:22.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:21 smithi043 ceph-mon[31929]: pgmap v586: 129 pgs: 129 active+clean; 2.2 GiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 145 MiB/s wr, 44 op/s 2024-03-26T08:38:22.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:21 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:38:22.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:21 smithi183 ceph-mon[35818]: pgmap v586: 129 pgs: 129 active+clean; 2.2 GiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 145 MiB/s wr, 44 op/s 2024-03-26T08:38:22.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:21 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:38:22.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:21 smithi052 ceph-mon[35071]: pgmap v586: 129 pgs: 129 active+clean; 2.2 GiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 145 MiB/s wr, 44 op/s 2024-03-26T08:38:22.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:21 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:38:23.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:22 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:23.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:22 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:38:23.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:22 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:38:23.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:22 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:23.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:22 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:38:23.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:22 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:38:23.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:22 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:23.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:22 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:38:23.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:22 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:38:24.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:23 smithi043 ceph-mon[31929]: pgmap v587: 129 pgs: 129 active+clean; 2.5 GiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 134 MiB/s wr, 35 op/s 2024-03-26T08:38:24.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:23 smithi183 ceph-mon[35818]: pgmap v587: 129 pgs: 129 active+clean; 2.5 GiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 134 MiB/s wr, 35 op/s 2024-03-26T08:38:24.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:23 smithi052 ceph-mon[35071]: pgmap v587: 129 pgs: 129 active+clean; 2.5 GiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 134 MiB/s wr, 35 op/s 2024-03-26T08:38:26.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:25 smithi043 ceph-mon[31929]: pgmap v588: 129 pgs: 129 active+clean; 2.8 GiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 147 MiB/s wr, 39 op/s 2024-03-26T08:38:26.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:25 smithi183 ceph-mon[35818]: pgmap v588: 129 pgs: 129 active+clean; 2.8 GiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 147 MiB/s wr, 39 op/s 2024-03-26T08:38:26.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:25 smithi052 ceph-mon[35071]: pgmap v588: 129 pgs: 129 active+clean; 2.8 GiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 147 MiB/s wr, 39 op/s 2024-03-26T08:38:28.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:27 smithi043 ceph-mon[31929]: pgmap v589: 129 pgs: 129 active+clean; 3.5 GiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 181 MiB/s wr, 48 op/s 2024-03-26T08:38:28.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:27 smithi183 ceph-mon[35818]: pgmap v589: 129 pgs: 129 active+clean; 3.5 GiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 181 MiB/s wr, 48 op/s 2024-03-26T08:38:28.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:27 smithi052 ceph-mon[35071]: pgmap v589: 129 pgs: 129 active+clean; 3.5 GiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 181 MiB/s wr, 48 op/s 2024-03-26T08:38:30.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:29 smithi183 ceph-mon[35818]: pgmap v590: 129 pgs: 129 active+clean; 3.6 GiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 155 MiB/s wr, 41 op/s 2024-03-26T08:38:30.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:29 smithi052 ceph-mon[35071]: pgmap v590: 129 pgs: 129 active+clean; 3.6 GiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 155 MiB/s wr, 41 op/s 2024-03-26T08:38:30.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:29 smithi043 ceph-mon[31929]: pgmap v590: 129 pgs: 129 active+clean; 3.6 GiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 155 MiB/s wr, 41 op/s 2024-03-26T08:38:32.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:31 smithi183 ceph-mon[35818]: pgmap v591: 129 pgs: 129 active+clean; 4.0 GiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 177 MiB/s wr, 47 op/s 2024-03-26T08:38:32.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:31 smithi052 ceph-mon[35071]: pgmap v591: 129 pgs: 129 active+clean; 4.0 GiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 177 MiB/s wr, 47 op/s 2024-03-26T08:38:32.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:31 smithi043 ceph-mon[31929]: pgmap v591: 129 pgs: 129 active+clean; 4.0 GiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 177 MiB/s wr, 47 op/s 2024-03-26T08:38:34.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:33 smithi183 ceph-mon[35818]: pgmap v592: 129 pgs: 129 active+clean; 4.2 GiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 163 MiB/s wr, 42 op/s 2024-03-26T08:38:34.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:33 smithi052 ceph-mon[35071]: pgmap v592: 129 pgs: 129 active+clean; 4.2 GiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 163 MiB/s wr, 42 op/s 2024-03-26T08:38:34.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:33 smithi043 ceph-mon[31929]: pgmap v592: 129 pgs: 129 active+clean; 4.2 GiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 163 MiB/s wr, 42 op/s 2024-03-26T08:38:36.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:35 smithi183 ceph-mon[35818]: pgmap v593: 129 pgs: 129 active+clean; 4.4 GiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 166 MiB/s wr, 43 op/s 2024-03-26T08:38:36.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:35 smithi052 ceph-mon[35071]: pgmap v593: 129 pgs: 129 active+clean; 4.4 GiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 166 MiB/s wr, 43 op/s 2024-03-26T08:38:36.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:35 smithi043 ceph-mon[31929]: pgmap v593: 129 pgs: 129 active+clean; 4.4 GiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 166 MiB/s wr, 43 op/s 2024-03-26T08:38:38.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:37 smithi183 ceph-mon[35818]: pgmap v594: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 171 MiB/s wr, 45 op/s 2024-03-26T08:38:38.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:37 smithi052 ceph-mon[35071]: pgmap v594: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 171 MiB/s wr, 45 op/s 2024-03-26T08:38:38.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:37 smithi043 ceph-mon[31929]: pgmap v594: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 171 MiB/s wr, 45 op/s 2024-03-26T08:38:40.204 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:39 smithi043 ceph-mon[31929]: pgmap v595: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 121 MiB/s wr, 32 op/s 2024-03-26T08:38:40.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:39 smithi183 ceph-mon[35818]: pgmap v595: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 121 MiB/s wr, 32 op/s 2024-03-26T08:38:40.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:39 smithi052 ceph-mon[35071]: pgmap v595: 129 pgs: 129 active+clean; 4.9 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 121 MiB/s wr, 32 op/s 2024-03-26T08:38:42.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:41 smithi183 ceph-mon[35818]: pgmap v596: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 145 MiB/s wr, 38 op/s 2024-03-26T08:38:42.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:41 smithi052 ceph-mon[35071]: pgmap v596: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 145 MiB/s wr, 38 op/s 2024-03-26T08:38:42.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:41 smithi043 ceph-mon[31929]: pgmap v596: 129 pgs: 129 active+clean; 5.3 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 145 MiB/s wr, 38 op/s 2024-03-26T08:38:44.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:43 smithi183 ceph-mon[35818]: pgmap v597: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 134 MiB/s wr, 35 op/s 2024-03-26T08:38:44.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:43 smithi052 ceph-mon[35071]: pgmap v597: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 134 MiB/s wr, 35 op/s 2024-03-26T08:38:44.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:43 smithi043 ceph-mon[31929]: pgmap v597: 129 pgs: 129 active+clean; 5.5 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 134 MiB/s wr, 35 op/s 2024-03-26T08:38:46.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:45 smithi183 ceph-mon[35818]: pgmap v598: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 136 MiB/s wr, 35 op/s 2024-03-26T08:38:46.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:45 smithi052 ceph-mon[35071]: pgmap v598: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 136 MiB/s wr, 35 op/s 2024-03-26T08:38:46.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:45 smithi043 ceph-mon[31929]: pgmap v598: 129 pgs: 129 active+clean; 5.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 136 MiB/s wr, 35 op/s 2024-03-26T08:38:48.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:47 smithi183 ceph-mon[35818]: pgmap v599: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 150 MiB/s wr, 40 op/s 2024-03-26T08:38:48.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:47 smithi052 ceph-mon[35071]: pgmap v599: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 150 MiB/s wr, 40 op/s 2024-03-26T08:38:48.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:47 smithi043 ceph-mon[31929]: pgmap v599: 129 pgs: 129 active+clean; 6.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 150 MiB/s wr, 40 op/s 2024-03-26T08:38:50.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:49 smithi183 ceph-mon[35818]: pgmap v600: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 122 MiB/s wr, 32 op/s 2024-03-26T08:38:50.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:49 smithi052 ceph-mon[35071]: pgmap v600: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 122 MiB/s wr, 32 op/s 2024-03-26T08:38:50.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:49 smithi043 ceph-mon[31929]: pgmap v600: 129 pgs: 129 active+clean; 6.3 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 122 MiB/s wr, 32 op/s 2024-03-26T08:38:51.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:51 smithi043 ceph-mon[31929]: pgmap v601: 129 pgs: 129 active+clean; 6.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 146 MiB/s wr, 39 op/s 2024-03-26T08:38:51.797 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:51 smithi183 ceph-mon[35818]: pgmap v601: 129 pgs: 129 active+clean; 6.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 146 MiB/s wr, 39 op/s 2024-03-26T08:38:51.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:51 smithi052 ceph-mon[35071]: pgmap v601: 129 pgs: 129 active+clean; 6.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 146 MiB/s wr, 39 op/s 2024-03-26T08:38:53.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:53 smithi043 ceph-mon[31929]: pgmap v602: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 130 MiB/s wr, 34 op/s 2024-03-26T08:38:54.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:53 smithi183 ceph-mon[35818]: pgmap v602: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 130 MiB/s wr, 34 op/s 2024-03-26T08:38:54.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:53 smithi052 ceph-mon[35071]: pgmap v602: 129 pgs: 129 active+clean; 6.8 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 130 MiB/s wr, 34 op/s 2024-03-26T08:38:55.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:55 smithi043 ceph-mon[31929]: pgmap v603: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 136 MiB/s wr, 35 op/s 2024-03-26T08:38:56.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:55 smithi183 ceph-mon[35818]: pgmap v603: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 136 MiB/s wr, 35 op/s 2024-03-26T08:38:56.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:55 smithi052 ceph-mon[35071]: pgmap v603: 129 pgs: 129 active+clean; 7.1 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 136 MiB/s wr, 35 op/s 2024-03-26T08:38:57.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:57 smithi043 ceph-mon[31929]: pgmap v604: 129 pgs: 129 active+clean; 7.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 155 MiB/s wr, 41 op/s 2024-03-26T08:38:58.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:57 smithi183 ceph-mon[35818]: pgmap v604: 129 pgs: 129 active+clean; 7.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 155 MiB/s wr, 41 op/s 2024-03-26T08:38:58.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:57 smithi052 ceph-mon[35071]: pgmap v604: 129 pgs: 129 active+clean; 7.6 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 155 MiB/s wr, 41 op/s 2024-03-26T08:38:59.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:38:59 smithi043 ceph-mon[31929]: pgmap v605: 129 pgs: 129 active+clean; 7.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 127 MiB/s wr, 33 op/s 2024-03-26T08:39:00.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:38:59 smithi183 ceph-mon[35818]: pgmap v605: 129 pgs: 129 active+clean; 7.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 127 MiB/s wr, 33 op/s 2024-03-26T08:39:00.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:38:59 smithi052 ceph-mon[35071]: pgmap v605: 129 pgs: 129 active+clean; 7.7 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 127 MiB/s wr, 33 op/s 2024-03-26T08:39:01.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:01 smithi043 ceph-mon[31929]: pgmap v606: 129 pgs: 129 active+clean; 8.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 152 MiB/s wr, 40 op/s 2024-03-26T08:39:02.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:01 smithi183 ceph-mon[35818]: pgmap v606: 129 pgs: 129 active+clean; 8.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 152 MiB/s wr, 40 op/s 2024-03-26T08:39:02.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:01 smithi052 ceph-mon[35071]: pgmap v606: 129 pgs: 129 active+clean; 8.1 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 152 MiB/s wr, 40 op/s 2024-03-26T08:39:04.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:03 smithi183 ceph-mon[35818]: pgmap v607: 129 pgs: 129 active+clean; 8.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 139 MiB/s wr, 36 op/s 2024-03-26T08:39:04.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:03 smithi052 ceph-mon[35071]: pgmap v607: 129 pgs: 129 active+clean; 8.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 139 MiB/s wr, 36 op/s 2024-03-26T08:39:04.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:03 smithi043 ceph-mon[31929]: pgmap v607: 129 pgs: 129 active+clean; 8.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 139 MiB/s wr, 36 op/s 2024-03-26T08:39:06.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:05 smithi183 ceph-mon[35818]: pgmap v608: 129 pgs: 129 active+clean; 8.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 142 MiB/s wr, 37 op/s 2024-03-26T08:39:06.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:05 smithi052 ceph-mon[35071]: pgmap v608: 129 pgs: 129 active+clean; 8.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 142 MiB/s wr, 37 op/s 2024-03-26T08:39:06.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:05 smithi043 ceph-mon[31929]: pgmap v608: 129 pgs: 129 active+clean; 8.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 142 MiB/s wr, 37 op/s 2024-03-26T08:39:08.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:07 smithi183 ceph-mon[35818]: pgmap v609: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 152 MiB/s wr, 40 op/s 2024-03-26T08:39:08.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:07 smithi052 ceph-mon[35071]: pgmap v609: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 152 MiB/s wr, 40 op/s 2024-03-26T08:39:08.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:07 smithi043 ceph-mon[31929]: pgmap v609: 129 pgs: 129 active+clean; 8.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 152 MiB/s wr, 40 op/s 2024-03-26T08:39:10.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:09 smithi183 ceph-mon[35818]: pgmap v610: 129 pgs: 129 active+clean; 9.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 120 MiB/s wr, 31 op/s 2024-03-26T08:39:10.074 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:09 smithi052 ceph-mon[35071]: pgmap v610: 129 pgs: 129 active+clean; 9.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 120 MiB/s wr, 31 op/s 2024-03-26T08:39:10.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:09 smithi043 ceph-mon[31929]: pgmap v610: 129 pgs: 129 active+clean; 9.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 120 MiB/s wr, 31 op/s 2024-03-26T08:39:11.954 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:11 smithi043 ceph-mon[31929]: pgmap v611: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 145 MiB/s wr, 38 op/s 2024-03-26T08:39:12.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:11 smithi183 ceph-mon[35818]: pgmap v611: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 145 MiB/s wr, 38 op/s 2024-03-26T08:39:12.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:11 smithi052 ceph-mon[35071]: pgmap v611: 129 pgs: 129 active+clean; 9.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 145 MiB/s wr, 38 op/s 2024-03-26T08:39:13.189 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:39:13.189 INFO:tasks.workunit.client.0.smithi043.stdout: Children see throughput for 1 initial writers = 147129.69 kB/sec 2024-03-26T08:39:13.189 INFO:tasks.workunit.client.0.smithi043.stdout: Parent sees throughput for 1 initial writers = 147125.08 kB/sec 2024-03-26T08:39:13.189 INFO:tasks.workunit.client.0.smithi043.stdout: Min throughput per process = 147129.69 kB/sec 2024-03-26T08:39:13.189 INFO:tasks.workunit.client.0.smithi043.stdout: Max throughput per process = 147129.69 kB/sec 2024-03-26T08:39:13.189 INFO:tasks.workunit.client.0.smithi043.stdout: Avg throughput per process = 147129.69 kB/sec 2024-03-26T08:39:13.189 INFO:tasks.workunit.client.0.smithi043.stdout: Min xfer = 10485760.00 kB 2024-03-26T08:39:14.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:13 smithi183 ceph-mon[35818]: pgmap v612: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 129 MiB/s wr, 33 op/s 2024-03-26T08:39:14.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:13 smithi052 ceph-mon[35071]: pgmap v612: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 129 MiB/s wr, 33 op/s 2024-03-26T08:39:14.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:13 smithi043 ceph-mon[31929]: pgmap v612: 129 pgs: 129 active+clean; 9.6 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 129 MiB/s wr, 33 op/s 2024-03-26T08:39:16.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:15 smithi183 ceph-mon[35818]: pgmap v613: 129 pgs: 129 active+clean; 9.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 130 MiB/s wr, 34 op/s 2024-03-26T08:39:16.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:15 smithi052 ceph-mon[35071]: pgmap v613: 129 pgs: 129 active+clean; 9.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 130 MiB/s wr, 34 op/s 2024-03-26T08:39:16.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:15 smithi043 ceph-mon[31929]: pgmap v613: 129 pgs: 129 active+clean; 9.8 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 130 MiB/s wr, 34 op/s 2024-03-26T08:39:18.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:17 smithi183 ceph-mon[35818]: pgmap v614: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 127 MiB/s wr, 34 op/s 2024-03-26T08:39:18.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:17 smithi052 ceph-mon[35071]: pgmap v614: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 127 MiB/s wr, 34 op/s 2024-03-26T08:39:18.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:17 smithi043 ceph-mon[31929]: pgmap v614: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 127 MiB/s wr, 34 op/s 2024-03-26T08:39:20.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:19 smithi183 ceph-mon[35818]: pgmap v615: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 103 MiB/s wr, 27 op/s 2024-03-26T08:39:20.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:19 smithi052 ceph-mon[35071]: pgmap v615: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 103 MiB/s wr, 27 op/s 2024-03-26T08:39:20.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:19 smithi043 ceph-mon[31929]: pgmap v615: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 103 MiB/s wr, 27 op/s 2024-03-26T08:39:22.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:21 smithi183 ceph-mon[35818]: pgmap v616: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 165 MiB/s wr, 43 op/s 2024-03-26T08:39:22.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:21 smithi052 ceph-mon[35071]: pgmap v616: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 165 MiB/s wr, 43 op/s 2024-03-26T08:39:22.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:21 smithi043 ceph-mon[31929]: pgmap v616: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 165 MiB/s wr, 43 op/s 2024-03-26T08:39:24.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:23 smithi183 ceph-mon[35818]: pgmap v617: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 180 MiB/s wr, 46 op/s 2024-03-26T08:39:24.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:23 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:39:24.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:23 smithi052 ceph-mon[35071]: pgmap v617: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 180 MiB/s wr, 46 op/s 2024-03-26T08:39:24.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:23 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:39:24.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:23 smithi043 ceph-mon[31929]: pgmap v617: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 180 MiB/s wr, 46 op/s 2024-03-26T08:39:24.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:23 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:39:25.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:24 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:39:25.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:24 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:39:25.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:24 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:39:25.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:24 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:39:25.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:24 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:25.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:24 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:39:25.049 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:24 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:39:25.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:24 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:39:25.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:24 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:39:25.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:24 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:39:25.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:24 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:39:25.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:24 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:25.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:24 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:39:25.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:24 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:39:25.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:24 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:39:25.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:24 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:39:25.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:24 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:39:25.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:24 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:39:25.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:24 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:25.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:24 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:39:25.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:24 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' 2024-03-26T08:39:26.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:25 smithi183 ceph-mon[35818]: Adjusting osd_memory_target on smithi043 to 177.8M 2024-03-26T08:39:26.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:25 smithi183 ceph-mon[35818]: Unable to set osd_memory_target on smithi043 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-26T08:39:26.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:25 smithi183 ceph-mon[35818]: pgmap v618: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 233 MiB/s wr, 60 op/s 2024-03-26T08:39:26.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:25 smithi052 ceph-mon[35071]: Adjusting osd_memory_target on smithi043 to 177.8M 2024-03-26T08:39:26.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:25 smithi052 ceph-mon[35071]: Unable to set osd_memory_target on smithi043 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-26T08:39:26.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:25 smithi052 ceph-mon[35071]: pgmap v618: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 233 MiB/s wr, 60 op/s 2024-03-26T08:39:26.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:25 smithi043 ceph-mon[31929]: Adjusting osd_memory_target on smithi043 to 177.8M 2024-03-26T08:39:26.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:25 smithi043 ceph-mon[31929]: Unable to set osd_memory_target on smithi043 to 186497740: error parsing value: Value '186497740' is below minimum 939524096 2024-03-26T08:39:26.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:25 smithi043 ceph-mon[31929]: pgmap v618: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 233 MiB/s wr, 60 op/s 2024-03-26T08:39:28.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:27 smithi183 ceph-mon[35818]: pgmap v619: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 330 MiB/s wr, 85 op/s 2024-03-26T08:39:28.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:27 smithi052 ceph-mon[35071]: pgmap v619: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 330 MiB/s wr, 85 op/s 2024-03-26T08:39:28.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:27 smithi043 ceph-mon[31929]: pgmap v619: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 330 MiB/s wr, 85 op/s 2024-03-26T08:39:30.048 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:29 smithi183 ceph-mon[35818]: pgmap v620: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 338 MiB/s wr, 86 op/s 2024-03-26T08:39:30.072 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:29 smithi052 ceph-mon[35071]: pgmap v620: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 338 MiB/s wr, 86 op/s 2024-03-26T08:39:30.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:29 smithi043 ceph-mon[31929]: pgmap v620: 129 pgs: 129 active+clean; 10 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 338 MiB/s wr, 86 op/s 2024-03-26T08:39:32.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:31 smithi183 ceph-mon[35818]: pgmap v621: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 443 MiB/s wr, 113 op/s 2024-03-26T08:39:32.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:31 smithi052 ceph-mon[35071]: pgmap v621: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 443 MiB/s wr, 113 op/s 2024-03-26T08:39:32.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:31 smithi043 ceph-mon[31929]: pgmap v621: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 443 MiB/s wr, 113 op/s 2024-03-26T08:39:34.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:33 smithi183 ceph-mon[35818]: pgmap v622: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 434 MiB/s wr, 110 op/s 2024-03-26T08:39:34.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:33 smithi052 ceph-mon[35071]: pgmap v622: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 434 MiB/s wr, 110 op/s 2024-03-26T08:39:34.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:33 smithi043 ceph-mon[31929]: pgmap v622: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 434 MiB/s wr, 110 op/s 2024-03-26T08:39:36.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:35 smithi183 ceph-mon[35818]: pgmap v623: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 457 MiB/s wr, 115 op/s 2024-03-26T08:39:36.070 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:35 smithi052 ceph-mon[35071]: pgmap v623: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 457 MiB/s wr, 115 op/s 2024-03-26T08:39:36.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:35 smithi043 ceph-mon[31929]: pgmap v623: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 457 MiB/s wr, 115 op/s 2024-03-26T08:39:38.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:37 smithi183 ceph-mon[35818]: pgmap v624: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 500 MiB/s wr, 127 op/s 2024-03-26T08:39:38.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:37 smithi052 ceph-mon[35071]: pgmap v624: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 500 MiB/s wr, 127 op/s 2024-03-26T08:39:38.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:37 smithi043 ceph-mon[31929]: pgmap v624: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 500 MiB/s wr, 127 op/s 2024-03-26T08:39:38.963 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:39:38.963 INFO:tasks.workunit.client.0.smithi043.stdout: Children see throughput for 1 rewriters = 461253.41 kB/sec 2024-03-26T08:39:38.963 INFO:tasks.workunit.client.0.smithi043.stdout: Parent sees throughput for 1 rewriters = 461223.69 kB/sec 2024-03-26T08:39:38.963 INFO:tasks.workunit.client.0.smithi043.stdout: Min throughput per process = 461253.41 kB/sec 2024-03-26T08:39:38.963 INFO:tasks.workunit.client.0.smithi043.stdout: Max throughput per process = 461253.41 kB/sec 2024-03-26T08:39:38.963 INFO:tasks.workunit.client.0.smithi043.stdout: Avg throughput per process = 461253.41 kB/sec 2024-03-26T08:39:38.963 INFO:tasks.workunit.client.0.smithi043.stdout: Min xfer = 10485760.00 kB 2024-03-26T08:39:40.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:39 smithi183 ceph-mon[35818]: pgmap v625: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 411 MiB/s wr, 104 op/s 2024-03-26T08:39:40.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:39 smithi052 ceph-mon[35071]: pgmap v625: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 411 MiB/s wr, 104 op/s 2024-03-26T08:39:40.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:39 smithi043 ceph-mon[31929]: pgmap v625: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 411 MiB/s wr, 104 op/s 2024-03-26T08:39:42.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:41 smithi183 ceph-mon[35818]: pgmap v626: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 473 MiB/s wr, 121 op/s 2024-03-26T08:39:42.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:41 smithi052 ceph-mon[35071]: pgmap v626: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 473 MiB/s wr, 121 op/s 2024-03-26T08:39:42.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:41 smithi043 ceph-mon[31929]: pgmap v626: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 473 MiB/s wr, 121 op/s 2024-03-26T08:39:44.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:43 smithi183 ceph-mon[35818]: pgmap v627: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 396 MiB/s wr, 101 op/s 2024-03-26T08:39:44.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:43 smithi052 ceph-mon[35071]: pgmap v627: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 396 MiB/s wr, 101 op/s 2024-03-26T08:39:44.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:43 smithi043 ceph-mon[31929]: pgmap v627: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 396 MiB/s wr, 101 op/s 2024-03-26T08:39:44.811 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:39:44.812 INFO:tasks.workunit.client.0.smithi043.stdout: Children see throughput for 1 readers = 6020313.00 kB/sec 2024-03-26T08:39:44.812 INFO:tasks.workunit.client.0.smithi043.stdout: Parent sees throughput for 1 readers = 6017306.93 kB/sec 2024-03-26T08:39:44.812 INFO:tasks.workunit.client.0.smithi043.stdout: Min throughput per process = 6020313.00 kB/sec 2024-03-26T08:39:44.812 INFO:tasks.workunit.client.0.smithi043.stdout: Max throughput per process = 6020313.00 kB/sec 2024-03-26T08:39:44.812 INFO:tasks.workunit.client.0.smithi043.stdout: Avg throughput per process = 6020313.00 kB/sec 2024-03-26T08:39:44.812 INFO:tasks.workunit.client.0.smithi043.stdout: Min xfer = 10485760.00 kB 2024-03-26T08:39:46.047 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:45 smithi183 ceph-mon[35818]: pgmap v628: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 341 MiB/s wr, 87 op/s 2024-03-26T08:39:46.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:45 smithi052 ceph-mon[35071]: pgmap v628: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 341 MiB/s wr, 87 op/s 2024-03-26T08:39:46.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:45 smithi043 ceph-mon[31929]: pgmap v628: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 341 MiB/s wr, 87 op/s 2024-03-26T08:39:48.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:47 smithi052 ceph-mon[35071]: pgmap v629: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 269 MiB/s wr, 69 op/s 2024-03-26T08:39:48.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:47 smithi043 ceph-mon[31929]: pgmap v629: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 269 MiB/s wr, 69 op/s 2024-03-26T08:39:48.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:47 smithi183 ceph-mon[35818]: pgmap v629: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 269 MiB/s wr, 69 op/s 2024-03-26T08:39:49.551 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:39:49.551 INFO:tasks.workunit.client.0.smithi043.stdout: Children see throughput for 1 re-readers = 4021643.50 kB/sec 2024-03-26T08:39:49.551 INFO:tasks.workunit.client.0.smithi043.stdout: Parent sees throughput for 1 re-readers = 4020257.38 kB/sec 2024-03-26T08:39:49.551 INFO:tasks.workunit.client.0.smithi043.stdout: Min throughput per process = 4021643.50 kB/sec 2024-03-26T08:39:49.551 INFO:tasks.workunit.client.0.smithi043.stdout: Max throughput per process = 4021643.50 kB/sec 2024-03-26T08:39:49.551 INFO:tasks.workunit.client.0.smithi043.stdout: Avg throughput per process = 4021643.50 kB/sec 2024-03-26T08:39:49.551 INFO:tasks.workunit.client.0.smithi043.stdout: Min xfer = 10485760.00 kB 2024-03-26T08:39:50.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:49 smithi052 ceph-mon[35071]: pgmap v630: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 156 MiB/s wr, 40 op/s 2024-03-26T08:39:50.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:49 smithi043 ceph-mon[31929]: pgmap v630: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 156 MiB/s wr, 40 op/s 2024-03-26T08:39:50.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:49 smithi183 ceph-mon[35818]: pgmap v630: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 156 MiB/s wr, 40 op/s 2024-03-26T08:39:52.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:51 smithi052 ceph-mon[35071]: pgmap v631: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 131 MiB/s wr, 35 op/s 2024-03-26T08:39:52.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:51 smithi043 ceph-mon[31929]: pgmap v631: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 131 MiB/s wr, 35 op/s 2024-03-26T08:39:52.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:51 smithi183 ceph-mon[35818]: pgmap v631: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 131 MiB/s wr, 35 op/s 2024-03-26T08:39:54.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:53 smithi052 ceph-mon[35071]: pgmap v632: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 42 MiB/s wr, 12 op/s 2024-03-26T08:39:54.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:53 smithi043 ceph-mon[31929]: pgmap v632: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 42 MiB/s wr, 12 op/s 2024-03-26T08:39:54.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:53 smithi183 ceph-mon[35818]: pgmap v632: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 42 MiB/s wr, 12 op/s 2024-03-26T08:39:56.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:55 smithi043 ceph-mon[31929]: pgmap v633: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 4.3 MiB/s wr, 2 op/s 2024-03-26T08:39:56.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:55 smithi183 ceph-mon[35818]: pgmap v633: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 4.3 MiB/s wr, 2 op/s 2024-03-26T08:39:56.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:55 smithi052 ceph-mon[35071]: pgmap v633: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 4.3 MiB/s wr, 2 op/s 2024-03-26T08:39:58.100 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:57 smithi183 ceph-mon[35818]: pgmap v634: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-03-26T08:39:58.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:57 smithi043 ceph-mon[31929]: pgmap v634: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-03-26T08:39:58.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:57 smithi052 ceph-mon[35071]: pgmap v634: 129 pgs: 129 active+clean; 10 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-03-26T08:39:58.593 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:39:58.593 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:39:58.596 INFO:tasks.workunit.client.0.smithi043.stdout: 2024-03-26T08:39:58.596 INFO:tasks.workunit.client.0.smithi043.stdout:iozone test complete. 2024-03-26T08:39:58.600 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-26T08:39:58.600 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-26T08:39:58.740 INFO:tasks.workunit:Stopping ['suites/iozone.sh'] on client.0... 2024-03-26T08:39:58.740 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-26T08:39:59.850 DEBUG:teuthology.parallel:result is None 2024-03-26T08:39:59.851 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-26T08:39:59.956 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-26T08:39:59.956 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-26T08:39:59.968 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:40:00.090 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:39:59 smithi043 ceph-mon[31929]: pgmap v635: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 17 op/s 2024-03-26T08:40:00.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.197+0000 7f04e0c00640 1 -- 172.21.15.43:0/82321861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04dc1026d0 msgr2=0x7f04dc102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:00.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.197+0000 7f04e0c00640 1 --2- 172.21.15.43:0/82321861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04dc1026d0 0x7f04dc102b50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009900 tx=0x7f04c802bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:00.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.198+0000 7f04e0c00640 1 -- 172.21.15.43:0/82321861 shutdown_connections 2024-03-26T08:40:00.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.198+0000 7f04e0c00640 1 --2- 172.21.15.43:0/82321861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04dc103090 0x7f04dc109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:00.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.198+0000 7f04e0c00640 1 --2- 172.21.15.43:0/82321861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04dc1026d0 0x7f04dc102b50 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:00.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.198+0000 7f04e0c00640 1 --2- 172.21.15.43:0/82321861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04dc1014e0 0x7f04dc101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:00.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.198+0000 7f04e0c00640 1 -- 172.21.15.43:0/82321861 >> 172.21.15.43:0/82321861 conn(0x7f04dc0fca30 msgr2=0x7f04dc0fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:00.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.198+0000 7f04e0c00640 1 -- 172.21.15.43:0/82321861 shutdown_connections 2024-03-26T08:40:00.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.198+0000 7f04e0c00640 1 -- 172.21.15.43:0/82321861 wait complete. 2024-03-26T08:40:00.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.199+0000 7f04e0c00640 1 Processor -- start 2024-03-26T08:40:00.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.199+0000 7f04e0c00640 1 -- start start 2024-03-26T08:40:00.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.200+0000 7f04e0c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04dc1014e0 0x7f04dc199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:00.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.200+0000 7f04e0c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04dc1026d0 0x7f04dc199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:00.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.200+0000 7f04e0c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04dc103090 0x7f04dc1a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:00.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.200+0000 7f04e0c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc10d070 con 0x7f04dc1026d0 2024-03-26T08:40:00.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.200+0000 7f04e0c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc10cef0 con 0x7f04dc1014e0 2024-03-26T08:40:00.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.200+0000 7f04e0c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc10d1f0 con 0x7f04dc103090 2024-03-26T08:40:00.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.200+0000 7f04daa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04dc1026d0 0x7f04dc199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:00.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.200+0000 7f04daa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04dc1026d0 0x7f04dc199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39882/0 (socket says 172.21.15.43:39882) 2024-03-26T08:40:00.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.200+0000 7f04daa00640 1 -- 172.21.15.43:0/3230141491 learned_addr learned my addr 172.21.15.43:0/3230141491 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:40:00.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.200+0000 7f04db400640 1 --2- 172.21.15.43:0/3230141491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04dc1014e0 0x7f04dc199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:00.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.200+0000 7f04dbe00640 1 --2- 172.21.15.43:0/3230141491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04dc103090 0x7f04dc1a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:00.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.201+0000 7f04daa00640 1 -- 172.21.15.43:0/3230141491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04dc103090 msgr2=0x7f04dc1a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:00.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.201+0000 7f04daa00640 1 --2- 172.21.15.43:0/3230141491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04dc103090 0x7f04dc1a0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:00.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.201+0000 7f04daa00640 1 -- 172.21.15.43:0/3230141491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04dc1014e0 msgr2=0x7f04dc199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:00.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.201+0000 7f04daa00640 1 --2- 172.21.15.43:0/3230141491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04dc1014e0 0x7f04dc199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:00.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.201+0000 7f04daa00640 1 -- 172.21.15.43:0/3230141491 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04dc1a1560 con 0x7f04dc1026d0 2024-03-26T08:40:00.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.201+0000 7f04daa00640 1 --2- 172.21.15.43:0/3230141491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04dc1026d0 0x7f04dc199ed0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f04c8000c00 tx=0x7f04c80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:00.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.201+0000 7f04d3e00640 1 -- 172.21.15.43:0/3230141491 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c8043020 con 0x7f04dc1026d0 2024-03-26T08:40:00.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.201+0000 7f04d3e00640 1 -- 172.21.15.43:0/3230141491 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f04c80054c0 con 0x7f04dc1026d0 2024-03-26T08:40:00.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.201+0000 7f04d3e00640 1 -- 172.21.15.43:0/3230141491 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c8003d40 con 0x7f04dc1026d0 2024-03-26T08:40:00.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.201+0000 7f04e0c00640 1 -- 172.21.15.43:0/3230141491 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04dc1a17f0 con 0x7f04dc1026d0 2024-03-26T08:40:00.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.202+0000 7f04e0c00640 1 -- 172.21.15.43:0/3230141491 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04dc1a1d90 con 0x7f04dc1026d0 2024-03-26T08:40:00.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.203+0000 7f04e0c00640 1 -- 172.21.15.43:0/3230141491 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04dc1a2050 con 0x7f04dc1026d0 2024-03-26T08:40:00.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.206+0000 7f04d3e00640 1 -- 172.21.15.43:0/3230141491 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f04c8046020 con 0x7f04dc1026d0 2024-03-26T08:40:00.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.208+0000 7f04d3e00640 1 --2- 172.21.15.43:0/3230141491 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f04c00a2880 0x7f04c00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:00.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.208+0000 7f04d3e00640 1 -- 172.21.15.43:0/3230141491 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f04c80f1470 con 0x7f04dc1026d0 2024-03-26T08:40:00.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.208+0000 7f04db400640 1 --2- 172.21.15.43:0/3230141491 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f04c00a2880 0x7f04c00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:00.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.209+0000 7f04db400640 1 --2- 172.21.15.43:0/3230141491 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f04c00a2880 0x7f04c00a4d40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f04cc005fd0 tx=0x7f04cc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:00.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.213+0000 7f04d3e00640 1 -- 172.21.15.43:0/3230141491 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f04c804d020 con 0x7f04dc1026d0 2024-03-26T08:40:00.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:39:59 smithi183 ceph-mon[35818]: pgmap v635: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 17 op/s 2024-03-26T08:40:00.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:39:59 smithi052 ceph-mon[35071]: pgmap v635: 129 pgs: 129 active+clean; 9.3 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 17 op/s 2024-03-26T08:40:00.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.409+0000 7f04e0c00640 1 -- 172.21.15.43:0/3230141491 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04dc1a2220 con 0x7f04dc1026d0 2024-03-26T08:40:00.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.413+0000 7f04d3e00640 1 -- 172.21.15.43:0/3230141491 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24670 (secure 0 0 0) 0x7f04c80be250 con 0x7f04dc1026d0 2024-03-26T08:40:00.405 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:40:00.406 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":94,"fsid":"88759420-eb49-11ee-95d0-87774f69a715","created":"2024-03-26T08:20:27.817107+0000","modified":"2024-03-26T08:32:38.256877+0000","last_up_change":"2024-03-26T08:30:10.520976+0000","last_in_change":"2024-03-26T08:29:46.394360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:24:54.852427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:31:19.924902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:31:21.323881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T08:31:24.951015+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"28670673-9193-4fa3-9cc0-c1a5a6b27284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6803","nonce":936601146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6805","nonce":936601146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6809","nonce":936601146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6807","nonce":936601146}]},"public_addr":"172.21.15.43:6803/936601146","cluster_addr":"172.21.15.43:6805/936601146","heartbeat_back_addr":"172.21.15.43:6809/936601146","heartbeat_front_addr":"172.21.15.43:6807/936601146","state":["exists","up"]},{"osd":1,"uuid":"fa8ad6b6-277a-42b8-b0f8-b36fc219de2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6801","nonce":3268412328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6803","nonce":3268412328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6807","nonce":3268412328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6805","nonce":3268412328}]},"public_addr":"172.21.15.52:6801/3268412328","cluster_addr":"172.21.15.52:6803/3268412328","heartbeat_back_addr":"172.21.15.52:6807/3268412328","heartbeat_front_addr":"172.21.15.52:6805/3268412328","state":["exists","up"]},{"osd":2,"uuid":"ec0039c3-f9bd-419a-81de-fa4418359808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6801","nonce":2103180908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6803","nonce":2103180908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6807","nonce":2103180908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6805","nonce":2103180908}]},"public_addr":"172.21.15.183:6801/2103180908","cluster_addr":"172.21.15.183:6803/2103180908","heartbeat_back_addr":"172.21.15.183:6807/2103180908","heartbeat_front_addr":"172.21.15.183:6805/2103180908","state":["exists","up"]},{"osd":3,"uuid":"e7dcbf00-6586-4664-a69e-3cb0df8dfd30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6811","nonce":3722846033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6813","nonce":3722846033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6817","nonce":3722846033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6815","nonce":3722846033}]},"public_addr":"172.21.15.43:6811/3722846033","cluster_addr":"172.21.15.43:6813/3722846033","heartbeat_back_addr":"172.21.15.43:6817/3722846033","heartbeat_front_addr":"172.21.15.43:6815/3722846033","state":["exists","up"]},{"osd":4,"uuid":"db089a49-5e98-432d-9687-cdc044114c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6809","nonce":4028436523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6811","nonce":4028436523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6815","nonce":4028436523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6813","nonce":4028436523}]},"public_addr":"172.21.15.52:6809/4028436523","cluster_addr":"172.21.15.52:6811/4028436523","heartbeat_back_addr":"172.21.15.52:6815/4028436523","heartbeat_front_addr":"172.21.15.52:6813/4028436523","state":["exists","up"]},{"osd":5,"uuid":"556ac804-02ea-423b-a599-55baf5be4ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6809","nonce":787132252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6811","nonce":787132252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6815","nonce":787132252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6813","nonce":787132252}]},"public_addr":"172.21.15.183:6809/787132252","cluster_addr":"172.21.15.183:6811/787132252","heartbeat_back_addr":"172.21.15.183:6815/787132252","heartbeat_front_addr":"172.21.15.183:6813/787132252","state":["exists","up"]},{"osd":6,"uuid":"3de2a49d-6cef-4580-9d8a-f90cdfcecd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6819","nonce":4206861158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6821","nonce":4206861158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6825","nonce":4206861158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6823","nonce":4206861158}]},"public_addr":"172.21.15.43:6819/4206861158","cluster_addr":"172.21.15.43:6821/4206861158","heartbeat_back_addr":"172.21.15.43:6825/4206861158","heartbeat_front_addr":"172.21.15.43:6823/4206861158","state":["exists","up"]},{"osd":7,"uuid":"a3b832f7-7d7d-4a62-b679-db6da98275d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6817","nonce":3452064146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6819","nonce":3452064146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6823","nonce":3452064146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6821","nonce":3452064146}]},"public_addr":"172.21.15.52:6817/3452064146","cluster_addr":"172.21.15.52:6819/3452064146","heartbeat_back_addr":"172.21.15.52:6823/3452064146","heartbeat_front_addr":"172.21.15.52:6821/3452064146","state":["exists","up"]},{"osd":8,"uuid":"ddc6b07f-d703-414d-a59a-02316376b13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6817","nonce":1023131725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6819","nonce":1023131725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6823","nonce":1023131725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6821","nonce":1023131725}]},"public_addr":"172.21.15.183:6817/1023131725","cluster_addr":"172.21.15.183:6819/1023131725","heartbeat_back_addr":"172.21.15.183:6823/1023131725","heartbeat_front_addr":"172.21.15.183:6821/1023131725","state":["exists","up"]},{"osd":9,"uuid":"ac16bb0e-6232-45c8-b16a-4d878874c588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6827","nonce":3935195657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6829","nonce":3935195657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6833","nonce":3935195657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6831","nonce":3935195657}]},"public_addr":"172.21.15.43:6827/3935195657","cluster_addr":"172.21.15.43:6829/3935195657","heartbeat_back_addr":"172.21.15.43:6833/3935195657","heartbeat_front_addr":"172.21.15.43:6831/3935195657","state":["exists","up"]},{"osd":10,"uuid":"fd839188-f398-4cff-aa7b-36522ff1a8a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6825","nonce":377628320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6827","nonce":377628320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6831","nonce":377628320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6829","nonce":377628320}]},"public_addr":"172.21.15.52:6825/377628320","cluster_addr":"172.21.15.52:6827/377628320","heartbeat_back_addr":"172.21.15.52:6831/377628320","heartbeat_front_addr":"172.21.15.52:6829/377628320","state":["exists","up"]},{"osd":11,"uuid":"a40282a0-0c64-49ee-b64e-6f3f8119e827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6825","nonce":1373086433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6827","nonce":1373086433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6831","nonce":1373086433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6829","nonce":1373086433}]},"public_addr":"172.21.15.183:6825/1373086433","cluster_addr":"172.21.15.183:6827/1373086433","heartbeat_back_addr":"172.21.15.183:6831/1373086433","heartbeat_front_addr":"172.21.15.183:6829/1373086433","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:23:43.478905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:17.573532+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:50.286610+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:25:27.407663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:03.081294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:36.057416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:12.062013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:47.032271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:21.023201+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:57.364739+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:29:34.039995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:30:08.827577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6801/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/4240775295":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/510203140":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/4251957687":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:6801/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1885244665":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/298320867":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1301092625":"2024-03-27T08:21:36.149092+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:40:00.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.416+0000 7f04e0c00640 1 -- 172.21.15.43:0/3230141491 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f04c00a2880 msgr2=0x7f04c00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:00.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.416+0000 7f04e0c00640 1 --2- 172.21.15.43:0/3230141491 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f04c00a2880 0x7f04c00a4d40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f04cc005fd0 tx=0x7f04cc005850 comp rx=0 tx=0).stop 2024-03-26T08:40:00.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.416+0000 7f04e0c00640 1 -- 172.21.15.43:0/3230141491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04dc1026d0 msgr2=0x7f04dc199ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:00.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.416+0000 7f04e0c00640 1 --2- 172.21.15.43:0/3230141491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04dc1026d0 0x7f04dc199ed0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f04c8000c00 tx=0x7f04c80042f0 comp rx=0 tx=0).stop 2024-03-26T08:40:00.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.416+0000 7f04e0c00640 1 -- 172.21.15.43:0/3230141491 shutdown_connections 2024-03-26T08:40:00.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.416+0000 7f04e0c00640 1 --2- 172.21.15.43:0/3230141491 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f04c00a2880 0x7f04c00a4d40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:00.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.416+0000 7f04e0c00640 1 --2- 172.21.15.43:0/3230141491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04dc103090 0x7f04dc1a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:00.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.416+0000 7f04e0c00640 1 --2- 172.21.15.43:0/3230141491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04dc1026d0 0x7f04dc199ed0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:00.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.416+0000 7f04e0c00640 1 --2- 172.21.15.43:0/3230141491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04dc1014e0 0x7f04dc199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:00.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.416+0000 7f04e0c00640 1 -- 172.21.15.43:0/3230141491 >> 172.21.15.43:0/3230141491 conn(0x7f04dc0fca30 msgr2=0x7f04dc0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:00.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.416+0000 7f04e0c00640 1 -- 172.21.15.43:0/3230141491 shutdown_connections 2024-03-26T08:40:00.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.416+0000 7f04e0c00640 1 -- 172.21.15.43:0/3230141491 wait complete. 2024-03-26T08:40:00.426 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T08:24:54.852427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T08:31:19.924902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T08:31:21.323881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T08:31:24.951015+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T08:40:00.426 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T08:40:00.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.662+0000 7f484ae00640 1 -- 172.21.15.43:0/4068355500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4844076060 msgr2=0x7f4844076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:00.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.662+0000 7f484ae00640 1 --2- 172.21.15.43:0/4068355500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4844076060 0x7f4844076480 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4834009900 tx=0x7f483402bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:00.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.663+0000 7f484ae00640 1 -- 172.21.15.43:0/4068355500 shutdown_connections 2024-03-26T08:40:00.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.663+0000 7f484ae00640 1 --2- 172.21.15.43:0/4068355500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4844077c40 0x7f48440784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:00.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.663+0000 7f484ae00640 1 --2- 172.21.15.43:0/4068355500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48440772a0 0x7f4844077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:00.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.663+0000 7f484ae00640 1 --2- 172.21.15.43:0/4068355500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4844076060 0x7f4844076480 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:00.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.663+0000 7f484ae00640 1 -- 172.21.15.43:0/4068355500 >> 172.21.15.43:0/4068355500 conn(0x7f48440fee50 msgr2=0x7f4844101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:00.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.663+0000 7f484ae00640 1 -- 172.21.15.43:0/4068355500 shutdown_connections 2024-03-26T08:40:00.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.663+0000 7f484ae00640 1 -- 172.21.15.43:0/4068355500 wait complete. 2024-03-26T08:40:00.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.664+0000 7f484ae00640 1 Processor -- start 2024-03-26T08:40:00.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.664+0000 7f484ae00640 1 -- start start 2024-03-26T08:40:00.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.664+0000 7f484ae00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48440772a0 0x7f484419e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:00.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.664+0000 7f484ae00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4844077c40 0x7f484419e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:00.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.664+0000 7f484ae00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48441a36a0 0x7f48441a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:00.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.664+0000 7f484ae00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4844113860 con 0x7f4844077c40 2024-03-26T08:40:00.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.664+0000 7f484ae00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48441136e0 con 0x7f48441a36a0 2024-03-26T08:40:00.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.664+0000 7f484ae00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48441139e0 con 0x7f48440772a0 2024-03-26T08:40:00.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.664+0000 7f483be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4844077c40 0x7f484419e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:00.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.664+0000 7f483be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4844077c40 0x7f484419e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39900/0 (socket says 172.21.15.43:39900) 2024-03-26T08:40:00.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.664+0000 7f483be00640 1 -- 172.21.15.43:0/101695528 learned_addr learned my addr 172.21.15.43:0/101695528 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:40:00.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.665+0000 7f4843e00640 1 --2- 172.21.15.43:0/101695528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48440772a0 0x7f484419e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:00.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.665+0000 7f4848e00640 1 --2- 172.21.15.43:0/101695528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48441a36a0 0x7f48441a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:00.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.665+0000 7f483be00640 1 -- 172.21.15.43:0/101695528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48440772a0 msgr2=0x7f484419e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:00.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.665+0000 7f483be00640 1 --2- 172.21.15.43:0/101695528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48440772a0 0x7f484419e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:00.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.665+0000 7f483be00640 1 -- 172.21.15.43:0/101695528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48441a36a0 msgr2=0x7f48441a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:00.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.665+0000 7f483be00640 1 --2- 172.21.15.43:0/101695528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48441a36a0 0x7f48441a5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:00.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.665+0000 7f483be00640 1 -- 172.21.15.43:0/101695528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48441a6100 con 0x7f4844077c40 2024-03-26T08:40:00.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.665+0000 7f483be00640 1 --2- 172.21.15.43:0/101695528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4844077c40 0x7f484419e640 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f482c00d950 tx=0x7f482c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:00.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.665+0000 7f4841600640 1 -- 172.21.15.43:0/101695528 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c017070 con 0x7f4844077c40 2024-03-26T08:40:00.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.665+0000 7f4841600640 1 -- 172.21.15.43:0/101695528 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f482c009c30 con 0x7f4844077c40 2024-03-26T08:40:00.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.665+0000 7f4841600640 1 -- 172.21.15.43:0/101695528 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c00f620 con 0x7f4844077c40 2024-03-26T08:40:00.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.665+0000 7f484ae00640 1 -- 172.21.15.43:0/101695528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48441a63f0 con 0x7f4844077c40 2024-03-26T08:40:00.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.666+0000 7f484ae00640 1 -- 172.21.15.43:0/101695528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48441a6990 con 0x7f4844077c40 2024-03-26T08:40:00.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.667+0000 7f484ae00640 1 -- 172.21.15.43:0/101695528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4844110250 con 0x7f4844077c40 2024-03-26T08:40:00.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.669+0000 7f4841600640 1 -- 172.21.15.43:0/101695528 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f482c0109a0 con 0x7f4844077c40 2024-03-26T08:40:00.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.670+0000 7f4841600640 1 --2- 172.21.15.43:0/101695528 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f48280a2880 0x7f48280a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:00.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.670+0000 7f4841600640 1 -- 172.21.15.43:0/101695528 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f482c0cea60 con 0x7f4844077c40 2024-03-26T08:40:00.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.670+0000 7f4843e00640 1 --2- 172.21.15.43:0/101695528 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f48280a2880 0x7f48280a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:00.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.671+0000 7f4843e00640 1 --2- 172.21.15.43:0/101695528 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f48280a2880 0x7f48280a4d40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4834036040 tx=0x7f4834005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:00.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.675+0000 7f4841600640 1 -- 172.21.15.43:0/101695528 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f482c009de0 con 0x7f4844077c40 2024-03-26T08:40:00.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.874+0000 7f484ae00640 1 -- 172.21.15.43:0/101695528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f48440622a0 con 0x7f4844077c40 2024-03-26T08:40:00.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.875+0000 7f4841600640 1 -- 172.21.15.43:0/101695528 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f482c09b780 con 0x7f4844077c40 2024-03-26T08:40:00.867 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-26T08:40:00.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.878+0000 7f484ae00640 1 -- 172.21.15.43:0/101695528 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f48280a2880 msgr2=0x7f48280a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:00.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.878+0000 7f484ae00640 1 --2- 172.21.15.43:0/101695528 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f48280a2880 0x7f48280a4d40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4834036040 tx=0x7f4834005fd0 comp rx=0 tx=0).stop 2024-03-26T08:40:00.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.878+0000 7f484ae00640 1 -- 172.21.15.43:0/101695528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4844077c40 msgr2=0x7f484419e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:00.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.878+0000 7f484ae00640 1 --2- 172.21.15.43:0/101695528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4844077c40 0x7f484419e640 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f482c00d950 tx=0x7f482c00de10 comp rx=0 tx=0).stop 2024-03-26T08:40:00.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.878+0000 7f484ae00640 1 -- 172.21.15.43:0/101695528 shutdown_connections 2024-03-26T08:40:00.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.878+0000 7f484ae00640 1 --2- 172.21.15.43:0/101695528 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f48280a2880 0x7f48280a4d40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:00.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.878+0000 7f484ae00640 1 --2- 172.21.15.43:0/101695528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f48441a36a0 0x7f48441a5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:00.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.878+0000 7f484ae00640 1 --2- 172.21.15.43:0/101695528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4844077c40 0x7f484419e640 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:00.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.878+0000 7f484ae00640 1 --2- 172.21.15.43:0/101695528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48440772a0 0x7f484419e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:00.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.878+0000 7f484ae00640 1 -- 172.21.15.43:0/101695528 >> 172.21.15.43:0/101695528 conn(0x7f48440fee50 msgr2=0x7f4844100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:00.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.878+0000 7f484ae00640 1 -- 172.21.15.43:0/101695528 shutdown_connections 2024-03-26T08:40:00.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:00.878+0000 7f484ae00640 1 -- 172.21.15.43:0/101695528 wait complete. 2024-03-26T08:40:00.887 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T08:40:01.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.115+0000 7f9ae4c00640 1 -- 172.21.15.43:0/137274664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ae0109770 msgr2=0x7f9ae0110000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:01.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.115+0000 7f9ae4c00640 1 --2- 172.21.15.43:0/137274664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ae0109770 0x7f9ae0110000 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8009ca0 tx=0x7f9ad802c0f0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.115+0000 7f9ae4c00640 1 -- 172.21.15.43:0/137274664 shutdown_connections 2024-03-26T08:40:01.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.115+0000 7f9ae4c00640 1 --2- 172.21.15.43:0/137274664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ae0109770 0x7f9ae0110000 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.115+0000 7f9ae4c00640 1 --2- 172.21.15.43:0/137274664 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ae0108db0 0x7f9ae0109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.115+0000 7f9ae4c00640 1 --2- 172.21.15.43:0/137274664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ae0107bc0 0x7f9ae0107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.115+0000 7f9ae4c00640 1 -- 172.21.15.43:0/137274664 >> 172.21.15.43:0/137274664 conn(0x7f9ae0076b70 msgr2=0x7f9ae0078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:01.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.115+0000 7f9ae4c00640 1 -- 172.21.15.43:0/137274664 shutdown_connections 2024-03-26T08:40:01.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.115+0000 7f9ae4c00640 1 -- 172.21.15.43:0/137274664 wait complete. 2024-03-26T08:40:01.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.116+0000 7f9ae4c00640 1 Processor -- start 2024-03-26T08:40:01.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.116+0000 7f9ae4c00640 1 -- start start 2024-03-26T08:40:01.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.116+0000 7f9ae4c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ae0107bc0 0x7f9ae019dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:01.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.116+0000 7f9ae4c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ae0108db0 0x7f9ae019e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:01.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.116+0000 7f9ae4c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ae0109770 0x7f9ae01a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:01.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.116+0000 7f9ae4c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae01136f0 con 0x7f9ae0109770 2024-03-26T08:40:01.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.116+0000 7f9ae4c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae0113570 con 0x7f9ae0107bc0 2024-03-26T08:40:01.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.116+0000 7f9ae4c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae0113870 con 0x7f9ae0108db0 2024-03-26T08:40:01.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.117+0000 7f9adfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ae0109770 0x7f9ae01a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:01.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.117+0000 7f9adfe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ae0109770 0x7f9ae01a52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39918/0 (socket says 172.21.15.43:39918) 2024-03-26T08:40:01.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.117+0000 7f9adfe00640 1 -- 172.21.15.43:0/101409413 learned_addr learned my addr 172.21.15.43:0/101409413 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:40:01.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.117+0000 7f9adf400640 1 --2- 172.21.15.43:0/101409413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ae0107bc0 0x7f9ae019dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:01.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.117+0000 7f9ad7e00640 1 --2- 172.21.15.43:0/101409413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ae0108db0 0x7f9ae019e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:01.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.117+0000 7f9adfe00640 1 -- 172.21.15.43:0/101409413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ae0108db0 msgr2=0x7f9ae019e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:01.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.117+0000 7f9adfe00640 1 --2- 172.21.15.43:0/101409413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ae0108db0 0x7f9ae019e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.117+0000 7f9adfe00640 1 -- 172.21.15.43:0/101409413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ae0107bc0 msgr2=0x7f9ae019dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:01.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.117+0000 7f9adfe00640 1 --2- 172.21.15.43:0/101409413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ae0107bc0 0x7f9ae019dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.117+0000 7f9adfe00640 1 -- 172.21.15.43:0/101409413 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae01a5960 con 0x7f9ae0109770 2024-03-26T08:40:01.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.118+0000 7f9adfe00640 1 --2- 172.21.15.43:0/101409413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ae0109770 0x7f9ae01a52f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9ad802c700 tx=0x7f9ad80328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:01.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.118+0000 7f9adcc00640 1 -- 172.21.15.43:0/101409413 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8041020 con 0x7f9ae0109770 2024-03-26T08:40:01.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.118+0000 7f9adcc00640 1 -- 172.21.15.43:0/101409413 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9ad8003e20 con 0x7f9ae0109770 2024-03-26T08:40:01.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.118+0000 7f9ae4c00640 1 -- 172.21.15.43:0/101409413 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae01a5bf0 con 0x7f9ae0109770 2024-03-26T08:40:01.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.118+0000 7f9adcc00640 1 -- 172.21.15.43:0/101409413 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8039040 con 0x7f9ae0109770 2024-03-26T08:40:01.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.118+0000 7f9ae4c00640 1 -- 172.21.15.43:0/101409413 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae01a60e0 con 0x7f9ae0109770 2024-03-26T08:40:01.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.119+0000 7f9ae4c00640 1 -- 172.21.15.43:0/101409413 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae010ee80 con 0x7f9ae0109770 2024-03-26T08:40:01.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.121+0000 7f9adcc00640 1 -- 172.21.15.43:0/101409413 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f9ad8044020 con 0x7f9ae0109770 2024-03-26T08:40:01.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.123+0000 7f9adcc00640 1 --2- 172.21.15.43:0/101409413 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f9ac40a2800 0x7f9ac40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:01.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.124+0000 7f9adcc00640 1 -- 172.21.15.43:0/101409413 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f9ad80f1350 con 0x7f9ae0109770 2024-03-26T08:40:01.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.124+0000 7f9adf400640 1 --2- 172.21.15.43:0/101409413 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f9ac40a2800 0x7f9ac40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:01.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.124+0000 7f9adf400640 1 --2- 172.21.15.43:0/101409413 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f9ac40a2800 0x7f9ac40a4cc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0005fd0 tx=0x7f9ad0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:01.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.128+0000 7f9adcc00640 1 -- 172.21.15.43:0/101409413 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9ad804b020 con 0x7f9ae0109770 2024-03-26T08:40:01.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:00 smithi043 ceph-mon[31929]: overall HEALTH_OK 2024-03-26T08:40:01.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:00 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3230141491' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:01.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:00 smithi183 ceph-mon[35818]: overall HEALTH_OK 2024-03-26T08:40:01.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:00 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3230141491' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:01.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.325+0000 7f9ae4c00640 1 -- 172.21.15.43:0/101409413 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9ae00622a0 con 0x7f9ae0109770 2024-03-26T08:40:01.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.326+0000 7f9adcc00640 1 -- 172.21.15.43:0/101409413 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9ad803b2f0 con 0x7f9ae0109770 2024-03-26T08:40:01.317 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-26T08:40:01.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.328+0000 7f9ae4c00640 1 -- 172.21.15.43:0/101409413 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f9ac40a2800 msgr2=0x7f9ac40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:01.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.328+0000 7f9ae4c00640 1 --2- 172.21.15.43:0/101409413 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f9ac40a2800 0x7f9ac40a4cc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0005fd0 tx=0x7f9ad0005850 comp rx=0 tx=0).stop 2024-03-26T08:40:01.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.328+0000 7f9ae4c00640 1 -- 172.21.15.43:0/101409413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ae0109770 msgr2=0x7f9ae01a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:01.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.328+0000 7f9ae4c00640 1 --2- 172.21.15.43:0/101409413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ae0109770 0x7f9ae01a52f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9ad802c700 tx=0x7f9ad80328a0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.328+0000 7f9ae4c00640 1 -- 172.21.15.43:0/101409413 shutdown_connections 2024-03-26T08:40:01.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.328+0000 7f9ae4c00640 1 --2- 172.21.15.43:0/101409413 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f9ac40a2800 0x7f9ac40a4cc0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.328+0000 7f9ae4c00640 1 --2- 172.21.15.43:0/101409413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ae0109770 0x7f9ae01a52f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.328+0000 7f9ae4c00640 1 --2- 172.21.15.43:0/101409413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ae0108db0 0x7f9ae019e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.328+0000 7f9ae4c00640 1 --2- 172.21.15.43:0/101409413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ae0107bc0 0x7f9ae019dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.328+0000 7f9ae4c00640 1 -- 172.21.15.43:0/101409413 >> 172.21.15.43:0/101409413 conn(0x7f9ae0076b70 msgr2=0x7f9ae010c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:01.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.328+0000 7f9ae4c00640 1 -- 172.21.15.43:0/101409413 shutdown_connections 2024-03-26T08:40:01.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.329+0000 7f9ae4c00640 1 -- 172.21.15.43:0/101409413 wait complete. 2024-03-26T08:40:01.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:00 smithi052 ceph-mon[35071]: overall HEALTH_OK 2024-03-26T08:40:01.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:00 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3230141491' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:01.339 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T08:40:01.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.565+0000 7fbe14c00640 1 -- 172.21.15.43:0/670372243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe101037c0 msgr2=0x7fbe10103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:01.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.565+0000 7fbe14c00640 1 --2- 172.21.15.43:0/670372243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe101037c0 0x7fbe10103be0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbe00009900 tx=0x7fbe0002bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:01.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.565+0000 7fbe14c00640 1 -- 172.21.15.43:0/670372243 shutdown_connections 2024-03-26T08:40:01.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.565+0000 7fbe14c00640 1 --2- 172.21.15.43:0/670372243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe10105370 0x7fbe1010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.565+0000 7fbe14c00640 1 --2- 172.21.15.43:0/670372243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe101049b0 0x7fbe10104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.565+0000 7fbe14c00640 1 --2- 172.21.15.43:0/670372243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe101037c0 0x7fbe10103be0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.565+0000 7fbe14c00640 1 -- 172.21.15.43:0/670372243 >> 172.21.15.43:0/670372243 conn(0x7fbe100fed50 msgr2=0x7fbe10101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:01.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.566+0000 7fbe14c00640 1 -- 172.21.15.43:0/670372243 shutdown_connections 2024-03-26T08:40:01.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.566+0000 7fbe14c00640 1 -- 172.21.15.43:0/670372243 wait complete. 2024-03-26T08:40:01.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.567+0000 7fbe14c00640 1 Processor -- start 2024-03-26T08:40:01.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.567+0000 7fbe14c00640 1 -- start start 2024-03-26T08:40:01.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.567+0000 7fbe14c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe101037c0 0x7fbe101999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:01.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.567+0000 7fbe14c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe101049b0 0x7fbe10199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:01.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.567+0000 7fbe14c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe10105370 0x7fbe101a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:01.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.567+0000 7fbe14c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1010f2e0 con 0x7fbe10105370 2024-03-26T08:40:01.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.567+0000 7fbe14c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1010f160 con 0x7fbe101049b0 2024-03-26T08:40:01.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.567+0000 7fbe14c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1010f460 con 0x7fbe101037c0 2024-03-26T08:40:01.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.568+0000 7fbe0fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe10105370 0x7fbe101a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:01.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.568+0000 7fbe0fe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe10105370 0x7fbe101a0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39932/0 (socket says 172.21.15.43:39932) 2024-03-26T08:40:01.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.568+0000 7fbe0fe00640 1 -- 172.21.15.43:0/1899654374 learned_addr learned my addr 172.21.15.43:0/1899654374 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:40:01.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.568+0000 7fbe0f400640 1 --2- 172.21.15.43:0/1899654374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe101037c0 0x7fbe101999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:01.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.568+0000 7fbe0ea00640 1 --2- 172.21.15.43:0/1899654374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe101049b0 0x7fbe10199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:01.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.568+0000 7fbe0fe00640 1 -- 172.21.15.43:0/1899654374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe101037c0 msgr2=0x7fbe101999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:01.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.568+0000 7fbe0fe00640 1 --2- 172.21.15.43:0/1899654374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe101037c0 0x7fbe101999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.568+0000 7fbe0fe00640 1 -- 172.21.15.43:0/1899654374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe101049b0 msgr2=0x7fbe10199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:01.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.568+0000 7fbe0fe00640 1 --2- 172.21.15.43:0/1899654374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe101049b0 0x7fbe10199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.568+0000 7fbe0fe00640 1 -- 172.21.15.43:0/1899654374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe101a1610 con 0x7fbe10105370 2024-03-26T08:40:01.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.568+0000 7fbe0fe00640 1 --2- 172.21.15.43:0/1899654374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe10105370 0x7fbe101a0fa0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800eaa0 tx=0x7fbe0800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:01.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.569+0000 7fbe07e00640 1 -- 172.21.15.43:0/1899654374 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0800cdb0 con 0x7fbe10105370 2024-03-26T08:40:01.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.569+0000 7fbe14c00640 1 -- 172.21.15.43:0/1899654374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe101a1900 con 0x7fbe10105370 2024-03-26T08:40:01.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.569+0000 7fbe07e00640 1 -- 172.21.15.43:0/1899654374 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbe08005020 con 0x7fbe10105370 2024-03-26T08:40:01.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.569+0000 7fbe07e00640 1 -- 172.21.15.43:0/1899654374 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0801f060 con 0x7fbe10105370 2024-03-26T08:40:01.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.569+0000 7fbe14c00640 1 -- 172.21.15.43:0/1899654374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe101a1e00 con 0x7fbe10105370 2024-03-26T08:40:01.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.570+0000 7fbe14c00640 1 -- 172.21.15.43:0/1899654374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe1010abd0 con 0x7fbe10105370 2024-03-26T08:40:01.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.572+0000 7fbe07e00640 1 -- 172.21.15.43:0/1899654374 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fbe080051d0 con 0x7fbe10105370 2024-03-26T08:40:01.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.574+0000 7fbe07e00640 1 --2- 172.21.15.43:0/1899654374 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fbdf40a2800 0x7fbdf40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:01.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.574+0000 7fbe07e00640 1 -- 172.21.15.43:0/1899654374 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7fbe080cf3d0 con 0x7fbe10105370 2024-03-26T08:40:01.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.574+0000 7fbe0f400640 1 --2- 172.21.15.43:0/1899654374 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fbdf40a2800 0x7fbdf40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:01.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.575+0000 7fbe0f400640 1 --2- 172.21.15.43:0/1899654374 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fbdf40a2800 0x7fbdf40a4cc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbe00006010 tx=0x7fbe00005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:01.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.578+0000 7fbe07e00640 1 -- 172.21.15.43:0/1899654374 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbe08016030 con 0x7fbe10105370 2024-03-26T08:40:01.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.783+0000 7fbe14c00640 1 -- 172.21.15.43:0/1899654374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbe100622a0 con 0x7fbe10105370 2024-03-26T08:40:01.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.784+0000 7fbe07e00640 1 -- 172.21.15.43:0/1899654374 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbe0809c0f0 con 0x7fbe10105370 2024-03-26T08:40:01.776 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-26T08:40:01.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.786+0000 7fbe14c00640 1 -- 172.21.15.43:0/1899654374 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fbdf40a2800 msgr2=0x7fbdf40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:01.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.786+0000 7fbe14c00640 1 --2- 172.21.15.43:0/1899654374 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fbdf40a2800 0x7fbdf40a4cc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbe00006010 tx=0x7fbe00005be0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.786+0000 7fbe14c00640 1 -- 172.21.15.43:0/1899654374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe10105370 msgr2=0x7fbe101a0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:01.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.786+0000 7fbe14c00640 1 --2- 172.21.15.43:0/1899654374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe10105370 0x7fbe101a0fa0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800eaa0 tx=0x7fbe0800ef60 comp rx=0 tx=0).stop 2024-03-26T08:40:01.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.787+0000 7fbe14c00640 1 -- 172.21.15.43:0/1899654374 shutdown_connections 2024-03-26T08:40:01.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.787+0000 7fbe14c00640 1 --2- 172.21.15.43:0/1899654374 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fbdf40a2800 0x7fbdf40a4cc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.787+0000 7fbe14c00640 1 --2- 172.21.15.43:0/1899654374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe10105370 0x7fbe101a0fa0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.787+0000 7fbe14c00640 1 --2- 172.21.15.43:0/1899654374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe101049b0 0x7fbe10199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.787+0000 7fbe14c00640 1 --2- 172.21.15.43:0/1899654374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe101037c0 0x7fbe101999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.787+0000 7fbe14c00640 1 -- 172.21.15.43:0/1899654374 >> 172.21.15.43:0/1899654374 conn(0x7fbe100fed50 msgr2=0x7fbe10107da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:01.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.787+0000 7fbe14c00640 1 -- 172.21.15.43:0/1899654374 shutdown_connections 2024-03-26T08:40:01.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:01.787+0000 7fbe14c00640 1 -- 172.21.15.43:0/1899654374 wait complete. 2024-03-26T08:40:01.798 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T08:40:02.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.035+0000 7f06f7600640 1 -- 172.21.15.43:0/2985507867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f01038e0 msgr2=0x7f06f0103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:02.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.035+0000 7f06f7600640 1 --2- 172.21.15.43:0/2985507867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f01038e0 0x7f06f0103d00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f06e4009900 tx=0x7f06e402bc50 comp rx=0 tx=0).stop 2024-03-26T08:40:02.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.036+0000 7f06f7600640 1 -- 172.21.15.43:0/2985507867 shutdown_connections 2024-03-26T08:40:02.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.036+0000 7f06f7600640 1 --2- 172.21.15.43:0/2985507867 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f06f0105490 0x7f06f010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:02.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.036+0000 7f06f7600640 1 --2- 172.21.15.43:0/2985507867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06f0104ad0 0x7f06f0104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:02.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.036+0000 7f06f7600640 1 --2- 172.21.15.43:0/2985507867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f01038e0 0x7f06f0103d00 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:02.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.036+0000 7f06f7600640 1 -- 172.21.15.43:0/2985507867 >> 172.21.15.43:0/2985507867 conn(0x7f06f00fee50 msgr2=0x7f06f0101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:02.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.036+0000 7f06f7600640 1 -- 172.21.15.43:0/2985507867 shutdown_connections 2024-03-26T08:40:02.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.036+0000 7f06f7600640 1 -- 172.21.15.43:0/2985507867 wait complete. 2024-03-26T08:40:02.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.037+0000 7f06f7600640 1 Processor -- start 2024-03-26T08:40:02.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.037+0000 7f06f7600640 1 -- start start 2024-03-26T08:40:02.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.038+0000 7f06f7600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06f01038e0 0x7f06f0199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:02.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.038+0000 7f06f7600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f0104ad0 0x7f06f019a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:02.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.038+0000 7f06f7600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f06f0105490 0x7f06f01a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:02.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.038+0000 7f06f7600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f010f3e0 con 0x7f06f0104ad0 2024-03-26T08:40:02.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.038+0000 7f06f7600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f010f260 con 0x7f06f0105490 2024-03-26T08:40:02.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.038+0000 7f06f7600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f010f560 con 0x7f06f01038e0 2024-03-26T08:40:02.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.038+0000 7f06efe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f0104ad0 0x7f06f019a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:02.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.038+0000 7f06efe00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f0104ad0 0x7f06f019a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39942/0 (socket says 172.21.15.43:39942) 2024-03-26T08:40:02.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.038+0000 7f06efe00640 1 -- 172.21.15.43:0/3961070671 learned_addr learned my addr 172.21.15.43:0/3961070671 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:40:02.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.038+0000 7f06f4c00640 1 --2- 172.21.15.43:0/3961070671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06f01038e0 0x7f06f0199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:02.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.038+0000 7f06f5600640 1 --2- 172.21.15.43:0/3961070671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f06f0105490 0x7f06f01a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:02.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.038+0000 7f06efe00640 1 -- 172.21.15.43:0/3961070671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06f01038e0 msgr2=0x7f06f0199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:02.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.038+0000 7f06efe00640 1 --2- 172.21.15.43:0/3961070671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06f01038e0 0x7f06f0199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:02.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.038+0000 7f06efe00640 1 -- 172.21.15.43:0/3961070671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f06f0105490 msgr2=0x7f06f01a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:02.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.038+0000 7f06efe00640 1 --2- 172.21.15.43:0/3961070671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f06f0105490 0x7f06f01a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:02.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.038+0000 7f06efe00640 1 -- 172.21.15.43:0/3961070671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f01a1790 con 0x7f06f0104ad0 2024-03-26T08:40:02.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.038+0000 7f06f5600640 1 --2- 172.21.15.43:0/3961070671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f06f0105490 0x7f06f01a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:40:02.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.039+0000 7f06efe00640 1 --2- 172.21.15.43:0/3961070671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f0104ad0 0x7f06f019a0b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f06e000d950 tx=0x7f06e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:02.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.039+0000 7f06ed600640 1 -- 172.21.15.43:0/3961070671 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e0017070 con 0x7f06f0104ad0 2024-03-26T08:40:02.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.039+0000 7f06ed600640 1 -- 172.21.15.43:0/3961070671 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f06e0009c30 con 0x7f06f0104ad0 2024-03-26T08:40:02.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.039+0000 7f06f7600640 1 -- 172.21.15.43:0/3961070671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06f01a1a80 con 0x7f06f0104ad0 2024-03-26T08:40:02.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.039+0000 7f06ed600640 1 -- 172.21.15.43:0/3961070671 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e000f620 con 0x7f06f0104ad0 2024-03-26T08:40:02.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.039+0000 7f06f7600640 1 -- 172.21.15.43:0/3961070671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f01a1fd0 con 0x7f06f0104ad0 2024-03-26T08:40:02.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.041+0000 7f06f7600640 1 -- 172.21.15.43:0/3961070671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06f010ad70 con 0x7f06f0104ad0 2024-03-26T08:40:02.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.043+0000 7f06ed600640 1 -- 172.21.15.43:0/3961070671 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f06e00109a0 con 0x7f06f0104ad0 2024-03-26T08:40:02.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.044+0000 7f06ed600640 1 --2- 172.21.15.43:0/3961070671 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f06d80a2800 0x7f06d80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:02.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.044+0000 7f06ed600640 1 -- 172.21.15.43:0/3961070671 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f06e00ceb40 con 0x7f06f0104ad0 2024-03-26T08:40:02.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.044+0000 7f06f4c00640 1 --2- 172.21.15.43:0/3961070671 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f06d80a2800 0x7f06d80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:02.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.045+0000 7f06f4c00640 1 --2- 172.21.15.43:0/3961070671 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f06d80a2800 0x7f06d80a4cc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f06e4006010 tx=0x7f06e4005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:02.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.049+0000 7f06ed600640 1 -- 172.21.15.43:0/3961070671 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f06e0009de0 con 0x7f06f0104ad0 2024-03-26T08:40:02.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:01 smithi043 ceph-mon[31929]: pgmap v636: 129 pgs: 129 active+clean; 3.7 GiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 KiB/s wr, 137 op/s 2024-03-26T08:40:02.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:01 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/101695528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:40:02.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:01 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/101409413' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:40:02.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:01 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1899654374' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:40:02.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.253+0000 7f06f7600640 1 -- 172.21.15.43:0/3961070671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f06f00622d0 con 0x7f06f0104ad0 2024-03-26T08:40:02.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.254+0000 7f06ed600640 1 -- 172.21.15.43:0/3961070671 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7f06e009b800 con 0x7f06f0104ad0 2024-03-26T08:40:02.245 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-26T08:40:02.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.256+0000 7f06f7600640 1 -- 172.21.15.43:0/3961070671 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f06d80a2800 msgr2=0x7f06d80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:02.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.256+0000 7f06f7600640 1 --2- 172.21.15.43:0/3961070671 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f06d80a2800 0x7f06d80a4cc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f06e4006010 tx=0x7f06e4005be0 comp rx=0 tx=0).stop 2024-03-26T08:40:02.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.256+0000 7f06f7600640 1 -- 172.21.15.43:0/3961070671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f0104ad0 msgr2=0x7f06f019a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:02.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.256+0000 7f06f7600640 1 --2- 172.21.15.43:0/3961070671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f0104ad0 0x7f06f019a0b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f06e000d950 tx=0x7f06e000de10 comp rx=0 tx=0).stop 2024-03-26T08:40:02.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.256+0000 7f06f7600640 1 -- 172.21.15.43:0/3961070671 shutdown_connections 2024-03-26T08:40:02.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.256+0000 7f06f7600640 1 --2- 172.21.15.43:0/3961070671 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f06d80a2800 0x7f06d80a4cc0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:02.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.256+0000 7f06f7600640 1 --2- 172.21.15.43:0/3961070671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f06f0105490 0x7f06f01a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:02.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.256+0000 7f06f7600640 1 --2- 172.21.15.43:0/3961070671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f0104ad0 0x7f06f019a0b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:02.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.256+0000 7f06f7600640 1 --2- 172.21.15.43:0/3961070671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06f01038e0 0x7f06f0199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:02.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.256+0000 7f06f7600640 1 -- 172.21.15.43:0/3961070671 >> 172.21.15.43:0/3961070671 conn(0x7f06f00fee50 msgr2=0x7f06f0107f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:02.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.257+0000 7f06f7600640 1 -- 172.21.15.43:0/3961070671 shutdown_connections 2024-03-26T08:40:02.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.257+0000 7f06f7600640 1 -- 172.21.15.43:0/3961070671 wait complete. 2024-03-26T08:40:02.266 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-26T08:40:02.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:01 smithi183 ceph-mon[35818]: pgmap v636: 129 pgs: 129 active+clean; 3.7 GiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 KiB/s wr, 137 op/s 2024-03-26T08:40:02.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:01 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/101695528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:40:02.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:01 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/101409413' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:40:02.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:01 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1899654374' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:40:02.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:01 smithi052 ceph-mon[35071]: pgmap v636: 129 pgs: 129 active+clean; 3.7 GiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 KiB/s wr, 137 op/s 2024-03-26T08:40:02.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:01 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/101695528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:40:02.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:01 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/101409413' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:40:02.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:01 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1899654374' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:40:02.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.495+0000 7fa389e00640 1 -- 172.21.15.43:0/1205265421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3841096b0 msgr2=0x7fa38410ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:02.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.495+0000 7fa389e00640 1 --2- 172.21.15.43:0/1205265421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3841096b0 0x7fa38410ff40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa37c009ca0 tx=0x7fa37c02c100 comp rx=0 tx=0).stop 2024-03-26T08:40:02.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.495+0000 7fa389e00640 1 -- 172.21.15.43:0/1205265421 shutdown_connections 2024-03-26T08:40:02.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.495+0000 7fa389e00640 1 --2- 172.21.15.43:0/1205265421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3841096b0 0x7fa38410ff40 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:02.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.495+0000 7fa389e00640 1 --2- 172.21.15.43:0/1205265421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa384108cf0 0x7fa384109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:02.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.495+0000 7fa389e00640 1 --2- 172.21.15.43:0/1205265421 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa384107b00 0x7fa384107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:02.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.495+0000 7fa389e00640 1 -- 172.21.15.43:0/1205265421 >> 172.21.15.43:0/1205265421 conn(0x7fa384076ab0 msgr2=0x7fa384078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:02.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.495+0000 7fa389e00640 1 -- 172.21.15.43:0/1205265421 shutdown_connections 2024-03-26T08:40:02.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.495+0000 7fa389e00640 1 -- 172.21.15.43:0/1205265421 wait complete. 2024-03-26T08:40:02.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.496+0000 7fa389e00640 1 Processor -- start 2024-03-26T08:40:02.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.496+0000 7fa389e00640 1 -- start start 2024-03-26T08:40:02.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.496+0000 7fa389e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa384107b00 0x7fa38419dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:02.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.496+0000 7fa389e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa384108cf0 0x7fa38419e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:02.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.496+0000 7fa389e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3841096b0 0x7fa3841a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:02.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.496+0000 7fa389e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa384113630 con 0x7fa3841096b0 2024-03-26T08:40:02.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.496+0000 7fa389e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3841134b0 con 0x7fa384108cf0 2024-03-26T08:40:02.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.496+0000 7fa389e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3841137b0 con 0x7fa384107b00 2024-03-26T08:40:02.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.497+0000 7fa389400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3841096b0 0x7fa3841a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:02.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.497+0000 7fa389400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3841096b0 0x7fa3841a5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39968/0 (socket says 172.21.15.43:39968) 2024-03-26T08:40:02.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.497+0000 7fa389400640 1 -- 172.21.15.43:0/2596703184 learned_addr learned my addr 172.21.15.43:0/2596703184 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:40:02.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.497+0000 7fa383e00640 1 --2- 172.21.15.43:0/2596703184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa384108cf0 0x7fa38419e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:02.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.497+0000 7fa388a00640 1 --2- 172.21.15.43:0/2596703184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa384107b00 0x7fa38419dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:02.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.497+0000 7fa389400640 1 -- 172.21.15.43:0/2596703184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa384107b00 msgr2=0x7fa38419dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:02.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.497+0000 7fa389400640 1 --2- 172.21.15.43:0/2596703184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa384107b00 0x7fa38419dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:02.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.497+0000 7fa389400640 1 -- 172.21.15.43:0/2596703184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa384108cf0 msgr2=0x7fa38419e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:02.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.497+0000 7fa389400640 1 --2- 172.21.15.43:0/2596703184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa384108cf0 0x7fa38419e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:02.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.497+0000 7fa389400640 1 -- 172.21.15.43:0/2596703184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3841a58e0 con 0x7fa3841096b0 2024-03-26T08:40:02.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.497+0000 7fa388a00640 1 --2- 172.21.15.43:0/2596703184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa384107b00 0x7fa38419dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:40:02.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.497+0000 7fa383e00640 1 --2- 172.21.15.43:0/2596703184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa384108cf0 0x7fa38419e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:40:02.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.497+0000 7fa389400640 1 --2- 172.21.15.43:0/2596703184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3841096b0 0x7fa3841a5240 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa37c032520 tx=0x7fa37c034880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:02.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.498+0000 7fa381600640 1 -- 172.21.15.43:0/2596703184 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c039040 con 0x7fa3841096b0 2024-03-26T08:40:02.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.498+0000 7fa381600640 1 -- 172.21.15.43:0/2596703184 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa37c044070 con 0x7fa3841096b0 2024-03-26T08:40:02.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.498+0000 7fa381600640 1 -- 172.21.15.43:0/2596703184 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c0043c0 con 0x7fa3841096b0 2024-03-26T08:40:02.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.498+0000 7fa389e00640 1 -- 172.21.15.43:0/2596703184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3841a5b70 con 0x7fa3841096b0 2024-03-26T08:40:02.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.498+0000 7fa389e00640 1 -- 172.21.15.43:0/2596703184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3841a6060 con 0x7fa3841096b0 2024-03-26T08:40:02.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.499+0000 7fa389e00640 1 -- 172.21.15.43:0/2596703184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa38410eea0 con 0x7fa3841096b0 2024-03-26T08:40:02.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.501+0000 7fa381600640 1 -- 172.21.15.43:0/2596703184 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa37c004560 con 0x7fa3841096b0 2024-03-26T08:40:02.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.504+0000 7fa381600640 1 --2- 172.21.15.43:0/2596703184 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa36c0a2800 0x7fa36c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:02.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.504+0000 7fa381600640 1 -- 172.21.15.43:0/2596703184 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7fa37c0f1c10 con 0x7fa3841096b0 2024-03-26T08:40:02.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.504+0000 7fa388a00640 1 --2- 172.21.15.43:0/2596703184 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa36c0a2800 0x7fa36c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:02.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.505+0000 7fa388a00640 1 --2- 172.21.15.43:0/2596703184 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa36c0a2800 0x7fa36c0a4cc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa38419eb70 tx=0x7fa378005e90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:02.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.506+0000 7fa381600640 1 -- 172.21.15.43:0/2596703184 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa37c0be8d0 con 0x7fa3841096b0 2024-03-26T08:40:02.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.765+0000 7fa389e00640 1 -- 172.21.15.43:0/2596703184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7fa384062340 con 0x7fa3841096b0 2024-03-26T08:40:02.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.779+0000 7fa381600640 1 -- 172.21.15.43:0/2596703184 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+739831 (secure 0 0 0) 0x7fa37c0be1d0 con 0x7fa3841096b0 2024-03-26T08:40:02.783 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:40:02.783 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-26T08:40:02.783 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 18, 2024-03-26T08:40:02.783 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.783 INFO:teuthology.orchestra.run.smithi043.stdout: "active_gid": 14152, 2024-03-26T08:40:02.784 INFO:teuthology.orchestra.run.smithi043.stdout: "active_name": "x", 2024-03-26T08:40:02.784 INFO:teuthology.orchestra.run.smithi043.stdout: "active_addrs": { 2024-03-26T08:40:02.784 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-03-26T08:40:02.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:02.784 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-03-26T08:40:02.784 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6800", 2024-03-26T08:40:02.784 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3390816575 2024-03-26T08:40:02.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:02.784 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-03-26T08:40:02.784 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6801", 2024-03-26T08:40:02.784 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3390816575 2024-03-26T08:40:02.784 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:02.784 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-03-26T08:40:02.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.785 INFO:teuthology.orchestra.run.smithi043.stdout: "active_addr": "172.21.15.43:6801/3390816575", 2024-03-26T08:40:02.785 INFO:teuthology.orchestra.run.smithi043.stdout: "active_change": "2024-03-26T08:21:36.149425+0000", 2024-03-26T08:40:02.785 INFO:teuthology.orchestra.run.smithi043.stdout: "active_mgr_features": 4540701547738038271, 2024-03-26T08:40:02.785 INFO:teuthology.orchestra.run.smithi043.stdout: "available": true, 2024-03-26T08:40:02.785 INFO:teuthology.orchestra.run.smithi043.stdout: "standbys": [ 2024-03-26T08:40:02.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:02.785 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 24109, 2024-03-26T08:40:02.785 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "z", 2024-03-26T08:40:02.785 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_features": 4540701547738038271, 2024-03-26T08:40:02.785 INFO:teuthology.orchestra.run.smithi043.stdout: "available_modules": [ 2024-03-26T08:40:02.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:02.785 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "alerts", 2024-03-26T08:40:02.785 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:02.786 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:02.786 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:02.786 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-03-26T08:40:02.786 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-03-26T08:40:02.786 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:02.786 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.786 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.786 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-03-26T08:40:02.786 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.786 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.786 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.786 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How frequently to reexamine health status", 2024-03-26T08:40:02.787 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.787 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.787 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.787 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:02.787 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:02.787 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.787 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.787 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.787 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.787 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.787 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.787 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:02.787 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:02.788 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:02.788 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:02.788 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:02.788 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:02.788 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:02.788 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:02.788 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.788 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.788 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.788 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.788 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:02.788 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:02.789 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.789 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.789 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.789 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.789 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.789 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.789 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.789 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.789 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.789 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.789 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.789 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:02.789 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:02.789 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.790 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.790 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.790 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:02.790 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.790 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.790 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:02.790 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:02.790 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:02.790 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:02.790 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:02.790 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:02.790 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:02.790 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:02.791 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.791 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.791 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.791 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.791 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:02.791 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:02.791 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.791 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.791 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.791 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.791 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.791 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.791 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.792 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.792 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.792 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.792 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.792 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_destination": { 2024-03-26T08:40:02.792 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_destination", 2024-03-26T08:40:02.792 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.792 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.792 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.792 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.792 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.792 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.792 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.792 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email address to send alerts to", 2024-03-26T08:40:02.793 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.793 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.793 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.793 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_from_name": { 2024-03-26T08:40:02.793 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_from_name", 2024-03-26T08:40:02.793 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.793 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.793 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.793 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "Ceph", 2024-03-26T08:40:02.793 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.793 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.793 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.793 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email From: name", 2024-03-26T08:40:02.793 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.794 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.794 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.794 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_host": { 2024-03-26T08:40:02.794 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_host", 2024-03-26T08:40:02.794 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.794 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.794 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.794 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.794 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.794 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.794 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.794 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP server", 2024-03-26T08:40:02.794 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.794 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.795 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.795 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_password": { 2024-03-26T08:40:02.795 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_password", 2024-03-26T08:40:02.795 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.795 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.795 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.795 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.795 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.795 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.795 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.795 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Password to authenticate with", 2024-03-26T08:40:02.795 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.795 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.796 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.796 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_port": { 2024-03-26T08:40:02.796 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_port", 2024-03-26T08:40:02.796 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:02.796 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.796 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.796 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "465", 2024-03-26T08:40:02.796 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.796 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.796 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.796 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP port", 2024-03-26T08:40:02.796 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.796 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.796 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.797 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_sender": { 2024-03-26T08:40:02.797 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_sender", 2024-03-26T08:40:02.797 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.797 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.797 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.797 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.797 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.797 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.797 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.797 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP envelope sender", 2024-03-26T08:40:02.797 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.797 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.798 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.798 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_ssl": { 2024-03-26T08:40:02.798 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_ssl", 2024-03-26T08:40:02.798 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.798 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.798 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.798 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.798 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.798 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.798 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.798 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-26T08:40:02.799 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.799 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.799 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.799 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_user": { 2024-03-26T08:40:02.799 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_user", 2024-03-26T08:40:02.799 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.799 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.799 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.799 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.799 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.799 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.799 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.799 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "User to authenticate as", 2024-03-26T08:40:02.800 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.800 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.800 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.800 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:02.800 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:02.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:02.800 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "balancer", 2024-03-26T08:40:02.800 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:02.800 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:02.800 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:02.800 INFO:teuthology.orchestra.run.smithi043.stdout: "active": { 2024-03-26T08:40:02.800 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "active", 2024-03-26T08:40:02.801 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.801 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.801 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.801 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.801 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.801 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.801 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.801 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically balance PGs across cluster", 2024-03-26T08:40:02.801 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.801 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.801 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.801 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_time": { 2024-03-26T08:40:02.801 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_time", 2024-03-26T08:40:02.802 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.802 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.802 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.802 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0000", 2024-03-26T08:40:02.802 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.802 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.802 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.802 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "beginning time of day to automatically balance", 2024-03-26T08:40:02.802 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T08:40:02.802 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.802 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.802 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_weekday": { 2024-03-26T08:40:02.803 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_weekday", 2024-03-26T08:40:02.803 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-03-26T08:40:02.803 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.803 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.803 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-03-26T08:40:02.803 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-03-26T08:40:02.803 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-03-26T08:40:02.803 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.803 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-26T08:40:02.803 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T08:40:02.803 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.803 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.803 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_max_iterations": { 2024-03-26T08:40:02.803 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_max_iterations", 2024-03-26T08:40:02.804 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-03-26T08:40:02.804 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.804 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.804 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "25", 2024-03-26T08:40:02.804 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-03-26T08:40:02.804 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "250", 2024-03-26T08:40:02.804 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.804 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-26T08:40:02.804 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.804 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.804 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.804 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_metrics": { 2024-03-26T08:40:02.804 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_metrics", 2024-03-26T08:40:02.804 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.804 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.805 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.805 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "pgs,objects,bytes", 2024-03-26T08:40:02.805 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.805 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.805 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.805 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-26T08:40:02.805 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-26T08:40:02.805 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.805 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.805 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_step": { 2024-03-26T08:40:02.805 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_step", 2024-03-26T08:40:02.805 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-03-26T08:40:02.805 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.805 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.806 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.5", 2024-03-26T08:40:02.806 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0.001", 2024-03-26T08:40:02.806 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "0.999", 2024-03-26T08:40:02.806 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.806 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "aggressiveness of optimization", 2024-03-26T08:40:02.806 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-26T08:40:02.806 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.806 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.806 INFO:teuthology.orchestra.run.smithi043.stdout: "end_time": { 2024-03-26T08:40:02.806 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_time", 2024-03-26T08:40:02.806 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.806 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.806 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.807 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2359", 2024-03-26T08:40:02.807 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.807 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.807 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.807 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ending time of day to automatically balance", 2024-03-26T08:40:02.807 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T08:40:02.807 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.807 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.807 INFO:teuthology.orchestra.run.smithi043.stdout: "end_weekday": { 2024-03-26T08:40:02.807 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_weekday", 2024-03-26T08:40:02.807 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-03-26T08:40:02.807 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.808 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.808 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-03-26T08:40:02.808 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-03-26T08:40:02.808 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-03-26T08:40:02.808 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.808 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-26T08:40:02.808 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T08:40:02.808 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.808 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.808 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:02.808 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:02.808 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.809 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.809 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.809 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.809 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.809 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.809 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:02.809 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:02.809 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:02.809 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:02.809 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:02.809 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:02.809 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:02.809 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:02.809 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.810 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.810 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.810 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.810 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:02.810 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:02.810 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.810 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.810 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.810 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.810 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.810 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.810 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.811 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.811 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.811 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.811 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.811 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:02.811 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:02.811 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.811 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.811 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.811 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:02.811 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.811 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.811 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:02.812 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:02.812 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:02.812 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:02.812 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:02.812 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:02.812 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:02.812 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:02.812 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.812 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.812 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.812 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.812 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:02.812 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:02.812 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.813 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.813 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.813 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.813 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.813 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.813 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.813 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.813 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.813 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.813 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.813 INFO:teuthology.orchestra.run.smithi043.stdout: "min_score": { 2024-03-26T08:40:02.813 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_score", 2024-03-26T08:40:02.813 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-03-26T08:40:02.814 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.814 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.814 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-03-26T08:40:02.814 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.814 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.814 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.814 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-26T08:40:02.814 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.814 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.814 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.814 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-03-26T08:40:02.814 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-03-26T08:40:02.814 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.815 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.815 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.815 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "upmap", 2024-03-26T08:40:02.815 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.815 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.815 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:02.815 INFO:teuthology.orchestra.run.smithi043.stdout: "crush-compat", 2024-03-26T08:40:02.815 INFO:teuthology.orchestra.run.smithi043.stdout: "none", 2024-03-26T08:40:02.815 INFO:teuthology.orchestra.run.smithi043.stdout: "read", 2024-03-26T08:40:02.815 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap", 2024-03-26T08:40:02.815 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap-read" 2024-03-26T08:40:02.815 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:02.815 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Balancer mode", 2024-03-26T08:40:02.815 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.816 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.816 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.816 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ids": { 2024-03-26T08:40:02.816 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_ids", 2024-03-26T08:40:02.816 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.816 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.816 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.816 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.816 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.816 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.816 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.816 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-26T08:40:02.816 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.816 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.817 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.817 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-03-26T08:40:02.817 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-03-26T08:40:02.817 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:02.817 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.817 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.817 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-03-26T08:40:02.817 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.817 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.817 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.817 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-26T08:40:02.817 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.817 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.817 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.818 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_deviation": { 2024-03-26T08:40:02.818 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_deviation", 2024-03-26T08:40:02.818 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:02.818 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.818 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.818 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-03-26T08:40:02.818 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-03-26T08:40:02.818 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.818 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.818 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-26T08:40:02.818 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-26T08:40:02.818 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.819 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.819 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_optimizations": { 2024-03-26T08:40:02.819 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_optimizations", 2024-03-26T08:40:02.819 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-03-26T08:40:02.819 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.819 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.819 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-03-26T08:40:02.819 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.819 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.819 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.819 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-26T08:40:02.819 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.819 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.819 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.820 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:02.820 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:02.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:02.820 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm", 2024-03-26T08:40:02.820 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:02.820 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:02.820 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:02.820 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_down_multiplier": { 2024-03-26T08:40:02.820 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_down_multiplier", 2024-03-26T08:40:02.820 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-03-26T08:40:02.820 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.820 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.820 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-03-26T08:40:02.821 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.821 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.821 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.821 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-26T08:40:02.821 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.821 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.821 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.821 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_refresh_rate": { 2024-03-26T08:40:02.821 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_refresh_rate", 2024-03-26T08:40:02.821 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:02.821 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.821 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.821 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "20", 2024-03-26T08:40:02.821 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.822 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.822 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.822 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-26T08:40:02.822 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.822 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.822 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.822 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_starting_port": { 2024-03-26T08:40:02.822 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_starting_port", 2024-03-26T08:40:02.822 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:02.822 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.822 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.822 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4721", 2024-03-26T08:40:02.822 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.822 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.823 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.823 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-26T08:40:02.823 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.823 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.823 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.823 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_ptrace": { 2024-03-26T08:40:02.823 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_ptrace", 2024-03-26T08:40:02.823 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.823 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.823 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.823 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.824 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.824 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.824 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.824 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-26T08:40:02.824 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-26T08:40:02.824 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.824 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.824 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_interval": { 2024-03-26T08:40:02.824 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_interval", 2024-03-26T08:40:02.824 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:02.824 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.824 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.824 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-03-26T08:40:02.825 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.825 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.825 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.825 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-26T08:40:02.825 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.825 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.825 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.825 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_memory_target_ratio": { 2024-03-26T08:40:02.825 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_memory_target_ratio", 2024-03-26T08:40:02.825 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-03-26T08:40:02.825 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.825 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.825 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.7", 2024-03-26T08:40:02.825 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.826 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.826 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.826 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-26T08:40:02.826 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.826 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.826 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.826 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm_log_destination": { 2024-03-26T08:40:02.826 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm_log_destination", 2024-03-26T08:40:02.826 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.826 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.826 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.826 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.826 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.827 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.827 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:02.827 INFO:teuthology.orchestra.run.smithi043.stdout: "file", 2024-03-26T08:40:02.827 INFO:teuthology.orchestra.run.smithi043.stdout: "file,syslog", 2024-03-26T08:40:02.827 INFO:teuthology.orchestra.run.smithi043.stdout: "syslog" 2024-03-26T08:40:02.827 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:02.827 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-26T08:40:02.827 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.827 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.827 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.827 INFO:teuthology.orchestra.run.smithi043.stdout: "cgroups_split": { 2024-03-26T08:40:02.827 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cgroups_split", 2024-03-26T08:40:02.827 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.827 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.828 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.828 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.828 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.828 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.828 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.828 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-26T08:40:02.828 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.828 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.828 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.828 INFO:teuthology.orchestra.run.smithi043.stdout: "config_checks_enabled": { 2024-03-26T08:40:02.828 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_checks_enabled", 2024-03-26T08:40:02.828 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.828 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.828 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.829 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.829 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.829 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.829 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.829 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-26T08:40:02.829 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.829 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.829 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.829 INFO:teuthology.orchestra.run.smithi043.stdout: "config_dashboard": { 2024-03-26T08:40:02.829 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_dashboard", 2024-03-26T08:40:02.829 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.829 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.829 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.829 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.830 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.830 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.830 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.830 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-26T08:40:02.830 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.830 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.830 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.830 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_alertmanager": { 2024-03-26T08:40:02.830 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_alertmanager", 2024-03-26T08:40:02.830 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.830 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.830 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.830 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-26T08:40:02.831 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.831 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.831 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.831 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-03-26T08:40:02.831 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.831 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.831 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.831 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_base": { 2024-03-26T08:40:02.831 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_base", 2024-03-26T08:40:02.831 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.831 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.831 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.831 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-26T08:40:02.831 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.832 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.832 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.832 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Container image name, without the tag", 2024-03-26T08:40:02.832 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.832 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.832 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.832 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_elasticsearch": { 2024-03-26T08:40:02.832 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_elasticsearch", 2024-03-26T08:40:02.832 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.832 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.832 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.832 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-26T08:40:02.832 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.832 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.833 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.833 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "elasticsearch container image", 2024-03-26T08:40:02.833 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.833 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.833 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.833 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_grafana": { 2024-03-26T08:40:02.833 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_grafana", 2024-03-26T08:40:02.833 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.833 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.833 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.833 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-26T08:40:02.833 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.833 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.833 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.833 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-03-26T08:40:02.834 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.834 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.834 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.834 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_haproxy": { 2024-03-26T08:40:02.834 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_haproxy", 2024-03-26T08:40:02.834 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.834 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.834 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.834 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-26T08:40:02.834 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.834 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.834 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.835 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "HAproxy container image", 2024-03-26T08:40:02.835 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.835 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.835 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.835 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_agent": { 2024-03-26T08:40:02.835 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_agent", 2024-03-26T08:40:02.835 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.835 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.835 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.835 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-26T08:40:02.835 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.835 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.835 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.836 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger agent container image", 2024-03-26T08:40:02.836 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.836 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.836 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.836 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_collector": { 2024-03-26T08:40:02.836 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_collector", 2024-03-26T08:40:02.836 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.836 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.836 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.836 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-26T08:40:02.836 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.836 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.836 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.837 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger collector container image", 2024-03-26T08:40:02.837 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.837 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.837 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.837 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_query": { 2024-03-26T08:40:02.837 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_query", 2024-03-26T08:40:02.837 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.837 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.837 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.837 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-26T08:40:02.837 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.837 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.837 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.837 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger query container image", 2024-03-26T08:40:02.838 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.838 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.838 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.838 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_keepalived": { 2024-03-26T08:40:02.838 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_keepalived", 2024-03-26T08:40:02.838 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.838 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.838 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.838 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-26T08:40:02.838 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.838 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.838 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.838 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Keepalived container image", 2024-03-26T08:40:02.839 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.839 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.839 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.839 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_loki": { 2024-03-26T08:40:02.839 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_loki", 2024-03-26T08:40:02.839 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.839 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.839 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.839 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-26T08:40:02.839 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.839 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.839 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.839 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Loki container image", 2024-03-26T08:40:02.839 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.840 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.840 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.840 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_node_exporter": { 2024-03-26T08:40:02.840 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_node_exporter", 2024-03-26T08:40:02.840 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.840 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.840 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.840 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-26T08:40:02.840 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.840 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.840 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.840 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-03-26T08:40:02.840 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.840 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.841 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.841 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_nvmeof": { 2024-03-26T08:40:02.841 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_nvmeof", 2024-03-26T08:40:02.841 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.841 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.841 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.841 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-26T08:40:02.841 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.841 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.841 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.841 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Nvme-of container image", 2024-03-26T08:40:02.841 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.841 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.841 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.842 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_prometheus": { 2024-03-26T08:40:02.842 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_prometheus", 2024-03-26T08:40:02.842 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.842 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.842 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.842 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-26T08:40:02.842 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.842 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.842 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.842 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-03-26T08:40:02.842 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.843 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.843 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.843 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_promtail": { 2024-03-26T08:40:02.843 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_promtail", 2024-03-26T08:40:02.843 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.843 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.843 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.843 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-26T08:40:02.843 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.843 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.843 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.843 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Promtail container image", 2024-03-26T08:40:02.843 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.844 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.844 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.844 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_snmp_gateway": { 2024-03-26T08:40:02.844 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_snmp_gateway", 2024-03-26T08:40:02.844 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.844 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.844 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.844 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-26T08:40:02.844 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.844 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.844 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.844 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SNMP Gateway container image", 2024-03-26T08:40:02.844 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.845 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.845 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.845 INFO:teuthology.orchestra.run.smithi043.stdout: "container_init": { 2024-03-26T08:40:02.845 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_init", 2024-03-26T08:40:02.845 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.845 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.845 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.845 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.845 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.845 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.845 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.845 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Run podman/docker with `--init`", 2024-03-26T08:40:02.846 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.846 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.846 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.846 INFO:teuthology.orchestra.run.smithi043.stdout: "daemon_cache_timeout": { 2024-03-26T08:40:02.846 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "daemon_cache_timeout", 2024-03-26T08:40:02.846 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:02.846 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.846 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.846 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-03-26T08:40:02.846 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.846 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.846 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.847 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-26T08:40:02.847 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.847 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.847 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.847 INFO:teuthology.orchestra.run.smithi043.stdout: "default_cephadm_command_timeout": { 2024-03-26T08:40:02.847 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_cephadm_command_timeout", 2024-03-26T08:40:02.847 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:02.847 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.847 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.847 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "900", 2024-03-26T08:40:02.847 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.847 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.847 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.847 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-26T08:40:02.848 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.848 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.848 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.848 INFO:teuthology.orchestra.run.smithi043.stdout: "default_registry": { 2024-03-26T08:40:02.848 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_registry", 2024-03-26T08:40:02.848 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.848 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.848 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.848 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io", 2024-03-26T08:40:02.848 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.848 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.848 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.848 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-26T08:40:02.848 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.848 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.849 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.849 INFO:teuthology.orchestra.run.smithi043.stdout: "device_cache_timeout": { 2024-03-26T08:40:02.849 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_cache_timeout", 2024-03-26T08:40:02.849 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:02.849 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.849 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.849 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1800", 2024-03-26T08:40:02.849 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.849 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.849 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.849 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache device inventory", 2024-03-26T08:40:02.849 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.849 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.850 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.850 INFO:teuthology.orchestra.run.smithi043.stdout: "device_enhanced_scan": { 2024-03-26T08:40:02.850 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_enhanced_scan", 2024-03-26T08:40:02.850 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.850 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.850 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.850 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.850 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.850 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.850 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.850 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-26T08:40:02.850 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.851 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.851 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.851 INFO:teuthology.orchestra.run.smithi043.stdout: "facts_cache_timeout": { 2024-03-26T08:40:02.851 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "facts_cache_timeout", 2024-03-26T08:40:02.851 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:02.851 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.851 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.851 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-03-26T08:40:02.851 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.851 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.851 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.851 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache host facts data", 2024-03-26T08:40:02.851 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.852 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.852 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.852 INFO:teuthology.orchestra.run.smithi043.stdout: "host_check_interval": { 2024-03-26T08:40:02.852 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "host_check_interval", 2024-03-26T08:40:02.852 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:02.852 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.852 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.852 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-03-26T08:40:02.852 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.852 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.852 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.852 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to perform a host check", 2024-03-26T08:40:02.852 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.852 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.853 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.853 INFO:teuthology.orchestra.run.smithi043.stdout: "hw_monitoring": { 2024-03-26T08:40:02.853 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hw_monitoring", 2024-03-26T08:40:02.853 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.853 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.853 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.853 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.853 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.853 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.853 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.853 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-26T08:40:02.853 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.853 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.853 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.854 INFO:teuthology.orchestra.run.smithi043.stdout: "inventory_list_all": { 2024-03-26T08:40:02.854 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "inventory_list_all", 2024-03-26T08:40:02.854 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.854 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.854 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.854 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.854 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.854 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.854 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.854 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-26T08:40:02.854 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.854 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.854 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.855 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:02.855 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:02.855 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.855 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.855 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.855 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.855 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.855 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.855 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:02.855 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:02.855 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:02.855 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:02.855 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:02.856 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:02.856 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:02.856 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:02.856 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.856 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.856 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.856 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.856 INFO:teuthology.orchestra.run.smithi043.stdout: "log_refresh_metadata": { 2024-03-26T08:40:02.856 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_refresh_metadata", 2024-03-26T08:40:02.856 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.856 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.856 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.856 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.857 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.857 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.857 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.857 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-26T08:40:02.857 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.857 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.857 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.857 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:02.857 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:02.857 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.857 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.857 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.857 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.857 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.857 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.857 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.858 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-26T08:40:02.858 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.858 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.858 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.858 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:02.858 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:02.858 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.858 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.858 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.858 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:02.858 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.858 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.859 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:02.859 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:02.859 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:02.859 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:02.859 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:02.859 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:02.859 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:02.859 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:02.859 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.859 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.859 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.859 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.859 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:02.860 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:02.860 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.860 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.860 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.860 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.860 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.860 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.860 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.860 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.860 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.860 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.860 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.860 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-26T08:40:02.860 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-26T08:40:02.860 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.861 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.861 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.861 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.861 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.861 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.861 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.861 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-26T08:40:02.861 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.861 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.861 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.861 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-26T08:40:02.861 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-26T08:40:02.862 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.862 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.862 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.862 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "*", 2024-03-26T08:40:02.862 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.862 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.862 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.862 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-26T08:40:02.862 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.863 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.863 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.863 INFO:teuthology.orchestra.run.smithi043.stdout: "max_count_per_host": { 2024-03-26T08:40:02.863 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_count_per_host", 2024-03-26T08:40:02.863 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:02.863 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.863 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.863 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-03-26T08:40:02.863 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.863 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.863 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.863 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of daemons per service per host", 2024-03-26T08:40:02.863 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.863 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.864 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.864 INFO:teuthology.orchestra.run.smithi043.stdout: "max_osd_draining_count": { 2024-03-26T08:40:02.864 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_osd_draining_count", 2024-03-26T08:40:02.864 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:02.864 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.864 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.864 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-03-26T08:40:02.864 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.864 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.864 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.864 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-26T08:40:02.864 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.864 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.865 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.865 INFO:teuthology.orchestra.run.smithi043.stdout: "migration_current": { 2024-03-26T08:40:02.865 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "migration_current", 2024-03-26T08:40:02.865 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:02.865 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.865 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.865 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.865 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.865 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.865 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.865 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "internal - do not modify", 2024-03-26T08:40:02.865 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.865 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.866 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.866 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-03-26T08:40:02.866 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-03-26T08:40:02.866 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.866 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.866 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.866 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "root", 2024-03-26T08:40:02.866 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.866 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.866 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:02.866 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm-package", 2024-03-26T08:40:02.866 INFO:teuthology.orchestra.run.smithi043.stdout: "root" 2024-03-26T08:40:02.866 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:02.867 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "mode for remote execution of cephadm", 2024-03-26T08:40:02.867 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.867 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.867 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.867 INFO:teuthology.orchestra.run.smithi043.stdout: "oob_default_addr": { 2024-03-26T08:40:02.867 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "oob_default_addr", 2024-03-26T08:40:02.867 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.867 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.867 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.867 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "169.254.1.1", 2024-03-26T08:40:02.867 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.867 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.867 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.867 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-26T08:40:02.868 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.868 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.868 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.868 INFO:teuthology.orchestra.run.smithi043.stdout: "prometheus_alerts_path": { 2024-03-26T08:40:02.868 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus_alerts_path", 2024-03-26T08:40:02.868 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.868 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.868 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.868 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-26T08:40:02.868 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.868 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.868 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.868 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-26T08:40:02.868 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.869 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.869 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.869 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_insecure": { 2024-03-26T08:40:02.869 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_insecure", 2024-03-26T08:40:02.869 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.869 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.869 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.869 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.869 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.869 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.869 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.869 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-26T08:40:02.869 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.870 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.870 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.870 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_password": { 2024-03-26T08:40:02.870 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_password", 2024-03-26T08:40:02.870 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.870 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.870 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.870 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.870 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.870 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.870 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.870 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-26T08:40:02.870 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.871 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.871 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.871 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_url": { 2024-03-26T08:40:02.871 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_url", 2024-03-26T08:40:02.871 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.871 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.871 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.871 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.871 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.871 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.871 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.871 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-26T08:40:02.872 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.872 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.872 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.872 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_username": { 2024-03-26T08:40:02.872 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_username", 2024-03-26T08:40:02.872 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.872 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.872 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.872 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.872 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.872 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.872 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.872 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-26T08:40:02.872 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.873 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.873 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.873 INFO:teuthology.orchestra.run.smithi043.stdout: "secure_monitoring_stack": { 2024-03-26T08:40:02.873 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "secure_monitoring_stack", 2024-03-26T08:40:02.873 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.873 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.873 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.873 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.873 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.873 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.873 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.873 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-26T08:40:02.873 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.874 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.874 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.874 INFO:teuthology.orchestra.run.smithi043.stdout: "service_discovery_port": { 2024-03-26T08:40:02.874 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "service_discovery_port", 2024-03-26T08:40:02.874 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:02.874 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.874 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.874 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8765", 2024-03-26T08:40:02.874 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.874 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.874 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.875 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "cephadm service discovery port", 2024-03-26T08:40:02.875 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.875 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.875 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.875 INFO:teuthology.orchestra.run.smithi043.stdout: "ssh_config_file": { 2024-03-26T08:40:02.875 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssh_config_file", 2024-03-26T08:40:02.875 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.875 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.875 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.875 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.875 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.875 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.875 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.875 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-26T08:40:02.875 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.876 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.876 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.876 INFO:teuthology.orchestra.run.smithi043.stdout: "use_agent": { 2024-03-26T08:40:02.876 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_agent", 2024-03-26T08:40:02.876 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.876 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.876 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.876 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.876 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.876 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.876 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.876 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-26T08:40:02.877 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.877 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.877 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.877 INFO:teuthology.orchestra.run.smithi043.stdout: "use_repo_digest": { 2024-03-26T08:40:02.877 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_repo_digest", 2024-03-26T08:40:02.877 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.877 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.877 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.877 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.877 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.877 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.877 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.877 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-26T08:40:02.877 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.878 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.878 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.878 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_failed_host_check": { 2024-03-26T08:40:02.878 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_failed_host_check", 2024-03-26T08:40:02.878 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.878 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.878 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.878 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.878 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.878 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.878 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.878 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if the host check fails", 2024-03-26T08:40:02.878 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.879 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.879 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.879 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_daemons": { 2024-03-26T08:40:02.879 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_daemons", 2024-03-26T08:40:02.879 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.879 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.879 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.879 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.879 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.879 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.879 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.879 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-26T08:40:02.879 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.880 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.880 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.880 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_hosts": { 2024-03-26T08:40:02.880 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_hosts", 2024-03-26T08:40:02.880 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.880 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.880 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.880 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.880 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.880 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.880 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.880 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-26T08:40:02.880 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.880 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.881 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.881 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:02.881 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:02.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:02.881 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crash", 2024-03-26T08:40:02.881 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:02.881 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:02.881 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:02.881 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:02.881 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:02.881 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.881 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.881 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.881 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.881 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.882 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.882 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:02.882 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:02.882 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:02.882 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:02.882 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:02.882 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:02.882 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:02.882 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:02.882 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.882 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.882 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.882 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.883 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:02.883 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:02.883 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.883 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.883 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.883 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.883 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.883 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.883 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.883 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.883 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.883 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.883 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.884 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:02.884 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:02.884 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.884 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.884 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.884 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:02.884 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.884 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.884 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:02.884 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:02.884 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:02.884 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:02.885 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:02.885 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:02.885 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:02.885 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:02.885 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.885 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.885 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.885 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.885 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:02.885 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:02.885 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.885 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.885 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.885 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.886 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.886 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.886 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.886 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.886 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.886 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.886 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.886 INFO:teuthology.orchestra.run.smithi043.stdout: "retain_interval": { 2024-03-26T08:40:02.886 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retain_interval", 2024-03-26T08:40:02.886 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:02.886 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.886 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.886 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "31536000", 2024-03-26T08:40:02.887 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.887 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.887 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.887 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-26T08:40:02.887 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.887 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.887 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.887 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_recent_interval": { 2024-03-26T08:40:02.887 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_recent_interval", 2024-03-26T08:40:02.887 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:02.887 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.887 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.887 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1209600", 2024-03-26T08:40:02.888 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.888 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.888 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.888 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-26T08:40:02.888 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.888 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.888 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.888 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:02.888 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:02.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:02.888 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dashboard", 2024-03-26T08:40:02.888 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:02.888 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:02.889 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:02.889 INFO:teuthology.orchestra.run.smithi043.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-26T08:40:02.889 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-26T08:40:02.889 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:02.889 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.889 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.889 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-03-26T08:40:02.889 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.889 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.889 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.889 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.889 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.889 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.889 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.890 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_HOST": { 2024-03-26T08:40:02.890 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-26T08:40:02.890 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.890 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.890 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.890 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.890 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.890 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.890 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.890 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.890 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.890 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.891 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.891 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-26T08:40:02.891 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-26T08:40:02.891 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.891 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.891 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.891 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.891 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.891 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.891 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.891 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.891 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.891 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.892 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.892 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_ENABLED": { 2024-03-26T08:40:02.892 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_ENABLED", 2024-03-26T08:40:02.892 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.892 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.892 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.892 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.892 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.892 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.892 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.892 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.892 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.892 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.892 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.893 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-26T08:40:02.893 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-26T08:40:02.893 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.893 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.893 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.893 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.893 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.893 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.893 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.893 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.893 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.893 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.893 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.893 INFO:teuthology.orchestra.run.smithi043.stdout: "ENABLE_BROWSABLE_API": { 2024-03-26T08:40:02.894 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-26T08:40:02.894 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.894 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.894 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.894 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.894 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.894 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.894 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.894 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.894 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.894 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.894 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.894 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-26T08:40:02.894 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-26T08:40:02.895 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.895 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.895 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.895 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.895 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.895 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.895 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.895 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.895 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.895 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.895 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.895 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-26T08:40:02.895 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-26T08:40:02.895 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.896 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.896 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.896 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.896 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.896 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.896 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.896 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.896 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.896 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.896 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.896 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-26T08:40:02.896 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-26T08:40:02.897 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.897 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.897 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.897 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.897 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.897 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.897 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.897 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.897 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.897 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.897 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.897 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-26T08:40:02.897 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-26T08:40:02.898 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.898 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.898 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.898 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.898 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.898 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.898 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.898 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.898 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.898 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.898 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.898 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-26T08:40:02.898 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-26T08:40:02.899 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.899 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.899 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.899 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.899 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.899 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.899 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.899 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.899 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.899 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.899 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.899 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-26T08:40:02.900 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-26T08:40:02.900 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.900 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.900 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.900 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.900 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.900 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.900 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.900 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.900 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.900 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.900 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.900 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-26T08:40:02.900 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-26T08:40:02.901 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.901 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.901 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.901 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.901 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.901 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.901 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.901 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.901 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.901 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.901 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.901 INFO:teuthology.orchestra.run.smithi043.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-26T08:40:02.901 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-26T08:40:02.902 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.902 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.902 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.902 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.902 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.902 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.902 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.902 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.902 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.902 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.902 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.902 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_PASSWORD": { 2024-03-26T08:40:02.902 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-26T08:40:02.903 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.903 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.903 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.903 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-03-26T08:40:02.903 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.903 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.903 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.903 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.903 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.903 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.903 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.903 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-26T08:40:02.903 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-26T08:40:02.904 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.904 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.904 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.904 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.904 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.904 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.904 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.904 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.904 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.904 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.904 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.904 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_URL": { 2024-03-26T08:40:02.904 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_URL", 2024-03-26T08:40:02.905 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.905 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.905 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.905 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.905 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.905 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.905 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.905 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.905 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.905 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.905 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.905 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_USERNAME": { 2024-03-26T08:40:02.906 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-26T08:40:02.906 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.906 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.906 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.906 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-03-26T08:40:02.906 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.906 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.906 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.906 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.906 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.906 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.906 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.906 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-26T08:40:02.907 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-26T08:40:02.907 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.907 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.907 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.907 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.907 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.907 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.907 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.907 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.907 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.907 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.907 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.908 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-26T08:40:02.908 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-26T08:40:02.908 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.908 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.908 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.908 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.908 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.908 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.908 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.908 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.908 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.908 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.908 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.908 INFO:teuthology.orchestra.run.smithi043.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-26T08:40:02.909 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-26T08:40:02.909 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.909 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.909 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.909 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.909 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.909 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.909 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.909 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.909 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.909 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.909 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.909 INFO:teuthology.orchestra.run.smithi043.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-26T08:40:02.909 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-26T08:40:02.910 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.910 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.910 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.910 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.910 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.910 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.910 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.910 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.910 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.910 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.910 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.910 INFO:teuthology.orchestra.run.smithi043.stdout: "MULTICLUSTER_CONFIG": { 2024-03-26T08:40:02.911 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-26T08:40:02.911 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.911 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.911 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.911 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "{}", 2024-03-26T08:40:02.911 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.911 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.911 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.911 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.911 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.911 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.911 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.911 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_HOST": { 2024-03-26T08:40:02.912 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-26T08:40:02.912 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.912 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.912 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.912 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.912 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.912 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.912 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.912 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.912 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.912 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.912 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.912 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-26T08:40:02.912 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-26T08:40:02.913 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.913 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.913 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.913 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.913 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.913 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.913 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.913 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.913 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.913 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.913 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.913 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-26T08:40:02.913 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-26T08:40:02.914 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.914 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.914 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.914 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.914 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.914 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.914 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.914 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.914 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.914 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.914 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.914 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-26T08:40:02.914 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-26T08:40:02.915 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.915 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.915 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.915 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.915 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.915 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.915 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.915 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.915 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.915 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.915 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.915 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-26T08:40:02.915 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-26T08:40:02.915 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.916 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.916 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.916 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.916 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.916 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.916 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.916 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.916 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.916 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.916 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.916 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-26T08:40:02.916 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-26T08:40:02.916 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.917 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.917 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.917 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.917 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.917 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.917 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.917 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.917 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.917 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.917 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.917 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-26T08:40:02.917 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-26T08:40:02.917 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.917 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.918 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.918 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.918 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.918 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.918 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.918 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.918 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.918 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.918 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.918 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-26T08:40:02.918 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-26T08:40:02.919 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.919 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.919 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.919 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.919 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.919 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.919 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.919 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.919 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.919 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.919 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.919 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-26T08:40:02.919 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-26T08:40:02.920 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.920 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.920 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.920 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.920 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.920 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.920 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.920 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.920 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.920 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.920 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.920 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_ENABLED": { 2024-03-26T08:40:02.921 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-26T08:40:02.921 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.921 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.921 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.921 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.921 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.921 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.921 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.921 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.921 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.921 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.921 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.922 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-26T08:40:02.922 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-26T08:40:02.922 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.922 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.922 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.922 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-26T08:40:02.922 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.922 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.922 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.922 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.922 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.922 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.922 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.923 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-26T08:40:02.923 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-26T08:40:02.923 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:02.923 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.923 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.923 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-03-26T08:40:02.923 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.923 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.923 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.923 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.923 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.923 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.923 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.924 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-26T08:40:02.924 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-26T08:40:02.924 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:02.924 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.924 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.924 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8", 2024-03-26T08:40:02.924 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.924 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.925 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.925 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.925 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.925 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.925 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.926 INFO:teuthology.orchestra.run.smithi043.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-26T08:40:02.926 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-26T08:40:02.926 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:02.926 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.926 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.926 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "45", 2024-03-26T08:40:02.926 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.926 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.926 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.926 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.926 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.926 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.926 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.926 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ACCESS_KEY": { 2024-03-26T08:40:02.927 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-26T08:40:02.927 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.927 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.927 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.927 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.927 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.927 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.927 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.927 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.927 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.927 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.927 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.927 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-26T08:40:02.927 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-26T08:40:02.927 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.928 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.928 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.928 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-03-26T08:40:02.928 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.928 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.928 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.928 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.928 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.928 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.928 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.928 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SECRET_KEY": { 2024-03-26T08:40:02.928 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-26T08:40:02.928 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.929 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.929 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.929 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.929 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.929 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.929 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.929 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.929 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.929 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.929 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.929 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SSL_VERIFY": { 2024-03-26T08:40:02.929 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-26T08:40:02.929 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.929 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.929 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.930 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.930 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.930 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.930 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.930 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.930 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.930 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.930 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.930 INFO:teuthology.orchestra.run.smithi043.stdout: "UNSAFE_TLS_v1_2": { 2024-03-26T08:40:02.930 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-26T08:40:02.930 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.930 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.930 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.930 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.931 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.931 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.931 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.931 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.931 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.931 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.931 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.931 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-26T08:40:02.931 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-26T08:40:02.931 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:02.931 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.931 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.931 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-03-26T08:40:02.931 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.932 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.932 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.932 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.932 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.932 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.932 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.932 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-26T08:40:02.932 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-26T08:40:02.932 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:02.932 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.932 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.932 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-03-26T08:40:02.932 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.932 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.933 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.933 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.933 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.933 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.933 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.933 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-26T08:40:02.933 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-26T08:40:02.933 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:02.933 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.933 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.933 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-03-26T08:40:02.933 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.933 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.934 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.934 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.934 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.934 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.934 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.934 INFO:teuthology.orchestra.run.smithi043.stdout: "cross_origin_url": { 2024-03-26T08:40:02.934 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cross_origin_url", 2024-03-26T08:40:02.934 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.934 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.934 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.934 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.934 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.934 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.934 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.935 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.935 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.935 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.935 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.935 INFO:teuthology.orchestra.run.smithi043.stdout: "crt_file": { 2024-03-26T08:40:02.935 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crt_file", 2024-03-26T08:40:02.935 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.935 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.935 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.935 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.935 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.935 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.935 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.935 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.936 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.936 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.936 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.936 INFO:teuthology.orchestra.run.smithi043.stdout: "debug": { 2024-03-26T08:40:02.936 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "debug", 2024-03-26T08:40:02.936 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.936 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.936 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.936 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.936 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.936 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.936 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.937 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable/disable debug options", 2024-03-26T08:40:02.937 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.937 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.937 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.937 INFO:teuthology.orchestra.run.smithi043.stdout: "jwt_token_ttl": { 2024-03-26T08:40:02.937 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "jwt_token_ttl", 2024-03-26T08:40:02.937 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:02.937 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.937 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.937 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "28800", 2024-03-26T08:40:02.937 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.937 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.937 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.937 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.938 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.938 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.938 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.938 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-03-26T08:40:02.938 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-03-26T08:40:02.938 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.938 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.938 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.938 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.938 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.938 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.938 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.938 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.938 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.939 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.939 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.939 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:02.939 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:02.939 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.939 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.939 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.939 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.939 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.939 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.939 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:02.939 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:02.939 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:02.939 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:02.942 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.942 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.942 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:02.942 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:02.942 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:02.942 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:02.942 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:02.942 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:02.942 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:02.942 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:02.942 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.942 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.942 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.943 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.943 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:02.943 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:02.943 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.943 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.943 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.943 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.943 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.943 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.943 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.943 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.943 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.943 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: "motd": { 2024-03-26T08:40:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "motd", 2024-03-26T08:40:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "The message of the day", 2024-03-26T08:40:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.945 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect_resolve_ip_addr": { 2024-03-26T08:40:02.945 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "redirect_resolve_ip_addr", 2024-03-26T08:40:02.945 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.945 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.945 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.945 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.945 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.945 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.945 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.945 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.945 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.945 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.946 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.946 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-03-26T08:40:02.946 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-03-26T08:40:02.946 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.946 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.946 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.946 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-03-26T08:40:02.946 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.946 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.946 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.946 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.946 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.946 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.946 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.947 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-03-26T08:40:02.947 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-03-26T08:40:02.947 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:02.947 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.947 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.947 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8080", 2024-03-26T08:40:02.947 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.947 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.947 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.947 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.947 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.947 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.947 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.948 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-03-26T08:40:02.948 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-03-26T08:40:02.948 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.948 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.948 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.948 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.948 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.948 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.948 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.948 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.948 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.948 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.948 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.949 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl_server_port": { 2024-03-26T08:40:02.949 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl_server_port", 2024-03-26T08:40:02.949 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:02.949 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.949 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.949 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8443", 2024-03-26T08:40:02.949 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.949 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.949 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.949 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.949 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.949 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.949 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.950 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-03-26T08:40:02.950 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-03-26T08:40:02.950 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.950 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.950 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.950 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "redirect", 2024-03-26T08:40:02.950 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.950 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.950 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:02.950 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:02.950 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect" 2024-03-26T08:40:02.950 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:02.951 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.951 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.951 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.951 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.951 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-03-26T08:40:02.951 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-03-26T08:40:02.951 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:02.951 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.951 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.951 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-03-26T08:40:02.951 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-03-26T08:40:02.951 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-03-26T08:40:02.951 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.952 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.952 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.952 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.952 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.952 INFO:teuthology.orchestra.run.smithi043.stdout: "url_prefix": { 2024-03-26T08:40:02.952 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url_prefix", 2024-03-26T08:40:02.952 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.952 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.952 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.952 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.952 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.952 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.952 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.952 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.953 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.953 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.953 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.953 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:02.953 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:02.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:02.953 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "devicehealth", 2024-03-26T08:40:02.953 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:02.953 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:02.953 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:02.953 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_monitoring": { 2024-03-26T08:40:02.953 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_monitoring", 2024-03-26T08:40:02.953 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.954 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.954 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.954 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.954 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.954 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.954 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.954 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "monitor device health metrics", 2024-03-26T08:40:02.954 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.954 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.954 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.954 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:02.954 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:02.954 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.955 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.955 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.955 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.955 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.955 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.955 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:02.955 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:02.955 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:02.955 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:02.955 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:02.955 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:02.955 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:02.955 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:02.955 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.955 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.956 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.956 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.956 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:02.956 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:02.956 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.956 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.956 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.956 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.956 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.956 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.956 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.956 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.956 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.956 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.957 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.957 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:02.957 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:02.957 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.957 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.957 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.957 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:02.957 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.957 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.957 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:02.957 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:02.957 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:02.957 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:02.957 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:02.958 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:02.958 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:02.958 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:02.958 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.958 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.958 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.958 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.958 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:02.958 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:02.958 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.958 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.958 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.958 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.958 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.959 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.959 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.959 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.959 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.959 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.959 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.959 INFO:teuthology.orchestra.run.smithi043.stdout: "mark_out_threshold": { 2024-03-26T08:40:02.959 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mark_out_threshold", 2024-03-26T08:40:02.959 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:02.959 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.959 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.959 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2419200", 2024-03-26T08:40:02.959 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.959 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.960 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.960 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-26T08:40:02.960 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.960 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.960 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.960 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": { 2024-03-26T08:40:02.960 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_name", 2024-03-26T08:40:02.960 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.960 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.960 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.960 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "device_health_metrics", 2024-03-26T08:40:02.960 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.960 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.960 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.960 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-26T08:40:02.960 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.961 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.961 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.961 INFO:teuthology.orchestra.run.smithi043.stdout: "retention_period": { 2024-03-26T08:40:02.961 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retention_period", 2024-03-26T08:40:02.961 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:02.961 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.961 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.961 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15552000", 2024-03-26T08:40:02.961 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.961 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.961 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.961 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain device health metrics", 2024-03-26T08:40:02.961 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.961 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.961 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.962 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_frequency": { 2024-03-26T08:40:02.962 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_frequency", 2024-03-26T08:40:02.962 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:02.962 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.962 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.962 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-03-26T08:40:02.962 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.962 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.962 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.962 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-26T08:40:02.962 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.962 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.963 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.963 INFO:teuthology.orchestra.run.smithi043.stdout: "self_heal": { 2024-03-26T08:40:02.963 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "self_heal", 2024-03-26T08:40:02.963 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.963 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.963 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.963 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:02.963 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.963 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.963 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.963 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-26T08:40:02.963 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.964 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.964 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.964 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-03-26T08:40:02.964 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-03-26T08:40:02.964 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:02.964 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.964 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.964 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-03-26T08:40:02.964 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.964 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.964 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.964 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and check device health", 2024-03-26T08:40:02.964 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.964 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.965 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.965 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_threshold": { 2024-03-26T08:40:02.965 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_threshold", 2024-03-26T08:40:02.965 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:02.965 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.965 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.965 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7257600", 2024-03-26T08:40:02.965 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.965 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.965 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.965 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-26T08:40:02.965 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.965 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.965 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.965 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:02.966 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:02.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:02.966 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "diskprediction_local", 2024-03-26T08:40:02.966 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:02.966 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:02.966 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:02.966 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:02.966 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:02.966 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.966 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.966 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.966 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.967 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.967 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.967 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:02.967 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:02.967 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:02.967 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:02.967 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:02.967 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:02.967 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:02.967 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:02.967 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.967 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.967 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.967 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.967 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:02.968 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:02.968 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.968 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.968 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.968 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.968 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.968 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.968 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.968 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.968 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.968 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.968 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.968 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:02.968 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:02.968 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.969 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.969 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.969 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:02.969 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.969 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.969 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:02.969 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:02.969 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:02.969 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:02.969 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:02.969 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:02.969 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:02.969 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:02.969 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.969 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.969 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.969 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.970 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:02.970 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:02.970 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.970 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.970 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.970 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.970 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.970 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.970 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.970 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.970 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.970 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.970 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.971 INFO:teuthology.orchestra.run.smithi043.stdout: "predict_interval": { 2024-03-26T08:40:02.971 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predict_interval", 2024-03-26T08:40:02.971 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.971 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.971 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.971 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-03-26T08:40:02.971 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.971 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.971 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.971 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.971 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.971 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.971 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.972 INFO:teuthology.orchestra.run.smithi043.stdout: "predictor_model": { 2024-03-26T08:40:02.972 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predictor_model", 2024-03-26T08:40:02.972 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.972 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.972 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.972 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "prophetstor", 2024-03-26T08:40:02.972 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.972 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.972 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.972 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.972 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.972 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.972 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.973 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-03-26T08:40:02.973 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-03-26T08:40:02.973 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.973 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.973 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.973 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-03-26T08:40:02.973 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.973 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.973 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.973 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.973 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.973 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.973 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.973 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:02.973 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:02.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:02.974 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "influx", 2024-03-26T08:40:02.974 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": false, 2024-03-26T08:40:02.974 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "influxdb python module not found", 2024-03-26T08:40:02.974 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:02.974 INFO:teuthology.orchestra.run.smithi043.stdout: "batch_size": { 2024-03-26T08:40:02.974 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "batch_size", 2024-03-26T08:40:02.974 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:02.974 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.974 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.974 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5000", 2024-03-26T08:40:02.974 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.974 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.974 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.975 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-26T08:40:02.975 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.975 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.975 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.975 INFO:teuthology.orchestra.run.smithi043.stdout: "database": { 2024-03-26T08:40:02.975 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "database", 2024-03-26T08:40:02.975 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.975 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.975 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.975 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "ceph", 2024-03-26T08:40:02.975 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.975 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.975 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.976 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-26T08:40:02.976 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.976 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.976 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.976 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": { 2024-03-26T08:40:02.976 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hostname", 2024-03-26T08:40:02.976 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.976 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.976 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.976 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.976 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.976 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.976 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.976 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server hostname", 2024-03-26T08:40:02.977 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.977 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.977 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.977 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-03-26T08:40:02.977 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-03-26T08:40:02.977 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:02.977 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.977 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.977 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "30", 2024-03-26T08:40:02.977 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "5", 2024-03-26T08:40:02.977 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.977 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.977 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-26T08:40:02.978 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.978 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.978 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.978 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:02.978 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:02.978 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.978 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.978 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.978 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.978 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.978 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.978 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:02.978 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:02.979 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:02.979 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:02.979 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:02.979 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:02.979 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:02.979 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:02.979 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.979 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.979 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.979 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.979 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:02.979 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:02.979 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.980 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.980 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.980 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.980 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.980 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.980 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.980 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.980 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.980 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.980 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.980 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:02.980 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:02.980 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.980 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.981 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.981 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:02.981 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.981 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.981 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:02.981 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:02.981 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:02.981 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:02.981 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:02.981 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:02.981 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:02.981 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:02.981 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.981 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.982 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.982 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.982 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:02.982 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:02.982 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.982 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.982 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.982 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.982 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.982 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.982 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.982 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.982 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.982 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.983 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.983 INFO:teuthology.orchestra.run.smithi043.stdout: "password": { 2024-03-26T08:40:02.983 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "password", 2024-03-26T08:40:02.983 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.983 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.983 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.983 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.983 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.983 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.983 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.983 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "password of InfluxDB server user", 2024-03-26T08:40:02.983 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.983 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.983 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.984 INFO:teuthology.orchestra.run.smithi043.stdout: "port": { 2024-03-26T08:40:02.984 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "port", 2024-03-26T08:40:02.984 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:02.984 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.984 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.984 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8086", 2024-03-26T08:40:02.984 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.984 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.984 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.984 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server port", 2024-03-26T08:40:02.984 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.984 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.984 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.985 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-03-26T08:40:02.985 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-03-26T08:40:02.985 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.985 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.985 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.985 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "false", 2024-03-26T08:40:02.985 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.985 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.985 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.985 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T08:40:02.985 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.985 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.985 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.986 INFO:teuthology.orchestra.run.smithi043.stdout: "threads": { 2024-03-26T08:40:02.986 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threads", 2024-03-26T08:40:02.986 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:02.986 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.986 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.986 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-03-26T08:40:02.986 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-03-26T08:40:02.986 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "32", 2024-03-26T08:40:02.986 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.986 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-26T08:40:02.986 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.986 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.986 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.986 INFO:teuthology.orchestra.run.smithi043.stdout: "username": { 2024-03-26T08:40:02.987 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "username", 2024-03-26T08:40:02.987 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.987 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.987 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.987 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.987 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.987 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.987 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.987 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "username of InfluxDB server user", 2024-03-26T08:40:02.987 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.987 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.987 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.987 INFO:teuthology.orchestra.run.smithi043.stdout: "verify_ssl": { 2024-03-26T08:40:02.987 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "verify_ssl", 2024-03-26T08:40:02.988 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.988 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.988 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:02.988 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "true", 2024-03-26T08:40:02.988 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.988 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.988 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.988 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T08:40:02.988 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.988 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.988 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.988 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:02.989 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:02.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:02.989 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "insights", 2024-03-26T08:40:02.989 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:02.989 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:02.989 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:02.989 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:02.989 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:02.989 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.989 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.989 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.989 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.989 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.990 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.990 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:02.990 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:02.990 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:02.990 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:02.990 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:02.990 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:02.990 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:02.990 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:02.990 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.990 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.990 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.990 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.994 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.994 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.994 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.994 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.994 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.994 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:02.994 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:02.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:02.994 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "iostat", 2024-03-26T08:40:02.994 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:02.994 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:02.994 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:02.994 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:02.995 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:02.995 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.995 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.995 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.995 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:02.995 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.995 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.995 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:02.995 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:02.995 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:02.995 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:02.995 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:02.995 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:02.995 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:02.995 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:02.995 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.996 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.996 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.996 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.996 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:02.996 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:02.996 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.996 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.996 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.996 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.996 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.996 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.996 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.997 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.997 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.997 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.997 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.997 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:02.997 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:02.997 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:02.997 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.997 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.997 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:02.997 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.997 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.997 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:02.998 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:02.998 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:02.998 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:02.998 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:02.998 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:02.998 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:02.998 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:02.998 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.998 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.998 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.998 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.998 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:02.998 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:02.998 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "k8sevents", 2024-03-26T08:40:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_event_retention_days": { 2024-03-26T08:40:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ceph_event_retention_days", 2024-03-26T08:40:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7", 2024-03-26T08:40:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-26T08:40:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: "config_check_secs": { 2024-03-26T08:40:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_check_secs", 2024-03-26T08:40:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-03-26T08:40:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "10", 2024-03-26T08:40:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-26T08:40:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.002 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.002 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.002 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.002 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.002 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.002 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.002 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.002 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.002 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.002 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.002 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.002 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.002 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.006 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.006 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.006 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.006 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.006 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.006 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.006 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.006 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.006 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.006 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.006 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.006 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "localpool", 2024-03-26T08:40:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: "failure_domain": { 2024-03-26T08:40:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "failure_domain", 2024-03-26T08:40:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "host", 2024-03-26T08:40:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "failure domain for any created local pool", 2024-03-26T08:40:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-26T08:40:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": { 2024-03-26T08:40:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_size", 2024-03-26T08:40:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default min_size for any created local pool", 2024-03-26T08:40:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-26T08:40:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rep": { 2024-03-26T08:40:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "num_rep", 2024-03-26T08:40:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.014 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.014 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.014 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3", 2024-03-26T08:40:03.014 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.014 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.014 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.014 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default replica count for any created local pool", 2024-03-26T08:40:03.014 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.014 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.014 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.014 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": { 2024-03-26T08:40:03.014 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_num", 2024-03-26T08:40:03.015 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.015 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.015 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.015 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "128", 2024-03-26T08:40:03.015 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.015 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.015 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.015 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default pg_num for any created local pool", 2024-03-26T08:40:03.015 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.015 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.015 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.015 INFO:teuthology.orchestra.run.smithi043.stdout: "prefix": { 2024-03-26T08:40:03.015 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prefix", 2024-03-26T08:40:03.015 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.016 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.016 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.016 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.016 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.016 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.016 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.016 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name prefix for any created local pool", 2024-03-26T08:40:03.016 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.016 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.016 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.016 INFO:teuthology.orchestra.run.smithi043.stdout: "subtree": { 2024-03-26T08:40:03.016 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "subtree", 2024-03-26T08:40:03.016 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.017 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.017 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.017 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "rack", 2024-03-26T08:40:03.017 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.017 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.017 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.017 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-26T08:40:03.017 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-26T08:40:03.017 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.017 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.017 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.017 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.018 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mds_autoscaler", 2024-03-26T08:40:03.018 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.018 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.018 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.018 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.018 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.018 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.018 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.018 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.018 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.018 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.018 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.018 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.018 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.019 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.019 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.019 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.019 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.019 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.019 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.019 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.019 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.019 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.019 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.019 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.019 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.023 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.023 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.023 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.023 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.023 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.023 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.023 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.023 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.023 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.023 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirroring", 2024-03-26T08:40:03.023 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.024 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.024 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.024 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.024 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.024 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.024 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.024 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.024 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.024 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.024 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.024 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.024 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.026 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.026 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.026 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.026 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.026 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.026 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.026 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.026 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.026 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.026 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.026 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.026 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "nfs", 2024-03-26T08:40:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.030 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.030 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.030 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.030 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.030 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.030 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.030 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.030 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.030 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.030 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.030 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.030 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.030 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.030 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.034 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.034 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.034 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.034 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.034 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.034 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.034 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.034 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.034 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.034 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.034 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.034 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-03-26T08:40:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: "fail_fs": { 2024-03-26T08:40:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "fail_fs", 2024-03-26T08:40:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-26T08:40:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator": { 2024-03-26T08:40:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-03-26T08:40:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm", 2024-03-26T08:40:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: "rook", 2024-03-26T08:40:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: "test_orchestrator" 2024-03-26T08:40:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Orchestrator backend", 2024-03-26T08:40:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.042 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.042 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.042 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.042 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_perf_query", 2024-03-26T08:40:03.042 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.042 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.042 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.042 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.042 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.042 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.042 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_support", 2024-03-26T08:40:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_autoscaler", 2024-03-26T08:40:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-03-26T08:40:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-03-26T08:40:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-03-26T08:40:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: "threshold": { 2024-03-26T08:40:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threshold", 2024-03-26T08:40:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-03-26T08:40:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-03-26T08:40:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1.0", 2024-03-26T08:40:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "scaling threshold", 2024-03-26T08:40:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-26T08:40:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.060 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.060 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.060 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "progress", 2024-03-26T08:40:03.060 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.060 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.060 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.060 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_pg_recovery_event": { 2024-03-26T08:40:03.060 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_pg_recovery_event", 2024-03-26T08:40:03.060 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.060 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.060 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.060 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.060 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.060 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.060 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.061 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-26T08:40:03.061 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.061 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.061 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.061 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-03-26T08:40:03.061 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-03-26T08:40:03.061 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.061 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.061 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.061 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.061 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.061 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.061 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.061 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.062 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.062 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.062 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.062 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.062 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.062 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.062 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.062 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.062 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.062 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.062 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.062 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.062 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.062 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.063 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.063 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.063 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.063 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.063 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.063 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.063 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.063 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.063 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.063 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.063 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.063 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.063 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.064 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.064 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.064 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.064 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.064 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.064 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.064 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.064 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.064 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.064 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.064 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.064 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.064 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.064 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.065 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.065 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.065 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.065 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.065 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.065 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.065 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.065 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.065 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.065 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.065 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.065 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.065 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.065 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.065 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.066 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.066 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.066 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.066 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.066 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.066 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.066 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.066 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.066 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.066 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.066 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.066 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.066 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.067 INFO:teuthology.orchestra.run.smithi043.stdout: "max_completed_events": { 2024-03-26T08:40:03.067 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_completed_events", 2024-03-26T08:40:03.067 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.067 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.067 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.067 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "50", 2024-03-26T08:40:03.067 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.067 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.067 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.067 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "number of past completed events to remember", 2024-03-26T08:40:03.067 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.067 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.067 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.068 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-03-26T08:40:03.068 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-03-26T08:40:03.068 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.068 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.068 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.068 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-03-26T08:40:03.068 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.068 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.068 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.068 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long the module is going to sleep", 2024-03-26T08:40:03.068 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.068 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.068 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.068 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.069 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.069 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus", 2024-03-26T08:40:03.069 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.069 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.069 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.069 INFO:teuthology.orchestra.run.smithi043.stdout: "cache": { 2024-03-26T08:40:03.069 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cache", 2024-03-26T08:40:03.069 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.069 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.069 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.069 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.069 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.070 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.070 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.070 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.070 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.070 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.070 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.070 INFO:teuthology.orchestra.run.smithi043.stdout: "exclude_perf_counters": { 2024-03-26T08:40:03.070 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "exclude_perf_counters", 2024-03-26T08:40:03.070 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.070 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.070 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.070 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.070 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.071 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.071 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.071 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-26T08:40:03.071 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-26T08:40:03.071 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.071 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.071 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.071 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.071 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.071 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.071 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.071 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.071 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.071 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.072 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.072 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.072 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.072 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.072 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.072 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.072 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.072 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.072 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.072 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.072 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.072 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.072 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.073 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.073 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.073 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.073 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.073 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.073 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.073 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.073 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.073 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.073 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.073 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.073 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.073 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.074 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.074 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.074 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.074 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.074 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.074 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.074 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.074 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.074 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.074 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.074 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.074 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.074 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.075 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.075 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.075 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.075 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.075 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.075 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.075 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.075 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.075 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.075 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.075 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.075 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.075 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.075 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.076 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.076 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.076 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.076 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.076 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.076 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools": { 2024-03-26T08:40:03.076 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools", 2024-03-26T08:40:03.076 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.076 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.076 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.076 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.076 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.076 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.077 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.077 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.077 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.077 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.077 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.077 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-26T08:40:03.077 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-26T08:40:03.077 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.077 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.077 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.077 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "300", 2024-03-26T08:40:03.077 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.077 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.078 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.078 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.078 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.078 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.078 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.078 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_interval": { 2024-03-26T08:40:03.078 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_interval", 2024-03-26T08:40:03.078 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-03-26T08:40:03.078 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.078 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.078 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15.0", 2024-03-26T08:40:03.078 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.078 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.079 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.079 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.079 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.079 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.079 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.079 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-03-26T08:40:03.079 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-03-26T08:40:03.079 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.079 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.079 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.079 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-03-26T08:40:03.079 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.079 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.080 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.080 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-26T08:40:03.080 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.080 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.080 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.080 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-03-26T08:40:03.080 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-03-26T08:40:03.080 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.080 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.080 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.080 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "9283", 2024-03-26T08:40:03.080 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.080 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.081 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.081 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-26T08:40:03.081 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.081 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.081 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.081 INFO:teuthology.orchestra.run.smithi043.stdout: "stale_cache_strategy": { 2024-03-26T08:40:03.081 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stale_cache_strategy", 2024-03-26T08:40:03.081 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.081 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.081 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.081 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "log", 2024-03-26T08:40:03.081 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.082 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.082 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.082 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.082 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.082 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.082 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.082 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-03-26T08:40:03.082 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-03-26T08:40:03.082 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.082 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.082 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.082 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "default", 2024-03-26T08:40:03.082 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.083 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.083 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.083 INFO:teuthology.orchestra.run.smithi043.stdout: "default", 2024-03-26T08:40:03.083 INFO:teuthology.orchestra.run.smithi043.stdout: "error" 2024-03-26T08:40:03.083 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.083 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.083 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.083 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.083 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.083 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-03-26T08:40:03.083 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-03-26T08:40:03.083 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.083 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.083 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.084 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-03-26T08:40:03.084 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-03-26T08:40:03.084 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-03-26T08:40:03.084 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.084 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.084 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.084 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.084 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.084 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.084 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.084 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_support", 2024-03-26T08:40:03.084 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.085 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.085 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.085 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.085 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.085 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.085 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.085 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.085 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.085 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.085 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.085 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.085 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.085 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.085 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.086 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.086 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.086 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.086 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.086 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.086 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.086 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.086 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.086 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.086 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.086 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.086 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.086 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.087 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.087 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.087 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.087 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.087 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.087 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.087 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.087 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.087 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.087 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.087 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.087 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.087 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.088 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.088 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.088 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.088 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.088 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.088 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.088 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.088 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.088 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.088 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.088 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.088 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.088 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.088 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.088 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.089 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.089 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.089 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.089 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.089 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.089 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.089 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.089 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.089 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.089 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.089 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.089 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.089 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.090 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_snap_create": { 2024-03-26T08:40:03.090 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_snap_create", 2024-03-26T08:40:03.090 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.090 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.090 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.090 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-03-26T08:40:03.090 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.090 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.090 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.090 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.090 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.090 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.090 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.091 INFO:teuthology.orchestra.run.smithi043.stdout: "mirror_snapshot_schedule": { 2024-03-26T08:40:03.091 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirror_snapshot_schedule", 2024-03-26T08:40:03.091 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.091 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.091 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.091 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.091 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.091 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.091 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.091 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.091 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.091 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.091 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.092 INFO:teuthology.orchestra.run.smithi043.stdout: "trash_purge_schedule": { 2024-03-26T08:40:03.092 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "trash_purge_schedule", 2024-03-26T08:40:03.092 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.092 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.092 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.092 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.092 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.092 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.092 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.092 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.092 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.092 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.093 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.093 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.093 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.093 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "restful", 2024-03-26T08:40:03.093 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.093 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.093 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.093 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_auth": { 2024-03-26T08:40:03.093 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_auth", 2024-03-26T08:40:03.093 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.093 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.093 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.094 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.094 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.094 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.094 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.094 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.094 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.094 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.094 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.094 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-03-26T08:40:03.094 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-03-26T08:40:03.094 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.094 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.094 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.095 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.095 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.095 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.095 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.095 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.095 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.095 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.095 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.095 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.095 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.096 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.096 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.096 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.096 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.096 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.096 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.096 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.096 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.096 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.096 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.096 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.096 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.096 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.097 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.097 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.097 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.097 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.097 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.097 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.097 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.097 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.097 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.097 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.097 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.097 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.098 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.098 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.098 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.098 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.098 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.098 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.098 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.098 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.098 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.098 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.098 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.098 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.099 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.099 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.099 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.099 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.099 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.099 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.099 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.099 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.099 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.099 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.099 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.099 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.099 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.099 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.100 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.100 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.100 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.100 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.100 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.100 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.100 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.100 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.100 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.100 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.100 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.100 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.101 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.101 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-03-26T08:40:03.101 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-03-26T08:40:03.101 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.101 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.101 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.101 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.101 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.101 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.101 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.101 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.101 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.101 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.102 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.102 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-03-26T08:40:03.102 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-03-26T08:40:03.102 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.102 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.102 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.102 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.102 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.102 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.102 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.102 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.102 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.103 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.103 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.103 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.103 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.103 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rgw", 2024-03-26T08:40:03.103 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.103 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.103 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.103 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.103 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.103 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.103 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.103 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.104 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.104 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.104 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.104 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.104 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.104 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.104 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.104 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.104 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.104 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.104 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.104 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.104 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.104 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.105 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.105 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.105 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.105 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.105 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.105 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.105 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.105 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.105 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.105 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.105 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.105 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.106 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.106 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.106 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.106 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.106 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.106 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.106 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.106 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.106 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.106 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.106 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.106 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.107 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.107 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.107 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.107 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.107 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.107 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.107 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.107 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.107 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.107 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.107 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.107 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.108 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.108 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.108 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.108 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.108 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.108 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.108 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.108 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.108 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.108 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.108 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.108 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.108 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.108 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rook", 2024-03-26T08:40:03.109 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.109 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.109 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.109 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.109 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.109 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.109 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.109 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.109 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.109 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.109 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.109 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.109 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.109 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.110 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.110 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.110 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.110 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.110 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.110 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.110 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.110 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.110 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.110 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.110 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.110 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.111 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.111 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.111 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.111 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.111 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.111 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.111 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.111 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.111 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.111 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.111 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.111 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.112 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.112 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.112 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.112 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.112 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.112 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.112 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.112 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.112 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.112 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.112 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.112 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.112 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.112 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.112 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.113 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.113 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.113 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.113 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.113 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.113 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.113 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.113 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.113 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.113 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.113 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.113 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.114 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.114 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.114 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.114 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.114 INFO:teuthology.orchestra.run.smithi043.stdout: "storage_class": { 2024-03-26T08:40:03.114 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "storage_class", 2024-03-26T08:40:03.114 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.114 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.114 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.114 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "local", 2024-03-26T08:40:03.114 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.114 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.114 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.115 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-26T08:40:03.115 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.115 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.115 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.115 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.115 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.115 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "selftest", 2024-03-26T08:40:03.115 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.115 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.115 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.115 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.115 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.116 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.116 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.116 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.116 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.116 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.116 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.116 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.116 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.116 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.116 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.116 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.116 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.116 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.116 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.116 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.117 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.117 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.117 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.117 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.117 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.117 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.117 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.117 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.117 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.117 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.117 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.117 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.117 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.117 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.118 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.118 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.118 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.118 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.118 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.118 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.118 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.118 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.118 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.118 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.118 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.118 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.118 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.119 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.119 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.119 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.119 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.119 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.119 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.119 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.119 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.119 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.119 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.119 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.119 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.119 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.120 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.120 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.120 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.120 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.120 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.120 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.120 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.120 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.120 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.120 INFO:teuthology.orchestra.run.smithi043.stdout: "roption1": { 2024-03-26T08:40:03.120 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption1", 2024-03-26T08:40:03.120 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.120 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.120 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.121 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.121 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.121 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.121 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.121 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.121 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.121 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.121 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.121 INFO:teuthology.orchestra.run.smithi043.stdout: "roption2": { 2024-03-26T08:40:03.121 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption2", 2024-03-26T08:40:03.121 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.121 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.121 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.122 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "xyz", 2024-03-26T08:40:03.122 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.122 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.122 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.122 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.122 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.122 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.122 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.122 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption1": { 2024-03-26T08:40:03.122 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption1", 2024-03-26T08:40:03.122 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.122 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.122 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.123 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.123 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.123 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.123 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.123 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.123 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.123 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.123 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.123 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption2": { 2024-03-26T08:40:03.123 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption2", 2024-03-26T08:40:03.123 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.123 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.123 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.123 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.124 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.124 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.124 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.125 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.125 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.125 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.125 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.125 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption3": { 2024-03-26T08:40:03.126 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption3", 2024-03-26T08:40:03.126 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-03-26T08:40:03.126 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.126 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.126 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.126 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.126 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.126 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.126 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.126 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.126 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.126 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.126 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption4": { 2024-03-26T08:40:03.127 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption4", 2024-03-26T08:40:03.127 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.127 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.127 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.127 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.127 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.127 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.127 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.127 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.127 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.127 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.127 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.127 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption5": { 2024-03-26T08:40:03.128 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption5", 2024-03-26T08:40:03.128 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.128 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.128 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.128 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.128 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.128 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.128 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.128 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.128 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.128 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.128 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.128 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption6": { 2024-03-26T08:40:03.129 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption6", 2024-03-26T08:40:03.129 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.129 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.129 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.129 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.129 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.129 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.129 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.129 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.129 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.129 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.129 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.129 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption7": { 2024-03-26T08:40:03.129 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption7", 2024-03-26T08:40:03.130 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.130 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.130 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.130 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.130 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-03-26T08:40:03.130 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "42", 2024-03-26T08:40:03.130 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.130 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.130 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.130 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.130 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.130 INFO:teuthology.orchestra.run.smithi043.stdout: "testkey": { 2024-03-26T08:40:03.131 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testkey", 2024-03-26T08:40:03.131 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.131 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.131 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.131 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.131 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.131 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.131 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.131 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.131 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.131 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.131 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.131 INFO:teuthology.orchestra.run.smithi043.stdout: "testlkey": { 2024-03-26T08:40:03.131 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testlkey", 2024-03-26T08:40:03.132 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.132 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.132 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.132 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.132 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.132 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.132 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.132 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.132 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.132 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.132 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.132 INFO:teuthology.orchestra.run.smithi043.stdout: "testnewline": { 2024-03-26T08:40:03.132 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testnewline", 2024-03-26T08:40:03.133 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.133 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.133 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.133 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.133 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.133 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.133 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.133 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.133 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.133 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.133 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.133 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.133 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.134 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snap_schedule", 2024-03-26T08:40:03.134 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.134 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.134 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.134 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_m_granularity": { 2024-03-26T08:40:03.134 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_m_granularity", 2024-03-26T08:40:03.134 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.134 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.134 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.134 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.134 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.134 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.134 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.134 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow minute scheduled snapshots", 2024-03-26T08:40:03.135 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.135 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.135 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.135 INFO:teuthology.orchestra.run.smithi043.stdout: "dump_on_update": { 2024-03-26T08:40:03.135 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dump_on_update", 2024-03-26T08:40:03.135 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.135 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.135 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.135 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.135 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.135 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.135 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.135 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "dump database to debug log on update", 2024-03-26T08:40:03.136 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.136 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.136 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.136 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.136 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.136 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.136 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.136 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.136 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.136 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.136 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.136 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.136 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.136 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.137 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.137 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.137 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.137 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.137 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.137 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.137 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.137 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.137 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.137 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.137 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.137 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.137 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.137 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.137 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.138 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.138 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.138 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.138 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.138 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.138 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.138 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.138 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.138 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.138 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.138 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.138 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.139 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.139 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.139 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.139 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.139 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.139 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.139 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.139 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.139 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.139 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.139 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.139 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.139 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.140 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.140 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.140 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.140 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.140 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.140 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.140 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.140 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.140 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.140 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.140 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.140 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.140 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.141 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.141 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.141 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.141 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.141 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stats", 2024-03-26T08:40:03.141 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.141 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.141 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.141 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.141 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.141 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.141 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.141 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.141 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.142 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.142 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.142 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.142 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.142 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.142 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.142 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.142 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.142 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.142 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.142 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.142 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.142 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.142 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.143 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.143 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.143 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.143 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.143 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.143 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.143 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.143 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.143 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.143 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.143 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.143 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.143 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.144 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.144 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.144 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.144 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.144 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.144 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.144 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.144 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.144 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.144 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.144 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.144 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.144 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.144 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.145 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.145 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.145 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.145 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.145 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.145 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.145 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.145 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.145 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.145 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.145 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.145 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.145 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.145 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.146 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.146 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.146 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.146 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.146 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.146 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.146 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.146 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "status", 2024-03-26T08:40:03.146 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.146 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.146 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.146 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.147 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.147 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.147 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.147 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.147 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.147 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.147 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.147 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.147 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.147 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.147 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.147 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.147 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.147 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.147 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.148 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.148 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.148 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.148 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.148 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.148 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.148 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.148 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.148 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.148 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.148 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.148 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.148 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.149 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.149 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.149 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.149 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.149 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.149 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.149 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.149 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.149 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.149 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.149 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.149 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.149 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.149 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.149 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.150 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.150 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.150 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.150 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.150 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.150 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.150 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.150 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.150 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.150 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.150 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.150 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.150 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.151 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.151 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.151 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.151 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.151 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.151 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.151 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.151 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.151 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.151 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.151 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.151 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telegraf", 2024-03-26T08:40:03.151 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.152 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.152 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.152 INFO:teuthology.orchestra.run.smithi043.stdout: "address": { 2024-03-26T08:40:03.152 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "address", 2024-03-26T08:40:03.152 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.152 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.152 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.152 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-26T08:40:03.152 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.152 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.152 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.152 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.152 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.153 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.153 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.153 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-03-26T08:40:03.153 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-03-26T08:40:03.153 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.153 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.153 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.153 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15", 2024-03-26T08:40:03.153 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.153 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.153 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.153 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.154 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.154 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.154 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.154 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.154 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.154 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.154 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.154 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.154 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.154 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.154 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.154 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.154 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.155 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.155 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.155 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.155 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.155 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.155 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.155 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.155 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.155 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.155 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.155 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.155 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.155 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.155 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.156 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.156 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.156 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.156 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.156 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.156 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.156 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.156 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.156 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.156 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.156 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.156 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.156 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.156 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.157 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.157 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.157 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.157 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.157 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.157 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.157 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.157 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.157 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.157 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.157 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.157 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.157 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.157 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.157 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.158 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.158 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.158 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.158 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.158 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.158 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.158 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.158 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.158 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.158 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.158 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.158 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.158 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.159 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.159 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.159 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telemetry", 2024-03-26T08:40:03.159 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.159 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.159 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.159 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_basic": { 2024-03-26T08:40:03.159 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_basic", 2024-03-26T08:40:03.159 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.159 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.159 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.159 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.160 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.160 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.160 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.160 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-26T08:40:03.160 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.160 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.160 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.160 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_crash": { 2024-03-26T08:40:03.160 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_crash", 2024-03-26T08:40:03.160 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.160 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.160 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.160 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.161 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.161 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.161 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.161 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-26T08:40:03.161 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.161 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.161 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.161 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_device": { 2024-03-26T08:40:03.161 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_device", 2024-03-26T08:40:03.161 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.161 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.161 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.162 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.162 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.162 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.162 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.162 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-26T08:40:03.162 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.162 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.162 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.162 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_ident": { 2024-03-26T08:40:03.162 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_ident", 2024-03-26T08:40:03.162 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.162 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.162 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.163 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.163 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.163 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.163 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.163 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-26T08:40:03.163 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.163 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.163 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.163 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_perf": { 2024-03-26T08:40:03.163 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_perf", 2024-03-26T08:40:03.163 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.163 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.163 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.164 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.164 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.164 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.164 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.164 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-26T08:40:03.164 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.164 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.164 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.164 INFO:teuthology.orchestra.run.smithi043.stdout: "contact": { 2024-03-26T08:40:03.164 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "contact", 2024-03-26T08:40:03.164 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.164 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.164 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.164 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.165 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.165 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.165 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.165 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.165 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.165 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.165 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.165 INFO:teuthology.orchestra.run.smithi043.stdout: "description": { 2024-03-26T08:40:03.165 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "description", 2024-03-26T08:40:03.165 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.165 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.165 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.165 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.166 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.166 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.166 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.166 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.166 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.166 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.166 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.166 INFO:teuthology.orchestra.run.smithi043.stdout: "device_url": { 2024-03-26T08:40:03.166 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_url", 2024-03-26T08:40:03.166 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.166 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.166 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.167 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-26T08:40:03.167 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.167 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.167 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.167 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.167 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.167 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.167 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.167 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-03-26T08:40:03.167 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-03-26T08:40:03.167 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.167 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.167 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.168 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.168 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.168 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.168 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.168 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.168 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.168 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.168 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.168 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-03-26T08:40:03.168 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-03-26T08:40:03.168 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.168 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.168 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.169 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "24", 2024-03-26T08:40:03.169 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "8", 2024-03-26T08:40:03.169 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.169 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.169 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.169 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.169 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.169 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.169 INFO:teuthology.orchestra.run.smithi043.stdout: "last_opt_revision": { 2024-03-26T08:40:03.169 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "last_opt_revision", 2024-03-26T08:40:03.169 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.169 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.169 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.169 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1", 2024-03-26T08:40:03.170 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.170 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.170 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.170 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.170 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.170 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.170 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.170 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard": { 2024-03-26T08:40:03.170 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard", 2024-03-26T08:40:03.170 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.170 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.170 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.170 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.171 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.171 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.171 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.171 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.171 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.171 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.171 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.171 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard_description": { 2024-03-26T08:40:03.171 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard_description", 2024-03-26T08:40:03.171 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.171 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.171 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.171 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.172 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.172 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.172 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.172 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.172 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.172 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.172 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.172 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.172 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.172 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.172 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.172 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.172 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.173 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.173 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.173 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.173 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.173 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.173 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.173 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.173 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.173 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.173 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.173 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.173 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.173 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.173 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.175 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.175 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.175 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.175 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.175 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.175 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.175 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.175 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.175 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.175 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.175 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.175 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.175 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.175 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.175 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.176 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: "organization": { 2024-03-26T08:40:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "organization", 2024-03-26T08:40:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.177 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.178 INFO:teuthology.orchestra.run.smithi043.stdout: "proxy": { 2024-03-26T08:40:03.178 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "proxy", 2024-03-26T08:40:03.178 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.178 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.178 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.178 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.178 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.178 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.178 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.178 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.178 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.178 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.178 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.179 INFO:teuthology.orchestra.run.smithi043.stdout: "url": { 2024-03-26T08:40:03.179 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url", 2024-03-26T08:40:03.179 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.179 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.179 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.179 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-26T08:40:03.179 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.179 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.179 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.179 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.179 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.179 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.179 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.179 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.179 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.180 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "test_orchestrator", 2024-03-26T08:40:03.180 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.180 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.180 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.180 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.180 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.180 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.180 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.180 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.180 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.180 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.180 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.180 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.180 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.181 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.181 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.181 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.181 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.181 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.181 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.181 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.181 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.181 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.181 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.181 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.181 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.182 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.182 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.182 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.182 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.182 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.182 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.182 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.182 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.182 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.182 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.182 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.182 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.182 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.182 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.183 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.184 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.184 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.184 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.184 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.184 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.184 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.184 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.184 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.184 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.184 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.184 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.184 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.184 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.184 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "volumes", 2024-03-26T08:40:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.185 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.186 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.187 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.188 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.189 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_clones": { 2024-03-26T08:40:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_clones", 2024-03-26T08:40:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4", 2024-03-26T08:40:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-26T08:40:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.191 INFO:teuthology.orchestra.run.smithi043.stdout: "periodic_async_work": { 2024-03-26T08:40:03.191 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "periodic_async_work", 2024-03-26T08:40:03.191 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.191 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.191 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.191 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.191 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.191 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.191 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.191 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Periodically check for async work", 2024-03-26T08:40:03.191 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.191 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.191 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.191 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_delay": { 2024-03-26T08:40:03.192 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_delay", 2024-03-26T08:40:03.192 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.192 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.192 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.192 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-03-26T08:40:03.192 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.192 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.192 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.192 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-26T08:40:03.192 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.192 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.192 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.192 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_no_wait": { 2024-03-26T08:40:03.193 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_no_wait", 2024-03-26T08:40:03.193 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.193 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.193 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.193 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.193 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.193 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.193 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.193 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-26T08:40:03.193 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.193 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.193 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.193 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.193 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.194 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix", 2024-03-26T08:40:03.194 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.194 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.194 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.194 INFO:teuthology.orchestra.run.smithi043.stdout: "discovery_interval": { 2024-03-26T08:40:03.194 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "discovery_interval", 2024-03-26T08:40:03.194 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-03-26T08:40:03.194 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.194 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.194 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "100", 2024-03-26T08:40:03.194 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.194 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: "identifier": { 2024-03-26T08:40:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "identifier", 2024-03-26T08:40:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.195 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.196 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.196 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.196 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.196 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.196 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-03-26T08:40:03.196 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-03-26T08:40:03.196 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.196 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.196 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.196 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-03-26T08:40:03.196 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.196 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.196 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.197 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.198 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.199 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.200 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.201 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_host": { 2024-03-26T08:40:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_host", 2024-03-26T08:40:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.202 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_port": { 2024-03-26T08:40:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_port", 2024-03-26T08:40:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10051", 2024-03-26T08:40:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_sender": { 2024-03-26T08:40:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_sender", 2024-03-26T08:40:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-26T08:40:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.204 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-03-26T08:40:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 34112, 2024-03-26T08:40:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "y", 2024-03-26T08:40:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_features": 4540701547738038271, 2024-03-26T08:40:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: "available_modules": [ 2024-03-26T08:40:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "alerts", 2024-03-26T08:40:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.205 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-03-26T08:40:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-03-26T08:40:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-03-26T08:40:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How frequently to reexamine health status", 2024-03-26T08:40:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.206 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.207 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.208 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.209 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.210 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_destination": { 2024-03-26T08:40:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_destination", 2024-03-26T08:40:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.211 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email address to send alerts to", 2024-03-26T08:40:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_from_name": { 2024-03-26T08:40:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_from_name", 2024-03-26T08:40:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.212 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "Ceph", 2024-03-26T08:40:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email From: name", 2024-03-26T08:40:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_host": { 2024-03-26T08:40:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_host", 2024-03-26T08:40:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP server", 2024-03-26T08:40:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_password": { 2024-03-26T08:40:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_password", 2024-03-26T08:40:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.214 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Password to authenticate with", 2024-03-26T08:40:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_port": { 2024-03-26T08:40:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_port", 2024-03-26T08:40:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "465", 2024-03-26T08:40:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.215 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP port", 2024-03-26T08:40:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_sender": { 2024-03-26T08:40:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_sender", 2024-03-26T08:40:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.216 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP envelope sender", 2024-03-26T08:40:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_ssl": { 2024-03-26T08:40:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_ssl", 2024-03-26T08:40:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.217 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-26T08:40:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_user": { 2024-03-26T08:40:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_user", 2024-03-26T08:40:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.218 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "User to authenticate as", 2024-03-26T08:40:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "balancer", 2024-03-26T08:40:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: "active": { 2024-03-26T08:40:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "active", 2024-03-26T08:40:03.219 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically balance PGs across cluster", 2024-03-26T08:40:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_time": { 2024-03-26T08:40:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_time", 2024-03-26T08:40:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.220 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0000", 2024-03-26T08:40:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "beginning time of day to automatically balance", 2024-03-26T08:40:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T08:40:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_weekday": { 2024-03-26T08:40:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_weekday", 2024-03-26T08:40:03.221 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-03-26T08:40:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-03-26T08:40:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-03-26T08:40:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-03-26T08:40:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-26T08:40:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T08:40:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_max_iterations": { 2024-03-26T08:40:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_max_iterations", 2024-03-26T08:40:03.222 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-03-26T08:40:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "25", 2024-03-26T08:40:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-03-26T08:40:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "250", 2024-03-26T08:40:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-26T08:40:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_metrics": { 2024-03-26T08:40:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_metrics", 2024-03-26T08:40:03.223 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "pgs,objects,bytes", 2024-03-26T08:40:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-26T08:40:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-26T08:40:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_step": { 2024-03-26T08:40:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_step", 2024-03-26T08:40:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-03-26T08:40:03.224 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.5", 2024-03-26T08:40:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0.001", 2024-03-26T08:40:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "0.999", 2024-03-26T08:40:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "aggressiveness of optimization", 2024-03-26T08:40:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-26T08:40:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: "end_time": { 2024-03-26T08:40:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_time", 2024-03-26T08:40:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.225 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2359", 2024-03-26T08:40:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ending time of day to automatically balance", 2024-03-26T08:40:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T08:40:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: "end_weekday": { 2024-03-26T08:40:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_weekday", 2024-03-26T08:40:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-03-26T08:40:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.226 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-03-26T08:40:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-03-26T08:40:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-03-26T08:40:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-26T08:40:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T08:40:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.228 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.230 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.230 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.230 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.230 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.230 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.230 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.230 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.230 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.230 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.230 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.230 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.230 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.231 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "min_score": { 2024-03-26T08:40:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_score", 2024-03-26T08:40:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-03-26T08:40:03.232 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-03-26T08:40:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-26T08:40:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-03-26T08:40:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-03-26T08:40:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.233 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "upmap", 2024-03-26T08:40:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: "crush-compat", 2024-03-26T08:40:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: "none", 2024-03-26T08:40:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: "read", 2024-03-26T08:40:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap", 2024-03-26T08:40:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap-read" 2024-03-26T08:40:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Balancer mode", 2024-03-26T08:40:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.234 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ids": { 2024-03-26T08:40:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_ids", 2024-03-26T08:40:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-26T08:40:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.235 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-03-26T08:40:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-03-26T08:40:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-03-26T08:40:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-26T08:40:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.236 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_deviation": { 2024-03-26T08:40:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_deviation", 2024-03-26T08:40:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-03-26T08:40:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-03-26T08:40:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-26T08:40:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-26T08:40:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.237 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_optimizations": { 2024-03-26T08:40:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_optimizations", 2024-03-26T08:40:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-03-26T08:40:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-03-26T08:40:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-26T08:40:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.238 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm", 2024-03-26T08:40:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_down_multiplier": { 2024-03-26T08:40:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_down_multiplier", 2024-03-26T08:40:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-03-26T08:40:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.239 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-03-26T08:40:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-26T08:40:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_refresh_rate": { 2024-03-26T08:40:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_refresh_rate", 2024-03-26T08:40:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "20", 2024-03-26T08:40:03.240 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-26T08:40:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_starting_port": { 2024-03-26T08:40:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_starting_port", 2024-03-26T08:40:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4721", 2024-03-26T08:40:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.241 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-26T08:40:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_ptrace": { 2024-03-26T08:40:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_ptrace", 2024-03-26T08:40:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.242 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-26T08:40:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-26T08:40:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_interval": { 2024-03-26T08:40:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_interval", 2024-03-26T08:40:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-03-26T08:40:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.243 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-26T08:40:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_memory_target_ratio": { 2024-03-26T08:40:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_memory_target_ratio", 2024-03-26T08:40:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-03-26T08:40:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.7", 2024-03-26T08:40:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.244 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-26T08:40:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm_log_destination": { 2024-03-26T08:40:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm_log_destination", 2024-03-26T08:40:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.245 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: "file", 2024-03-26T08:40:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: "file,syslog", 2024-03-26T08:40:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: "syslog" 2024-03-26T08:40:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-26T08:40:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: "cgroups_split": { 2024-03-26T08:40:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cgroups_split", 2024-03-26T08:40:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.246 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-26T08:40:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "config_checks_enabled": { 2024-03-26T08:40:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_checks_enabled", 2024-03-26T08:40:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.247 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-26T08:40:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "config_dashboard": { 2024-03-26T08:40:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_dashboard", 2024-03-26T08:40:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.248 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-26T08:40:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_alertmanager": { 2024-03-26T08:40:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_alertmanager", 2024-03-26T08:40:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-26T08:40:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.249 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-03-26T08:40:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_base": { 2024-03-26T08:40:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_base", 2024-03-26T08:40:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-26T08:40:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.250 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Container image name, without the tag", 2024-03-26T08:40:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_elasticsearch": { 2024-03-26T08:40:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_elasticsearch", 2024-03-26T08:40:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-26T08:40:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.251 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "elasticsearch container image", 2024-03-26T08:40:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_grafana": { 2024-03-26T08:40:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_grafana", 2024-03-26T08:40:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-26T08:40:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-03-26T08:40:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.252 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_haproxy": { 2024-03-26T08:40:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_haproxy", 2024-03-26T08:40:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-26T08:40:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "HAproxy container image", 2024-03-26T08:40:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_agent": { 2024-03-26T08:40:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_agent", 2024-03-26T08:40:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-26T08:40:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger agent container image", 2024-03-26T08:40:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_collector": { 2024-03-26T08:40:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_collector", 2024-03-26T08:40:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-26T08:40:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger collector container image", 2024-03-26T08:40:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_query": { 2024-03-26T08:40:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_query", 2024-03-26T08:40:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-26T08:40:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger query container image", 2024-03-26T08:40:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_keepalived": { 2024-03-26T08:40:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_keepalived", 2024-03-26T08:40:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-26T08:40:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Keepalived container image", 2024-03-26T08:40:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_loki": { 2024-03-26T08:40:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_loki", 2024-03-26T08:40:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-26T08:40:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Loki container image", 2024-03-26T08:40:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_node_exporter": { 2024-03-26T08:40:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_node_exporter", 2024-03-26T08:40:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-26T08:40:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-03-26T08:40:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_nvmeof": { 2024-03-26T08:40:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_nvmeof", 2024-03-26T08:40:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-26T08:40:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Nvme-of container image", 2024-03-26T08:40:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_prometheus": { 2024-03-26T08:40:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_prometheus", 2024-03-26T08:40:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-26T08:40:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-03-26T08:40:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_promtail": { 2024-03-26T08:40:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_promtail", 2024-03-26T08:40:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-26T08:40:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Promtail container image", 2024-03-26T08:40:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_snmp_gateway": { 2024-03-26T08:40:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_snmp_gateway", 2024-03-26T08:40:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-26T08:40:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SNMP Gateway container image", 2024-03-26T08:40:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "container_init": { 2024-03-26T08:40:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_init", 2024-03-26T08:40:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Run podman/docker with `--init`", 2024-03-26T08:40:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "daemon_cache_timeout": { 2024-03-26T08:40:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "daemon_cache_timeout", 2024-03-26T08:40:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-03-26T08:40:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-26T08:40:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "default_cephadm_command_timeout": { 2024-03-26T08:40:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_cephadm_command_timeout", 2024-03-26T08:40:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "900", 2024-03-26T08:40:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-26T08:40:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "default_registry": { 2024-03-26T08:40:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_registry", 2024-03-26T08:40:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io", 2024-03-26T08:40:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-26T08:40:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "device_cache_timeout": { 2024-03-26T08:40:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_cache_timeout", 2024-03-26T08:40:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1800", 2024-03-26T08:40:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache device inventory", 2024-03-26T08:40:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "device_enhanced_scan": { 2024-03-26T08:40:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_enhanced_scan", 2024-03-26T08:40:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-26T08:40:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "facts_cache_timeout": { 2024-03-26T08:40:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "facts_cache_timeout", 2024-03-26T08:40:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-03-26T08:40:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache host facts data", 2024-03-26T08:40:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "host_check_interval": { 2024-03-26T08:40:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "host_check_interval", 2024-03-26T08:40:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-03-26T08:40:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to perform a host check", 2024-03-26T08:40:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "hw_monitoring": { 2024-03-26T08:40:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hw_monitoring", 2024-03-26T08:40:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-26T08:40:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "inventory_list_all": { 2024-03-26T08:40:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "inventory_list_all", 2024-03-26T08:40:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-26T08:40:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: "log_refresh_metadata": { 2024-03-26T08:40:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_refresh_metadata", 2024-03-26T08:40:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-26T08:40:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-26T08:40:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-26T08:40:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-26T08:40:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-26T08:40:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-26T08:40:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-26T08:40:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "*", 2024-03-26T08:40:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-26T08:40:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: "max_count_per_host": { 2024-03-26T08:40:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_count_per_host", 2024-03-26T08:40:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-03-26T08:40:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of daemons per service per host", 2024-03-26T08:40:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: "max_osd_draining_count": { 2024-03-26T08:40:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_osd_draining_count", 2024-03-26T08:40:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-03-26T08:40:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-26T08:40:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.282 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.283 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.283 INFO:teuthology.orchestra.run.smithi043.stdout: "migration_current": { 2024-03-26T08:40:03.283 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "migration_current", 2024-03-26T08:40:03.283 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.283 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.283 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.283 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.283 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.283 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.283 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.283 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "internal - do not modify", 2024-03-26T08:40:03.283 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.283 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.284 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.284 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-03-26T08:40:03.284 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-03-26T08:40:03.284 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.284 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.284 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.284 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "root", 2024-03-26T08:40:03.284 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.284 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.284 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.284 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm-package", 2024-03-26T08:40:03.284 INFO:teuthology.orchestra.run.smithi043.stdout: "root" 2024-03-26T08:40:03.284 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.284 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "mode for remote execution of cephadm", 2024-03-26T08:40:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: "oob_default_addr": { 2024-03-26T08:40:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "oob_default_addr", 2024-03-26T08:40:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "169.254.1.1", 2024-03-26T08:40:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-26T08:40:03.285 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.286 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.286 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.286 INFO:teuthology.orchestra.run.smithi043.stdout: "prometheus_alerts_path": { 2024-03-26T08:40:03.286 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus_alerts_path", 2024-03-26T08:40:03.286 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.286 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.286 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.286 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-26T08:40:03.286 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.286 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.286 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.286 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-26T08:40:03.286 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.287 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.287 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.287 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_insecure": { 2024-03-26T08:40:03.287 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_insecure", 2024-03-26T08:40:03.287 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.287 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.287 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.287 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.287 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.287 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.287 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.287 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-26T08:40:03.287 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.287 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.288 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.288 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_password": { 2024-03-26T08:40:03.288 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_password", 2024-03-26T08:40:03.288 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.288 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.288 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.288 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.288 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.288 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.288 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.288 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-26T08:40:03.288 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.288 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_url": { 2024-03-26T08:40:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_url", 2024-03-26T08:40:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-26T08:40:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.289 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.290 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_username": { 2024-03-26T08:40:03.290 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_username", 2024-03-26T08:40:03.290 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.290 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.290 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.290 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.290 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.290 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.290 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.290 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-26T08:40:03.290 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.290 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.290 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.291 INFO:teuthology.orchestra.run.smithi043.stdout: "secure_monitoring_stack": { 2024-03-26T08:40:03.291 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "secure_monitoring_stack", 2024-03-26T08:40:03.291 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.291 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.291 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.291 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.291 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.291 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.291 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.291 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-26T08:40:03.291 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.291 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.291 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: "service_discovery_port": { 2024-03-26T08:40:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "service_discovery_port", 2024-03-26T08:40:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8765", 2024-03-26T08:40:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "cephadm service discovery port", 2024-03-26T08:40:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: "ssh_config_file": { 2024-03-26T08:40:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssh_config_file", 2024-03-26T08:40:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-26T08:40:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.293 INFO:teuthology.orchestra.run.smithi043.stdout: "use_agent": { 2024-03-26T08:40:03.294 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_agent", 2024-03-26T08:40:03.294 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.294 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.294 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.294 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.294 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.294 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.294 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.294 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-26T08:40:03.294 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.294 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.294 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.294 INFO:teuthology.orchestra.run.smithi043.stdout: "use_repo_digest": { 2024-03-26T08:40:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_repo_digest", 2024-03-26T08:40:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-26T08:40:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_failed_host_check": { 2024-03-26T08:40:03.295 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_failed_host_check", 2024-03-26T08:40:03.296 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.296 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.296 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.296 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.296 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.296 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.296 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.296 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if the host check fails", 2024-03-26T08:40:03.296 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.296 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.296 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.296 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_daemons": { 2024-03-26T08:40:03.296 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_daemons", 2024-03-26T08:40:03.296 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-26T08:40:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_hosts": { 2024-03-26T08:40:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_hosts", 2024-03-26T08:40:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.297 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.298 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.298 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.298 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.298 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.298 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.298 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-26T08:40:03.298 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.298 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.298 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.298 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.298 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.298 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crash", 2024-03-26T08:40:03.298 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.299 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.299 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.299 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.299 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.299 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.299 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.299 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.299 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.299 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.299 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.299 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.299 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.299 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.299 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.299 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.300 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.300 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.300 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.300 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.300 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.300 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.300 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.300 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.300 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.300 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.300 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.300 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.300 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.301 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.301 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.301 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.301 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.301 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.301 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.301 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.301 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.301 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.301 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.301 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.301 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.301 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.301 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.302 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.302 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.302 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.302 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.302 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.302 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.302 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.302 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.302 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.302 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.302 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.302 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.302 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.303 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.303 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.303 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.303 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.303 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.303 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.303 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.303 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.303 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.303 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.303 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.303 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.303 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.304 INFO:teuthology.orchestra.run.smithi043.stdout: "retain_interval": { 2024-03-26T08:40:03.304 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retain_interval", 2024-03-26T08:40:03.304 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.304 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.304 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.304 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "31536000", 2024-03-26T08:40:03.304 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.304 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.304 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.304 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-26T08:40:03.304 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.304 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.304 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.305 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_recent_interval": { 2024-03-26T08:40:03.305 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_recent_interval", 2024-03-26T08:40:03.305 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.305 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.305 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.305 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1209600", 2024-03-26T08:40:03.305 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.305 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.305 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.305 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-26T08:40:03.305 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.305 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.305 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.305 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.306 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.306 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dashboard", 2024-03-26T08:40:03.306 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.306 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.306 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.306 INFO:teuthology.orchestra.run.smithi043.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-26T08:40:03.306 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-26T08:40:03.306 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.306 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.306 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.306 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-03-26T08:40:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_HOST": { 2024-03-26T08:40:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-26T08:40:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.307 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.308 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.308 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.308 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.308 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.308 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.308 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-26T08:40:03.308 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-26T08:40:03.308 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.308 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.308 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.308 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.309 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.309 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.309 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.309 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.309 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.309 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.309 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.309 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_ENABLED": { 2024-03-26T08:40:03.309 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_ENABLED", 2024-03-26T08:40:03.309 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.309 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.309 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.309 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.309 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.309 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.310 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.310 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.310 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.310 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.310 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.310 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-26T08:40:03.310 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-26T08:40:03.310 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.310 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.310 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.310 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.310 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.310 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.310 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.311 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.311 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.311 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.311 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.311 INFO:teuthology.orchestra.run.smithi043.stdout: "ENABLE_BROWSABLE_API": { 2024-03-26T08:40:03.311 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-26T08:40:03.311 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.311 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.311 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.311 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.311 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.311 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.311 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.311 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.312 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.312 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.312 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.312 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-26T08:40:03.312 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-26T08:40:03.312 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.312 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.312 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.312 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.312 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.312 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.312 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.312 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.312 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.313 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.313 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.313 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-26T08:40:03.313 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-26T08:40:03.313 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.313 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.313 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.313 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.313 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.313 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.313 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.313 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.313 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-26T08:40:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-26T08:40:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.314 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.315 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.315 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-26T08:40:03.315 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-26T08:40:03.315 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.315 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.315 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.315 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.315 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.315 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.315 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.315 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.315 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.315 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-26T08:40:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-26T08:40:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-26T08:40:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-26T08:40:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.317 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-26T08:40:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-26T08:40:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.318 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-26T08:40:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-26T08:40:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_PASSWORD": { 2024-03-26T08:40:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-26T08:40:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-03-26T08:40:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-26T08:40:03.320 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-26T08:40:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_URL": { 2024-03-26T08:40:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_URL", 2024-03-26T08:40:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_USERNAME": { 2024-03-26T08:40:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-26T08:40:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.322 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-03-26T08:40:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-26T08:40:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-26T08:40:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.323 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-26T08:40:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-26T08:40:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.324 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.325 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.325 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.325 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.325 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.325 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.325 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.325 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.325 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.325 INFO:teuthology.orchestra.run.smithi043.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-26T08:40:03.325 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-26T08:40:03.325 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.325 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.325 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.326 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.326 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.326 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.326 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.326 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.326 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.326 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.326 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.326 INFO:teuthology.orchestra.run.smithi043.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-26T08:40:03.326 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-26T08:40:03.326 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.326 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.326 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.327 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.327 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.327 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.327 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.327 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.327 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.327 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.327 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.327 INFO:teuthology.orchestra.run.smithi043.stdout: "MULTICLUSTER_CONFIG": { 2024-03-26T08:40:03.327 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-26T08:40:03.327 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.327 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.327 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.327 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "{}", 2024-03-26T08:40:03.328 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.328 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.328 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.328 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.328 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.328 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.328 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.328 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_HOST": { 2024-03-26T08:40:03.328 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-26T08:40:03.328 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.328 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.328 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.328 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.328 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.329 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.329 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.329 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.329 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.329 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.329 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.329 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-26T08:40:03.329 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-26T08:40:03.329 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.329 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.329 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.329 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.329 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.329 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.330 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.330 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.330 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.330 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.330 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.330 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-26T08:40:03.330 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-26T08:40:03.330 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.330 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.333 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.333 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.333 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.333 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.333 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.333 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.333 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.333 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.334 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.334 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-26T08:40:03.334 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-26T08:40:03.334 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.334 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.334 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.334 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.334 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.334 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.334 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.334 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.334 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.335 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.335 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.335 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-26T08:40:03.335 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-26T08:40:03.335 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.335 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.335 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.335 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.335 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.335 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.335 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.335 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.335 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.336 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.336 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.336 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-26T08:40:03.336 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-26T08:40:03.336 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.336 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.336 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.336 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.336 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.336 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.336 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.336 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.336 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.336 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.337 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.337 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-26T08:40:03.337 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-26T08:40:03.337 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.337 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.337 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.337 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.337 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.337 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.337 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.337 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.337 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.337 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.337 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.338 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-26T08:40:03.338 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-26T08:40:03.338 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.338 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.338 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.338 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.338 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.338 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.338 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.338 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.338 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.338 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.338 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.339 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-26T08:40:03.339 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-26T08:40:03.339 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.339 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.339 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.339 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.339 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.339 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.339 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.339 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.339 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.339 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.339 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.340 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_ENABLED": { 2024-03-26T08:40:03.340 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-26T08:40:03.340 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.340 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.340 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.340 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.340 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.340 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.340 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.340 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.340 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.340 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.340 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.340 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-26T08:40:03.341 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-26T08:40:03.341 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.341 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.341 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.341 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-26T08:40:03.341 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.341 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.341 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.341 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.341 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.341 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.341 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.341 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-26T08:40:03.341 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-26T08:40:03.342 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.342 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.342 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.342 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-03-26T08:40:03.342 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.342 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.342 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.342 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.342 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.342 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.342 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.342 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-26T08:40:03.342 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-26T08:40:03.342 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.343 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.343 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.343 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8", 2024-03-26T08:40:03.343 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.343 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.343 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.343 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.343 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.343 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.343 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.343 INFO:teuthology.orchestra.run.smithi043.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-26T08:40:03.343 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-26T08:40:03.344 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.344 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.344 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.344 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "45", 2024-03-26T08:40:03.344 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.344 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.344 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.344 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.344 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.344 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.344 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.344 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ACCESS_KEY": { 2024-03-26T08:40:03.344 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-26T08:40:03.345 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.345 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.345 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.345 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.345 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.345 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.345 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.345 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.345 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.345 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.345 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.345 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-26T08:40:03.345 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-26T08:40:03.346 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.346 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.346 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.346 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-03-26T08:40:03.346 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.346 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.346 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.346 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.346 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.346 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.346 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.346 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SECRET_KEY": { 2024-03-26T08:40:03.346 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-26T08:40:03.347 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.347 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.347 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.347 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.347 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.347 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.347 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.347 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.347 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.347 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.347 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.347 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SSL_VERIFY": { 2024-03-26T08:40:03.347 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-26T08:40:03.347 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.347 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.348 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.348 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.348 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.348 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.348 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.348 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.348 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.348 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.348 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.348 INFO:teuthology.orchestra.run.smithi043.stdout: "UNSAFE_TLS_v1_2": { 2024-03-26T08:40:03.348 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-26T08:40:03.348 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.349 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.349 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.349 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.349 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.349 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.349 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.349 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.349 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.349 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.349 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.349 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-26T08:40:03.349 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-26T08:40:03.349 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.349 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.349 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.350 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-03-26T08:40:03.350 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.350 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.350 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.350 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.350 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.350 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.350 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.350 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-26T08:40:03.350 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-26T08:40:03.350 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.350 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.350 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.351 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-03-26T08:40:03.351 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.351 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.351 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.351 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.351 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.351 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.351 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.351 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-26T08:40:03.351 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-26T08:40:03.351 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.351 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.351 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.351 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-03-26T08:40:03.351 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.352 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.352 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.352 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.352 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.352 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.352 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.352 INFO:teuthology.orchestra.run.smithi043.stdout: "cross_origin_url": { 2024-03-26T08:40:03.352 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cross_origin_url", 2024-03-26T08:40:03.352 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.352 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.352 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.352 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.352 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.352 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.352 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.352 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.353 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.353 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.353 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.353 INFO:teuthology.orchestra.run.smithi043.stdout: "crt_file": { 2024-03-26T08:40:03.353 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crt_file", 2024-03-26T08:40:03.353 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.353 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.353 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.353 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.353 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.353 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.353 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.353 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.353 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.353 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.354 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.354 INFO:teuthology.orchestra.run.smithi043.stdout: "debug": { 2024-03-26T08:40:03.354 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "debug", 2024-03-26T08:40:03.354 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.354 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.354 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.354 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.354 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.354 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.354 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.354 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable/disable debug options", 2024-03-26T08:40:03.354 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.354 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.355 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.355 INFO:teuthology.orchestra.run.smithi043.stdout: "jwt_token_ttl": { 2024-03-26T08:40:03.355 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "jwt_token_ttl", 2024-03-26T08:40:03.355 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.355 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.355 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.355 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "28800", 2024-03-26T08:40:03.355 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.355 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.355 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.355 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.355 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.355 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.355 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.356 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-03-26T08:40:03.356 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-03-26T08:40:03.356 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.356 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.356 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.356 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.356 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.356 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.356 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.356 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.356 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.356 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.356 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.356 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.356 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.357 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.357 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.357 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.357 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.357 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.357 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.357 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.357 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.357 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.357 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.357 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.357 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.357 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.357 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.358 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.358 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.358 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.358 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.358 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.358 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.358 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.358 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.358 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.358 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.358 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.358 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.358 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.359 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.359 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.359 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.359 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.359 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.359 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.359 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.359 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.359 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.359 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.359 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.359 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.359 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.359 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.359 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.360 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.360 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.360 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.360 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.360 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.360 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.360 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.360 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.360 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.360 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.360 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.360 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.360 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.360 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.360 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.360 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.361 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.361 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.361 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.361 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.361 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.361 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.361 INFO:teuthology.orchestra.run.smithi043.stdout: "motd": { 2024-03-26T08:40:03.361 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "motd", 2024-03-26T08:40:03.361 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.361 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.361 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.361 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.361 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.361 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.362 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.362 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "The message of the day", 2024-03-26T08:40:03.362 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.362 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.362 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.362 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect_resolve_ip_addr": { 2024-03-26T08:40:03.362 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "redirect_resolve_ip_addr", 2024-03-26T08:40:03.362 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.362 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.362 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.362 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.362 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.363 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.363 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.363 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.363 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.363 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.363 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.363 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-03-26T08:40:03.363 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-03-26T08:40:03.363 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.363 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.363 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.363 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-03-26T08:40:03.363 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.364 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.364 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.364 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.364 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.364 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.364 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.364 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-03-26T08:40:03.364 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-03-26T08:40:03.364 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.364 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.364 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.364 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8080", 2024-03-26T08:40:03.364 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.364 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.365 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.365 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.365 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.365 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.365 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.365 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-03-26T08:40:03.365 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-03-26T08:40:03.365 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.365 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.365 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.365 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.365 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.365 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.365 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.366 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.366 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.366 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.366 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.366 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl_server_port": { 2024-03-26T08:40:03.366 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl_server_port", 2024-03-26T08:40:03.366 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.366 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.366 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.366 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8443", 2024-03-26T08:40:03.366 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.366 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.367 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.367 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.367 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.367 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.367 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.367 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-03-26T08:40:03.367 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-03-26T08:40:03.367 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.367 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.367 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.367 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "redirect", 2024-03-26T08:40:03.367 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.367 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.368 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.368 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.368 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect" 2024-03-26T08:40:03.368 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.368 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.368 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.368 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.368 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.368 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-03-26T08:40:03.368 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-03-26T08:40:03.368 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.368 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.368 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.369 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-03-26T08:40:03.369 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-03-26T08:40:03.369 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-03-26T08:40:03.369 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.369 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.369 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.369 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.369 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.369 INFO:teuthology.orchestra.run.smithi043.stdout: "url_prefix": { 2024-03-26T08:40:03.369 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url_prefix", 2024-03-26T08:40:03.369 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.369 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.369 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.370 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.370 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.370 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.370 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.370 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.370 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.370 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.370 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.370 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.370 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.370 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "devicehealth", 2024-03-26T08:40:03.370 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.371 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.371 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.371 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_monitoring": { 2024-03-26T08:40:03.371 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_monitoring", 2024-03-26T08:40:03.371 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.371 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.371 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.371 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.371 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.371 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.371 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.372 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "monitor device health metrics", 2024-03-26T08:40:03.372 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.372 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.372 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.372 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.372 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.372 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.372 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.372 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.372 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.372 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.372 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.372 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.373 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.373 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.373 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.373 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.373 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.373 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.373 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.373 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.373 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.373 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.373 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.373 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.373 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.374 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.374 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.374 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.374 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.374 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.374 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.374 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.374 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.374 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.374 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.374 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.374 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.375 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.375 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.375 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.375 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.375 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.375 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.375 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.375 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.375 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.375 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.375 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.375 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.375 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.375 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.376 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.376 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.376 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.376 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.376 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.376 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.376 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.376 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.376 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.376 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.376 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.376 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.376 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.376 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.377 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.377 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.377 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.377 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.377 INFO:teuthology.orchestra.run.smithi043.stdout: "mark_out_threshold": { 2024-03-26T08:40:03.377 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mark_out_threshold", 2024-03-26T08:40:03.377 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.377 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.377 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.377 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2419200", 2024-03-26T08:40:03.377 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.377 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.377 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.378 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-26T08:40:03.378 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.378 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.378 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.378 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": { 2024-03-26T08:40:03.378 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_name", 2024-03-26T08:40:03.378 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.378 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.378 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.378 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "device_health_metrics", 2024-03-26T08:40:03.378 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.378 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.379 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.379 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-26T08:40:03.379 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.379 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.379 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.379 INFO:teuthology.orchestra.run.smithi043.stdout: "retention_period": { 2024-03-26T08:40:03.379 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retention_period", 2024-03-26T08:40:03.379 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.379 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.379 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.379 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15552000", 2024-03-26T08:40:03.379 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.379 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.379 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.380 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain device health metrics", 2024-03-26T08:40:03.380 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.380 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.380 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.380 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_frequency": { 2024-03-26T08:40:03.380 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_frequency", 2024-03-26T08:40:03.380 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.380 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.380 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.380 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-03-26T08:40:03.380 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.380 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.380 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.381 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-26T08:40:03.381 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.381 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.381 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.381 INFO:teuthology.orchestra.run.smithi043.stdout: "self_heal": { 2024-03-26T08:40:03.381 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "self_heal", 2024-03-26T08:40:03.381 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.381 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.381 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.381 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.381 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.381 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.381 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.381 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-26T08:40:03.382 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.382 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.382 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.382 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-03-26T08:40:03.382 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-03-26T08:40:03.382 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.382 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.382 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.382 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-03-26T08:40:03.382 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.382 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.382 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.383 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and check device health", 2024-03-26T08:40:03.383 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.383 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.383 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.383 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_threshold": { 2024-03-26T08:40:03.383 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_threshold", 2024-03-26T08:40:03.383 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.383 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.383 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.383 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7257600", 2024-03-26T08:40:03.383 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.383 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.383 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.384 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-26T08:40:03.384 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.384 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.384 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.384 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.384 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.384 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "diskprediction_local", 2024-03-26T08:40:03.384 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.384 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.384 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.384 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.384 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.384 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.385 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.385 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.385 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.385 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.385 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.385 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.385 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.385 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.385 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.385 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.385 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.385 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.385 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.385 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.386 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.386 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.386 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.386 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.386 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.386 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.386 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.386 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.386 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.386 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.386 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.386 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.386 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.387 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.387 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.387 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.387 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.387 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.387 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.387 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.387 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.387 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.387 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.387 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.387 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.387 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.387 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.388 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.388 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.388 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.388 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.388 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.388 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.388 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.388 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.388 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.388 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.388 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.388 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.388 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.388 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.389 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.389 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.389 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.389 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.389 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.389 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.389 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.389 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.389 INFO:teuthology.orchestra.run.smithi043.stdout: "predict_interval": { 2024-03-26T08:40:03.389 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predict_interval", 2024-03-26T08:40:03.389 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.389 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.389 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.390 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-03-26T08:40:03.390 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.390 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.390 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.390 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.390 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.390 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.390 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.390 INFO:teuthology.orchestra.run.smithi043.stdout: "predictor_model": { 2024-03-26T08:40:03.390 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predictor_model", 2024-03-26T08:40:03.390 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.390 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.391 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.391 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "prophetstor", 2024-03-26T08:40:03.391 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.391 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.391 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.391 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.391 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.391 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.391 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.391 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-03-26T08:40:03.391 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-03-26T08:40:03.391 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.391 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.391 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.392 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-03-26T08:40:03.392 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.392 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.392 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.392 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.392 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.392 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.392 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.392 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.392 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.392 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "influx", 2024-03-26T08:40:03.392 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": false, 2024-03-26T08:40:03.393 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "influxdb python module not found", 2024-03-26T08:40:03.393 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.393 INFO:teuthology.orchestra.run.smithi043.stdout: "batch_size": { 2024-03-26T08:40:03.393 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "batch_size", 2024-03-26T08:40:03.393 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.393 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.393 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.393 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5000", 2024-03-26T08:40:03.393 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.393 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.393 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.393 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-26T08:40:03.393 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.393 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.393 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.394 INFO:teuthology.orchestra.run.smithi043.stdout: "database": { 2024-03-26T08:40:03.394 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "database", 2024-03-26T08:40:03.394 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.394 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.394 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.394 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "ceph", 2024-03-26T08:40:03.394 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.394 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.394 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.394 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-26T08:40:03.394 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.394 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.394 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.395 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": { 2024-03-26T08:40:03.395 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hostname", 2024-03-26T08:40:03.395 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.395 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.395 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.395 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.395 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.395 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.395 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.395 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server hostname", 2024-03-26T08:40:03.395 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.395 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.396 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.396 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-03-26T08:40:03.396 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-03-26T08:40:03.396 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.396 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.396 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.396 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "30", 2024-03-26T08:40:03.396 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "5", 2024-03-26T08:40:03.396 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.396 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.396 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-26T08:40:03.396 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.396 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.396 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.397 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.397 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.397 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.397 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.397 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.397 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.397 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.397 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.397 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.397 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.397 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.397 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.397 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.397 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.398 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.398 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.398 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.398 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.398 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.398 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.398 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.398 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.398 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.398 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.398 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.398 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.398 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.399 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.399 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.399 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.399 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.399 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.399 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.399 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.399 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.399 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.399 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.399 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.399 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.399 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.400 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.400 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.400 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.400 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.400 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.400 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.400 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.400 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.400 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.400 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.400 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.400 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.400 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.400 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.401 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.401 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.401 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.401 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.401 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.401 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.401 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.401 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.401 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.401 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.401 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.401 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.401 INFO:teuthology.orchestra.run.smithi043.stdout: "password": { 2024-03-26T08:40:03.402 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "password", 2024-03-26T08:40:03.402 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.402 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.402 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.402 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.402 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.402 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.402 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.402 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "password of InfluxDB server user", 2024-03-26T08:40:03.402 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.402 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.402 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.402 INFO:teuthology.orchestra.run.smithi043.stdout: "port": { 2024-03-26T08:40:03.402 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "port", 2024-03-26T08:40:03.403 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.403 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.403 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.403 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8086", 2024-03-26T08:40:03.403 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.403 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.403 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.403 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server port", 2024-03-26T08:40:03.403 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.403 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.403 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.403 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-03-26T08:40:03.403 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-03-26T08:40:03.404 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.404 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.404 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.404 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "false", 2024-03-26T08:40:03.404 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.404 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.404 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.404 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T08:40:03.404 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.404 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.404 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.404 INFO:teuthology.orchestra.run.smithi043.stdout: "threads": { 2024-03-26T08:40:03.404 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threads", 2024-03-26T08:40:03.404 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.405 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.405 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.405 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-03-26T08:40:03.405 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-03-26T08:40:03.405 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "32", 2024-03-26T08:40:03.405 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.405 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-26T08:40:03.405 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.405 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.405 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.405 INFO:teuthology.orchestra.run.smithi043.stdout: "username": { 2024-03-26T08:40:03.405 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "username", 2024-03-26T08:40:03.405 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.405 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.406 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.406 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.406 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.406 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.406 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.406 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "username of InfluxDB server user", 2024-03-26T08:40:03.406 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.406 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.406 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.406 INFO:teuthology.orchestra.run.smithi043.stdout: "verify_ssl": { 2024-03-26T08:40:03.406 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "verify_ssl", 2024-03-26T08:40:03.406 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.406 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.406 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.407 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "true", 2024-03-26T08:40:03.407 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.407 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.407 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.407 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T08:40:03.407 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.407 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.407 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.407 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.407 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.407 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "insights", 2024-03-26T08:40:03.407 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.407 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.408 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.408 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.408 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.408 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.408 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.408 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.408 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.408 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.408 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.408 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.408 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.408 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.408 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.409 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.409 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.409 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.409 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.409 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.409 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.409 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.409 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.409 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.409 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.409 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.409 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.409 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.409 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.410 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.410 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.410 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.410 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.410 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.410 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.410 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.410 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.410 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.410 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.410 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.410 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.410 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.411 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.411 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.411 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.411 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.411 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.411 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.411 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.411 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.411 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.411 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.411 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.411 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.411 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.411 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.412 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.412 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.412 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.412 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.412 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.412 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.412 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.412 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.412 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.412 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.412 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.412 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.412 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.412 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.412 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.413 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "iostat", 2024-03-26T08:40:03.413 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.413 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.413 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.413 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.413 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.413 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.413 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.413 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.413 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.413 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.413 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.413 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.414 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.414 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.414 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.414 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.414 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.414 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.414 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.414 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.414 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.414 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.414 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.414 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.415 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.415 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.415 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.415 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.415 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.415 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.415 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.415 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.415 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.415 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.415 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.415 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.415 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.415 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.416 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.416 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.416 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.416 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.416 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.416 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.416 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.416 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.416 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.416 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.416 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.416 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.416 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.416 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.416 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.416 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.417 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.417 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.417 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.417 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.417 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.417 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.417 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.417 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.417 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.417 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.417 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.417 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.417 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.417 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.418 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.418 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.418 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.418 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "k8sevents", 2024-03-26T08:40:03.418 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.418 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.418 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.418 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_event_retention_days": { 2024-03-26T08:40:03.418 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ceph_event_retention_days", 2024-03-26T08:40:03.418 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.418 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.418 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.419 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7", 2024-03-26T08:40:03.419 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.419 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.419 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.419 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-26T08:40:03.419 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.419 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.419 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.419 INFO:teuthology.orchestra.run.smithi043.stdout: "config_check_secs": { 2024-03-26T08:40:03.419 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_check_secs", 2024-03-26T08:40:03.419 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.419 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.419 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.419 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-03-26T08:40:03.419 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "10", 2024-03-26T08:40:03.420 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.420 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.420 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-26T08:40:03.420 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.420 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.420 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.420 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.420 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.420 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.420 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.420 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.420 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.420 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.420 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.421 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.421 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.421 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.421 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.421 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.421 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.421 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.421 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.421 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.421 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.421 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.421 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.421 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.422 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.422 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.422 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.422 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.422 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.422 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.422 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.422 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.422 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.422 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.422 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.422 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.422 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.423 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.423 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.423 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.423 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.423 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.423 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.423 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.423 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.423 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.423 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.423 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.423 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.423 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.423 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.423 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.424 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.424 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.424 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.424 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.424 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.424 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.424 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.424 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.424 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.424 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.424 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.424 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.424 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.425 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.425 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.425 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.425 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.425 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.425 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.425 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "localpool", 2024-03-26T08:40:03.425 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.425 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.425 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.425 INFO:teuthology.orchestra.run.smithi043.stdout: "failure_domain": { 2024-03-26T08:40:03.425 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "failure_domain", 2024-03-26T08:40:03.425 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.425 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.426 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.426 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "host", 2024-03-26T08:40:03.426 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.426 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.426 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.426 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "failure domain for any created local pool", 2024-03-26T08:40:03.426 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-26T08:40:03.426 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.426 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.426 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.426 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.426 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.426 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.427 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.427 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.427 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.427 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.427 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.427 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.427 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.427 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.427 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.427 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.427 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.427 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.427 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.427 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.427 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.427 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.428 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.428 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.428 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.428 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.428 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.428 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.428 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.428 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.428 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.428 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.428 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.428 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.428 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.429 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.429 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.429 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.429 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.429 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.429 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.429 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.429 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.429 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.429 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.429 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.429 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.429 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.429 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.429 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.430 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.430 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.430 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.430 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.430 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.430 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.430 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.430 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.430 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.430 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.430 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.430 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.430 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.431 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.431 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.431 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.431 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.431 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.431 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": { 2024-03-26T08:40:03.431 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_size", 2024-03-26T08:40:03.431 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.431 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.431 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.431 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.431 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.431 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.431 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.432 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default min_size for any created local pool", 2024-03-26T08:40:03.432 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-26T08:40:03.432 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.432 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.432 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rep": { 2024-03-26T08:40:03.432 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "num_rep", 2024-03-26T08:40:03.432 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.432 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.432 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.432 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3", 2024-03-26T08:40:03.432 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.432 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.432 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.432 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default replica count for any created local pool", 2024-03-26T08:40:03.433 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.433 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.433 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.433 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": { 2024-03-26T08:40:03.433 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_num", 2024-03-26T08:40:03.433 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.433 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.433 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.433 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "128", 2024-03-26T08:40:03.433 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.433 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.433 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.433 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default pg_num for any created local pool", 2024-03-26T08:40:03.433 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.433 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.434 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.434 INFO:teuthology.orchestra.run.smithi043.stdout: "prefix": { 2024-03-26T08:40:03.434 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prefix", 2024-03-26T08:40:03.434 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.434 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.434 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.434 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.434 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.435 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.435 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.435 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name prefix for any created local pool", 2024-03-26T08:40:03.435 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.435 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.435 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.435 INFO:teuthology.orchestra.run.smithi043.stdout: "subtree": { 2024-03-26T08:40:03.435 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "subtree", 2024-03-26T08:40:03.435 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.435 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.435 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.435 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "rack", 2024-03-26T08:40:03.435 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.435 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.436 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.436 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-26T08:40:03.436 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-26T08:40:03.436 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.436 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.436 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.436 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.436 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mds_autoscaler", 2024-03-26T08:40:03.436 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.436 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.436 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.436 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.437 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.437 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.437 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.437 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.437 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.437 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.437 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.437 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.437 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.437 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.437 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.437 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.437 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.437 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.438 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.438 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.438 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.438 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.438 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.438 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.438 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.438 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.438 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.438 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.438 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.438 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.438 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.439 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.439 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.439 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.439 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.439 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.439 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.439 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.439 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.439 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.439 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.439 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.439 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.439 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.440 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.440 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.440 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.440 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.440 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.440 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.440 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.440 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.440 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.440 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.440 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.440 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.440 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.440 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.441 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.441 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.441 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.441 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.441 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.441 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.441 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.441 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.441 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.441 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.441 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.441 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.441 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.442 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirroring", 2024-03-26T08:40:03.442 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.442 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.442 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.442 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.442 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.442 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.442 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.442 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.442 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.442 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.442 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.442 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.443 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.443 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.443 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.443 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.443 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.443 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.443 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.443 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.443 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.443 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.443 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.443 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.443 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.444 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.444 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.444 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.444 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.444 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.444 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.444 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.444 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.444 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.444 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.444 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.444 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.444 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.444 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.445 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.445 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.445 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.445 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.445 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.445 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.445 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.445 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.445 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.445 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.445 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.445 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.445 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.445 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.446 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.446 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.446 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.446 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.446 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.446 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.446 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.446 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.446 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.446 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.446 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.446 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.446 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.447 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.447 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.447 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.447 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.447 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.447 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "nfs", 2024-03-26T08:40:03.447 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.447 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.447 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.447 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.447 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.447 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.447 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.447 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.448 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.448 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.448 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.448 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.448 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.448 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.448 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.448 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.448 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.448 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.448 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.448 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.448 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.449 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.449 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.449 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.449 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.449 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.449 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.449 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.449 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.449 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.449 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.449 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.449 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.449 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.450 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.450 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.450 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.450 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.450 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.450 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.450 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.450 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.450 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.450 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.450 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.450 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.450 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.451 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.451 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.451 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.451 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.451 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.451 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.451 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.451 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.451 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.451 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.451 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.451 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.451 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.452 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.452 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.452 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.452 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.452 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.452 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.452 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.452 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.452 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.452 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.452 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.452 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-03-26T08:40:03.453 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.453 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.453 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.453 INFO:teuthology.orchestra.run.smithi043.stdout: "fail_fs": { 2024-03-26T08:40:03.453 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "fail_fs", 2024-03-26T08:40:03.453 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.453 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.453 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.453 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.453 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.453 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.453 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.453 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-26T08:40:03.453 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.454 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.454 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.454 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.454 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.454 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.454 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.454 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.454 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.454 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.454 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.454 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.454 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.455 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.455 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.455 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.455 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.455 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.455 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.455 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.455 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.455 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.455 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.455 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.455 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.455 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.455 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.455 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.456 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.456 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.456 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.456 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.456 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.456 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.456 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.456 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.456 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.456 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.456 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.456 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.456 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.456 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.457 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.457 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.457 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.457 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.457 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.457 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.457 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.457 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.457 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.457 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.457 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.457 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.457 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.457 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.458 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.458 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.458 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.458 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.458 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.458 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.458 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.458 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.458 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.458 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.458 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.458 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.458 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.459 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator": { 2024-03-26T08:40:03.459 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-03-26T08:40:03.459 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.459 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.459 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.459 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.459 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.459 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.459 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.459 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm", 2024-03-26T08:40:03.459 INFO:teuthology.orchestra.run.smithi043.stdout: "rook", 2024-03-26T08:40:03.459 INFO:teuthology.orchestra.run.smithi043.stdout: "test_orchestrator" 2024-03-26T08:40:03.459 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.460 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Orchestrator backend", 2024-03-26T08:40:03.460 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.460 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.460 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.460 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.460 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.460 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_perf_query", 2024-03-26T08:40:03.460 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.460 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.460 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.460 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.460 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.460 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.460 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.461 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.461 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.461 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.461 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.461 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.461 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.461 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.461 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.461 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.461 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.461 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.461 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.461 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.461 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.462 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.462 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.462 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.462 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.462 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.462 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.462 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.462 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.462 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.462 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.462 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.462 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.462 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.463 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.463 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.463 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.463 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.463 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.463 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.463 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.463 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.463 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.463 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.463 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.463 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.463 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.463 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.463 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.464 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.464 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.464 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.464 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.464 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.464 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.464 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.464 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.464 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.464 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.464 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.464 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.464 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.464 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.464 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.465 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.465 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.465 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.465 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.465 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.465 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.465 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.465 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_support", 2024-03-26T08:40:03.465 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.465 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.465 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.465 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.466 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.466 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.466 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.466 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.466 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.466 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.466 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.466 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.466 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.466 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.466 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.466 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.466 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.466 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.467 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.467 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.467 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.467 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.467 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.467 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.467 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.467 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.467 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.467 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.467 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.467 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.467 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.468 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.468 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.468 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.468 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.468 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.468 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.468 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.468 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.468 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.468 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.468 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.468 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.468 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.468 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.469 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.469 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.469 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.469 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.469 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.469 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.469 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.469 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.469 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.469 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.469 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.469 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.469 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.470 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.470 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.470 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.470 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.470 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.470 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.470 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.470 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.470 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.470 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.470 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.470 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.470 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.471 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_autoscaler", 2024-03-26T08:40:03.471 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.471 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.471 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.471 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.471 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.471 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.471 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.471 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.471 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.471 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.471 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.471 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.471 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.471 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.472 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.472 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.472 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.472 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.472 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.472 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.472 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.472 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.472 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.472 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.472 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.472 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.472 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.472 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.473 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.473 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.473 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.473 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.473 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.473 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.473 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.473 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.473 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.473 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.473 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.473 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.473 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.473 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.474 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.474 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.474 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.474 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.474 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.474 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.474 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.474 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.474 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.474 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.474 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.474 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.474 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.474 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.475 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.475 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.475 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.475 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.475 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.475 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.475 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.475 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.475 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.475 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.475 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.475 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.475 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.476 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-03-26T08:40:03.476 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-03-26T08:40:03.476 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.476 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.476 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.476 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-03-26T08:40:03.476 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.476 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.476 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.476 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.476 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.476 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.476 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.477 INFO:teuthology.orchestra.run.smithi043.stdout: "threshold": { 2024-03-26T08:40:03.477 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threshold", 2024-03-26T08:40:03.477 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-03-26T08:40:03.477 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.477 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.477 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-03-26T08:40:03.477 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1.0", 2024-03-26T08:40:03.477 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.477 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.477 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "scaling threshold", 2024-03-26T08:40:03.477 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-26T08:40:03.477 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.478 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.478 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.478 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.478 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "progress", 2024-03-26T08:40:03.478 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.478 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.478 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.478 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_pg_recovery_event": { 2024-03-26T08:40:03.478 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_pg_recovery_event", 2024-03-26T08:40:03.478 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.478 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.478 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.479 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.479 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.479 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.479 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.479 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-26T08:40:03.479 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.479 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.479 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.479 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-03-26T08:40:03.479 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-03-26T08:40:03.479 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.479 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.479 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.480 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.480 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.480 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.480 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.480 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.480 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.480 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.480 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.480 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.480 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.480 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.480 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.480 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.480 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.481 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.481 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.481 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.481 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.481 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.481 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.481 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.481 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.481 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.481 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.481 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.481 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.481 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.481 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.482 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.482 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.482 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.482 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.482 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.482 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.482 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.482 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.482 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.482 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.482 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.482 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.482 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.483 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.483 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.483 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.483 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.483 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.483 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.483 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.483 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.483 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.483 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.483 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.483 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.483 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.483 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.484 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.484 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.484 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.484 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.484 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.484 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.484 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.484 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.484 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.484 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.484 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.484 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.484 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.484 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.485 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.485 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.485 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.485 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.485 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.485 INFO:teuthology.orchestra.run.smithi043.stdout: "max_completed_events": { 2024-03-26T08:40:03.485 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_completed_events", 2024-03-26T08:40:03.485 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.485 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.485 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.485 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "50", 2024-03-26T08:40:03.485 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.485 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.485 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.485 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "number of past completed events to remember", 2024-03-26T08:40:03.486 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.486 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.486 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.486 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-03-26T08:40:03.486 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-03-26T08:40:03.486 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.486 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.486 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.486 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-03-26T08:40:03.486 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.486 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.486 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.486 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long the module is going to sleep", 2024-03-26T08:40:03.487 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.487 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.487 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.487 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.487 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.487 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus", 2024-03-26T08:40:03.487 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.487 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.487 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.487 INFO:teuthology.orchestra.run.smithi043.stdout: "cache": { 2024-03-26T08:40:03.487 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cache", 2024-03-26T08:40:03.487 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.487 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.487 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.488 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.488 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.488 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.488 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.488 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.488 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.488 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.488 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.488 INFO:teuthology.orchestra.run.smithi043.stdout: "exclude_perf_counters": { 2024-03-26T08:40:03.488 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "exclude_perf_counters", 2024-03-26T08:40:03.488 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.488 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.488 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.488 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.488 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.489 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.489 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.489 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-26T08:40:03.489 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-26T08:40:03.489 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.489 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.489 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.489 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.489 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.489 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.489 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.489 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.490 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.490 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.490 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.490 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.490 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.490 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.490 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.490 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.490 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.490 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.490 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.490 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.490 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.490 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.491 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.491 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.491 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.491 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.491 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.491 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.491 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.491 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.491 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.491 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.491 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.491 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.491 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.491 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.492 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.492 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.492 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.492 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.492 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.492 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.492 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.492 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.492 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.492 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.492 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.492 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.492 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.492 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.492 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.492 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.493 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.493 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.493 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.493 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.493 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.493 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.493 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.493 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.493 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.493 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.493 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.493 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.493 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.493 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.493 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.494 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.494 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools": { 2024-03-26T08:40:03.494 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools", 2024-03-26T08:40:03.494 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.494 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.494 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.494 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.494 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.494 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.494 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.494 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.494 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.494 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.495 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.495 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-26T08:40:03.495 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-26T08:40:03.495 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.495 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.495 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.495 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "300", 2024-03-26T08:40:03.495 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.495 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.495 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.495 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.495 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.495 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.496 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.496 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_interval": { 2024-03-26T08:40:03.496 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_interval", 2024-03-26T08:40:03.496 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-03-26T08:40:03.496 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.496 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.496 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15.0", 2024-03-26T08:40:03.496 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.496 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.496 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.496 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.496 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.497 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.497 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.497 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-03-26T08:40:03.497 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-03-26T08:40:03.497 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.497 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.497 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.497 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-03-26T08:40:03.497 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.497 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.497 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.497 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-26T08:40:03.497 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.498 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.498 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.498 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-03-26T08:40:03.498 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-03-26T08:40:03.498 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.498 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.498 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.498 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "9283", 2024-03-26T08:40:03.498 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.498 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.498 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.498 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-26T08:40:03.498 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.498 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.499 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.499 INFO:teuthology.orchestra.run.smithi043.stdout: "stale_cache_strategy": { 2024-03-26T08:40:03.499 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stale_cache_strategy", 2024-03-26T08:40:03.499 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.499 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.499 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.499 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "log", 2024-03-26T08:40:03.499 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.499 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.499 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.499 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.499 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.499 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.500 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.500 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-03-26T08:40:03.500 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-03-26T08:40:03.500 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.500 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.500 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.500 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "default", 2024-03-26T08:40:03.500 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.500 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.500 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.500 INFO:teuthology.orchestra.run.smithi043.stdout: "default", 2024-03-26T08:40:03.500 INFO:teuthology.orchestra.run.smithi043.stdout: "error" 2024-03-26T08:40:03.500 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.501 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.501 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.501 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.501 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.501 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-03-26T08:40:03.501 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-03-26T08:40:03.501 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.501 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.501 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.501 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-03-26T08:40:03.501 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-03-26T08:40:03.501 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-03-26T08:40:03.501 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.501 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.502 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.502 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.502 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.502 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.502 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.502 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_support", 2024-03-26T08:40:03.502 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.502 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.502 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.502 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.502 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.502 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.503 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.503 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.503 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.503 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.503 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.503 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.503 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.503 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.503 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.503 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.503 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.503 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.503 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.503 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.503 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.504 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.504 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.504 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.504 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.504 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.504 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.504 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.504 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.504 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.504 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.504 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.504 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.504 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.505 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.505 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.505 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.505 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.505 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.505 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.505 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.505 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.505 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.505 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.505 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.505 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.505 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.506 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.506 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.506 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.506 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.506 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.506 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.506 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.506 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.506 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.506 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.506 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.506 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.506 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.507 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.507 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.507 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.507 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.507 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.507 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.507 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.507 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.507 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.507 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_snap_create": { 2024-03-26T08:40:03.507 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_snap_create", 2024-03-26T08:40:03.507 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.507 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.508 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.508 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-03-26T08:40:03.508 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.508 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.508 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.508 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.508 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.508 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.508 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.508 INFO:teuthology.orchestra.run.smithi043.stdout: "mirror_snapshot_schedule": { 2024-03-26T08:40:03.508 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirror_snapshot_schedule", 2024-03-26T08:40:03.508 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.508 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.509 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.509 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.509 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.509 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.509 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.509 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.509 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.509 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.509 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.509 INFO:teuthology.orchestra.run.smithi043.stdout: "trash_purge_schedule": { 2024-03-26T08:40:03.509 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "trash_purge_schedule", 2024-03-26T08:40:03.509 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.509 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.509 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.510 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.510 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.510 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.510 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.510 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.510 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.510 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.510 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.510 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.510 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.510 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "restful", 2024-03-26T08:40:03.510 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.510 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.511 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.511 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_auth": { 2024-03-26T08:40:03.511 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_auth", 2024-03-26T08:40:03.511 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.511 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.511 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.511 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.511 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.511 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.511 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.511 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.511 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.511 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.511 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.512 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-03-26T08:40:03.512 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-03-26T08:40:03.512 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.512 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.512 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.512 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.512 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.512 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.512 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.512 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.512 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.512 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.512 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.513 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.513 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.513 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.513 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.513 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.513 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.513 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.513 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.513 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.513 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.513 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.513 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.513 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.513 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.513 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.514 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.514 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.514 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.514 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.514 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.514 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.514 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.514 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.514 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.514 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.514 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.514 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.515 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.515 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.515 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.515 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.515 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.515 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.515 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.515 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.515 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.515 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.515 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.515 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.515 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.516 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.516 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.516 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.516 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.516 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.516 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.516 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.516 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.516 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.516 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.516 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.516 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.516 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.517 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.517 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.517 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.517 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.517 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.517 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.517 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.517 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.517 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.517 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.517 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.517 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.517 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.517 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-03-26T08:40:03.518 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-03-26T08:40:03.518 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.518 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.518 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.518 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.518 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.518 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.518 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.518 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.518 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.518 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.518 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.519 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-03-26T08:40:03.519 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-03-26T08:40:03.519 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.519 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.519 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.519 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.519 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.519 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.519 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.519 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.519 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.519 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.519 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.519 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.519 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.520 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rgw", 2024-03-26T08:40:03.520 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.520 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.520 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.520 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.520 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.520 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.520 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.520 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.520 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.520 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.520 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.520 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.521 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.521 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.521 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.521 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.521 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.521 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.521 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.521 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.521 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.521 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.521 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.521 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.521 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.521 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.522 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.522 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.522 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.522 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.522 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.522 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.522 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.522 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.522 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.522 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.522 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.522 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.522 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.523 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.523 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.523 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.523 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.523 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.523 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.523 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.523 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.523 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.523 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.523 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.523 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.523 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.523 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.523 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.523 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.524 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.524 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.524 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.524 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.524 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.524 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.524 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.524 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.524 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.524 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.524 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.524 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.524 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.524 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.525 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.525 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.525 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rook", 2024-03-26T08:40:03.525 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.525 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.525 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.525 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.525 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.525 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.525 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.525 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.525 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.525 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.526 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.526 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.526 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.526 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.526 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.526 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.526 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.526 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.526 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.526 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.526 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.526 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.526 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.527 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.527 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.527 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.527 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.527 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.527 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.527 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.527 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.527 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.527 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.527 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.527 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.527 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.528 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.528 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.528 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.528 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.528 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.528 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.528 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.528 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.528 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.528 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.528 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.528 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.528 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.528 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.529 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.529 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.529 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.529 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.529 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.529 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.529 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.529 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.529 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.529 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.529 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.529 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.529 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.529 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.530 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.530 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.530 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.530 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.530 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.530 INFO:teuthology.orchestra.run.smithi043.stdout: "storage_class": { 2024-03-26T08:40:03.530 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "storage_class", 2024-03-26T08:40:03.530 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.530 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.530 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.530 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "local", 2024-03-26T08:40:03.530 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.530 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.531 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.531 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-26T08:40:03.531 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.531 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.531 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.531 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.531 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.531 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "selftest", 2024-03-26T08:40:03.531 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.531 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.531 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.531 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.532 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.532 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.532 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.532 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.532 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.532 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.532 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.532 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.532 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.532 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.532 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.532 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.532 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.532 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.533 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.533 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.533 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.533 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.533 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.533 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.533 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.533 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.533 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.533 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.533 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.533 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.533 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.533 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.534 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.534 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.534 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.534 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.534 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.534 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.534 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.534 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.534 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.534 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.534 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.534 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.534 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.535 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.535 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.535 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.535 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.535 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.535 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.535 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.535 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.535 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.535 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.537 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.537 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.537 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.537 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.537 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.537 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.537 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.538 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.538 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.538 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.538 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.538 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.538 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.538 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.538 INFO:teuthology.orchestra.run.smithi043.stdout: "roption1": { 2024-03-26T08:40:03.538 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption1", 2024-03-26T08:40:03.538 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.538 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.538 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.539 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.539 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.539 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.539 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.539 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.539 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.539 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.539 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.539 INFO:teuthology.orchestra.run.smithi043.stdout: "roption2": { 2024-03-26T08:40:03.539 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption2", 2024-03-26T08:40:03.539 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.539 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.539 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.540 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "xyz", 2024-03-26T08:40:03.540 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.540 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.540 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.540 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.540 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.540 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.540 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.540 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption1": { 2024-03-26T08:40:03.540 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption1", 2024-03-26T08:40:03.540 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.540 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.540 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.540 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.541 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.541 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.541 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.541 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.541 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.541 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.541 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.541 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption2": { 2024-03-26T08:40:03.541 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption2", 2024-03-26T08:40:03.541 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.541 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.541 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.541 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.541 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.542 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.542 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.542 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.542 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.542 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.542 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.542 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption3": { 2024-03-26T08:40:03.542 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption3", 2024-03-26T08:40:03.542 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-03-26T08:40:03.542 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.542 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.542 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.542 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.543 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.543 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.543 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.543 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.543 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.543 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.543 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption4": { 2024-03-26T08:40:03.543 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption4", 2024-03-26T08:40:03.543 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.543 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.543 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.543 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.544 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.544 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.544 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.544 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.544 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.544 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.544 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.544 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption5": { 2024-03-26T08:40:03.544 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption5", 2024-03-26T08:40:03.544 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.544 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.544 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.544 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.544 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.545 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.545 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.545 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.545 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.545 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.545 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.545 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption6": { 2024-03-26T08:40:03.545 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption6", 2024-03-26T08:40:03.545 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.545 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.545 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.545 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.545 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.545 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.546 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.546 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.546 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.546 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.546 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.546 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption7": { 2024-03-26T08:40:03.546 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption7", 2024-03-26T08:40:03.546 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.546 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.546 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.546 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.546 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-03-26T08:40:03.546 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "42", 2024-03-26T08:40:03.547 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.547 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.547 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.547 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.547 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.547 INFO:teuthology.orchestra.run.smithi043.stdout: "testkey": { 2024-03-26T08:40:03.547 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testkey", 2024-03-26T08:40:03.547 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.547 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.547 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.547 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.547 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.547 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.548 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.548 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.548 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.548 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.548 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.548 INFO:teuthology.orchestra.run.smithi043.stdout: "testlkey": { 2024-03-26T08:40:03.548 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testlkey", 2024-03-26T08:40:03.548 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.548 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.548 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.548 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.548 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.548 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.548 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.549 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.549 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.549 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.549 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.549 INFO:teuthology.orchestra.run.smithi043.stdout: "testnewline": { 2024-03-26T08:40:03.549 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testnewline", 2024-03-26T08:40:03.549 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.549 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.549 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.549 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.549 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.549 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.549 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.550 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.550 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.550 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.550 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.550 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.550 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.550 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snap_schedule", 2024-03-26T08:40:03.550 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.550 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.550 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.550 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_m_granularity": { 2024-03-26T08:40:03.550 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_m_granularity", 2024-03-26T08:40:03.550 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.551 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.551 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.551 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.551 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.551 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.551 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.551 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow minute scheduled snapshots", 2024-03-26T08:40:03.551 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.551 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.551 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.551 INFO:teuthology.orchestra.run.smithi043.stdout: "dump_on_update": { 2024-03-26T08:40:03.551 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dump_on_update", 2024-03-26T08:40:03.551 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.552 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.552 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.552 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.552 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.552 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.552 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.552 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "dump database to debug log on update", 2024-03-26T08:40:03.552 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.552 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.552 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.552 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.552 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.552 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.553 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.553 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.553 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.553 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.553 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.553 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.553 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.553 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.554 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.554 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.554 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.554 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.554 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.554 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.554 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.554 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.555 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.555 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.555 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.555 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.555 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.555 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.555 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.556 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.556 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.556 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.556 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.556 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.556 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.556 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.557 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.557 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.557 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.557 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.557 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.557 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.557 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.557 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.558 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.558 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.558 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.558 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.558 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.558 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.558 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.558 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.558 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.559 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.559 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.559 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.559 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.559 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.559 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.559 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.559 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.559 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.559 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.559 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.559 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.559 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.560 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.560 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.560 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.560 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.560 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.560 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stats", 2024-03-26T08:40:03.560 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.560 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.560 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.560 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.560 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.560 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.561 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.561 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.561 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.561 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.561 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.561 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.561 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.561 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.561 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.561 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.561 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.561 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.561 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.561 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.561 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.562 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.562 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.562 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.562 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.562 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.562 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.562 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.562 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.562 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.562 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.562 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.562 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.562 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.563 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.563 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.563 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.563 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.563 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.563 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.563 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.563 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.563 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.563 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.563 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.563 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.563 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.563 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.564 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.564 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.564 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.564 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.564 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.564 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.564 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.564 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.564 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.564 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.564 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.564 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.564 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.565 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.565 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.565 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.565 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.565 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.565 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.565 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.565 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.565 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.565 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.565 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "status", 2024-03-26T08:40:03.565 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.565 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.566 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.566 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.566 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.566 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.566 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.566 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.566 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.566 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.566 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.566 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.566 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.566 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.566 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.566 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.567 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.567 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.567 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.567 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.567 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.567 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.567 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.567 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.567 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.567 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.567 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.567 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.567 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.568 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.568 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.568 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.568 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.568 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.568 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.568 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.568 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.568 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.568 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.568 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.568 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.568 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.568 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.568 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.569 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.569 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.569 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.569 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.569 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.569 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.569 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.569 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.569 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.569 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.569 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.569 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.569 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.570 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.570 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.570 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.570 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.570 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.570 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.570 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.570 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.570 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.570 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.570 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.570 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.570 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.570 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.571 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telegraf", 2024-03-26T08:40:03.571 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.571 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.571 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.571 INFO:teuthology.orchestra.run.smithi043.stdout: "address": { 2024-03-26T08:40:03.571 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "address", 2024-03-26T08:40:03.571 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.571 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.571 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.571 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-26T08:40:03.571 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.571 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.572 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.572 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.572 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.572 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.572 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.572 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-03-26T08:40:03.572 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-03-26T08:40:03.572 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.572 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.572 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.572 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15", 2024-03-26T08:40:03.572 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.572 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.572 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.573 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.573 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.573 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.573 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.573 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.573 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.573 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.573 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.573 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.573 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.573 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.573 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.574 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.574 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.574 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.574 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.574 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.574 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.574 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.574 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.574 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.574 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.574 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.574 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.574 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.575 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.575 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.575 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.575 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.575 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.575 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.575 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.575 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.575 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.575 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.575 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.575 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.575 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.575 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.577 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.577 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.577 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.577 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.577 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.577 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.577 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.577 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.577 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.577 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.577 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.577 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.577 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.577 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.578 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.578 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.578 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.578 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.578 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.578 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telemetry", 2024-03-26T08:40:03.578 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.578 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.578 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.578 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_basic": { 2024-03-26T08:40:03.578 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_basic", 2024-03-26T08:40:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-26T08:40:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_crash": { 2024-03-26T08:40:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_crash", 2024-03-26T08:40:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.580 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.580 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.580 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.580 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.580 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.580 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.580 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-26T08:40:03.580 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.580 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.580 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.580 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_device": { 2024-03-26T08:40:03.580 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_device", 2024-03-26T08:40:03.580 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.580 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.581 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.581 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.581 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.581 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.581 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.581 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-26T08:40:03.581 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.581 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.581 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.581 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_ident": { 2024-03-26T08:40:03.581 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_ident", 2024-03-26T08:40:03.581 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.581 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-26T08:40:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_perf": { 2024-03-26T08:40:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_perf", 2024-03-26T08:40:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.583 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.583 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.583 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.583 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.583 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-26T08:40:03.583 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.583 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.583 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.583 INFO:teuthology.orchestra.run.smithi043.stdout: "contact": { 2024-03-26T08:40:03.583 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "contact", 2024-03-26T08:40:03.583 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.583 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.583 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.583 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.584 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.584 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.584 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.584 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.584 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.584 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.584 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.584 INFO:teuthology.orchestra.run.smithi043.stdout: "description": { 2024-03-26T08:40:03.584 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "description", 2024-03-26T08:40:03.584 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.584 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.584 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.584 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.584 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: "device_url": { 2024-03-26T08:40:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_url", 2024-03-26T08:40:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-26T08:40:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.586 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.586 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.586 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.586 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.586 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.586 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-03-26T08:40:03.586 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-03-26T08:40:03.586 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.586 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.586 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.586 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.586 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.586 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-03-26T08:40:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-03-26T08:40:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "24", 2024-03-26T08:40:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "8", 2024-03-26T08:40:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.588 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.588 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.588 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.588 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.588 INFO:teuthology.orchestra.run.smithi043.stdout: "last_opt_revision": { 2024-03-26T08:40:03.588 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "last_opt_revision", 2024-03-26T08:40:03.588 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.588 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.588 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.588 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1", 2024-03-26T08:40:03.588 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.588 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.588 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.588 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.589 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.589 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.589 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.589 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard": { 2024-03-26T08:40:03.589 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard", 2024-03-26T08:40:03.589 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.589 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.589 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.589 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.589 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.589 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.590 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.590 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.590 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.590 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.590 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.590 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard_description": { 2024-03-26T08:40:03.590 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard_description", 2024-03-26T08:40:03.590 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.590 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.590 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.590 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.590 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.590 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.590 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.591 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.591 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.591 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.591 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.591 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.591 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.591 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.591 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.591 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.591 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.591 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.591 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.591 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.593 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.593 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.593 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.593 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.593 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.593 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.593 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.593 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.593 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.593 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.593 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.593 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.593 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.593 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.594 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.594 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.594 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.594 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.594 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.594 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.594 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.594 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.594 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.594 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.594 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.594 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.594 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.594 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.595 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.595 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.595 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.595 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.595 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.595 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.595 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.595 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.595 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.595 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.595 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.595 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.595 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.595 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.596 INFO:teuthology.orchestra.run.smithi043.stdout: "organization": { 2024-03-26T08:40:03.596 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "organization", 2024-03-26T08:40:03.596 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.596 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.596 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.596 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.596 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.596 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.596 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.596 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.596 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.596 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.596 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.596 INFO:teuthology.orchestra.run.smithi043.stdout: "proxy": { 2024-03-26T08:40:03.597 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "proxy", 2024-03-26T08:40:03.597 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.597 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.597 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.597 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.597 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.597 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.597 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.597 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.597 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.597 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.597 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.597 INFO:teuthology.orchestra.run.smithi043.stdout: "url": { 2024-03-26T08:40:03.597 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url", 2024-03-26T08:40:03.598 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.598 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.598 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.598 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-26T08:40:03.598 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.598 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.598 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.598 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.598 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.598 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.598 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.598 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.598 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.599 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "test_orchestrator", 2024-03-26T08:40:03.599 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.599 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.599 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.599 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.599 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.599 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.599 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.599 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.599 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.599 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.599 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.599 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.599 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.600 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.600 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.600 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.600 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.600 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.600 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.600 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.600 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.600 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.600 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.600 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.600 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.600 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.600 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.601 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.601 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.601 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.601 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.601 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.601 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.601 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.601 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.601 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.601 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.601 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.601 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.601 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.602 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.602 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.602 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.602 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.602 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.602 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.602 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.602 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.602 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.602 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.602 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.602 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.602 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.602 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.602 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.603 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.603 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.603 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.603 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.603 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.603 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.603 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.603 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.603 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.603 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.603 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.603 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.603 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.603 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.604 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.604 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.604 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "volumes", 2024-03-26T08:40:03.604 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.604 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.604 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.604 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.604 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.604 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.604 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.604 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.604 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.604 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.605 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.605 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.605 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.605 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.605 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.605 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.605 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.605 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.605 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.605 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.605 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.605 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.605 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.605 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.606 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.606 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.606 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.606 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.606 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.606 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.606 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.606 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.606 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.606 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.606 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.606 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.606 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.606 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.607 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.607 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.607 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.607 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.607 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.607 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.607 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.607 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.607 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.607 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.607 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.607 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.607 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.607 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.607 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_clones": { 2024-03-26T08:40:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_clones", 2024-03-26T08:40:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4", 2024-03-26T08:40:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-26T08:40:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.610 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.610 INFO:teuthology.orchestra.run.smithi043.stdout: "periodic_async_work": { 2024-03-26T08:40:03.610 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "periodic_async_work", 2024-03-26T08:40:03.610 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.610 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.610 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.610 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.610 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.610 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.610 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.610 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Periodically check for async work", 2024-03-26T08:40:03.610 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.610 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.610 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.611 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_delay": { 2024-03-26T08:40:03.611 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_delay", 2024-03-26T08:40:03.611 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.611 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.611 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.611 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-03-26T08:40:03.611 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.611 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.611 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.611 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-26T08:40:03.611 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.611 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.611 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.612 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_no_wait": { 2024-03-26T08:40:03.612 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_no_wait", 2024-03-26T08:40:03.612 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.612 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.612 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.612 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.612 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.612 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.612 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.612 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-26T08:40:03.612 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.612 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.612 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.612 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix", 2024-03-26T08:40:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: "discovery_interval": { 2024-03-26T08:40:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "discovery_interval", 2024-03-26T08:40:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-03-26T08:40:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "100", 2024-03-26T08:40:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.614 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.614 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.614 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.614 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.614 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.614 INFO:teuthology.orchestra.run.smithi043.stdout: "identifier": { 2024-03-26T08:40:03.614 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "identifier", 2024-03-26T08:40:03.614 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.614 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.614 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.614 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.614 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.614 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.615 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.615 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.615 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.615 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.615 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.615 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-03-26T08:40:03.615 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-03-26T08:40:03.615 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.615 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.615 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.615 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-03-26T08:40:03.615 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.615 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.615 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.616 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.616 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.616 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.616 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.616 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.616 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.616 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.616 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.616 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.616 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.616 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.617 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.617 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.617 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.617 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.617 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.617 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.617 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.617 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.617 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.617 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.617 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.617 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.617 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.618 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.618 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.618 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.618 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.618 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.618 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.618 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.618 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.618 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.618 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.618 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.618 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.618 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.619 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.619 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.619 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.619 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.619 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.619 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.619 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.619 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.619 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.619 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.619 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.619 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.619 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.619 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.620 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.620 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.620 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.620 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.620 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.620 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.620 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.620 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.620 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.620 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.620 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.620 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.620 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.620 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.621 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.621 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.621 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.621 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.621 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.621 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_host": { 2024-03-26T08:40:03.621 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_host", 2024-03-26T08:40:03.621 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.621 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.621 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.621 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.621 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.621 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.621 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.622 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.622 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.622 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.622 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.622 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_port": { 2024-03-26T08:40:03.622 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_port", 2024-03-26T08:40:03.622 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.622 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.622 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.622 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10051", 2024-03-26T08:40:03.622 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.622 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.622 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.622 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.623 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.623 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.623 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.623 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_sender": { 2024-03-26T08:40:03.623 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_sender", 2024-03-26T08:40:03.623 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.623 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.623 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.623 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-26T08:40:03.623 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.623 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.623 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.623 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.623 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-03-26T08:40:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: "modules": [ 2024-03-26T08:40:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm", 2024-03-26T08:40:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: "dashboard", 2024-03-26T08:40:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: "iostat", 2024-03-26T08:40:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: "nfs", 2024-03-26T08:40:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: "restful" 2024-03-26T08:40:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: "available_modules": [ 2024-03-26T08:40:03.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.625 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "alerts", 2024-03-26T08:40:03.625 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.625 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.625 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.625 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-03-26T08:40:03.625 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-03-26T08:40:03.625 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.625 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.625 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.625 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-03-26T08:40:03.625 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.625 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.625 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.626 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How frequently to reexamine health status", 2024-03-26T08:40:03.626 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.626 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.626 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.626 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.626 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.626 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.626 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.626 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.626 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.626 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.626 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.626 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.627 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.627 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.627 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.627 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.627 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.627 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.627 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.627 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.627 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.627 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.627 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.627 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.627 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.628 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.628 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.628 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.628 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.628 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.628 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.628 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.628 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.628 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.628 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.628 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.628 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.628 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.629 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.629 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.629 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.629 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.629 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.629 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.629 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.629 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.629 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.629 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.629 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.629 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.629 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.629 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.629 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.630 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.630 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.630 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.630 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.630 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.630 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.630 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.630 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.630 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.630 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.630 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.630 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.630 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.631 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.631 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.631 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.631 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_destination": { 2024-03-26T08:40:03.631 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_destination", 2024-03-26T08:40:03.631 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.631 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.631 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.631 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.631 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.631 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.631 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.631 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email address to send alerts to", 2024-03-26T08:40:03.632 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.632 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.632 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.632 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_from_name": { 2024-03-26T08:40:03.632 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_from_name", 2024-03-26T08:40:03.632 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.632 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.632 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.632 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "Ceph", 2024-03-26T08:40:03.632 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.632 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.632 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.633 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Email From: name", 2024-03-26T08:40:03.633 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.633 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.633 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.633 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_host": { 2024-03-26T08:40:03.633 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_host", 2024-03-26T08:40:03.633 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.633 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.633 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.633 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.633 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.633 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.633 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.634 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP server", 2024-03-26T08:40:03.634 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.634 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.634 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.634 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_password": { 2024-03-26T08:40:03.634 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_password", 2024-03-26T08:40:03.634 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.634 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.634 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.634 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.634 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.634 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.634 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Password to authenticate with", 2024-03-26T08:40:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_port": { 2024-03-26T08:40:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_port", 2024-03-26T08:40:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "465", 2024-03-26T08:40:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP port", 2024-03-26T08:40:03.636 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.636 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.636 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.636 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_sender": { 2024-03-26T08:40:03.636 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_sender", 2024-03-26T08:40:03.636 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.636 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.636 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.636 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.636 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.636 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.636 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.636 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SMTP envelope sender", 2024-03-26T08:40:03.636 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.637 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.637 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.637 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_ssl": { 2024-03-26T08:40:03.637 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_ssl", 2024-03-26T08:40:03.637 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.637 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.637 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.637 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.637 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.637 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.637 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.637 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-26T08:40:03.637 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.637 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.638 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.638 INFO:teuthology.orchestra.run.smithi043.stdout: "smtp_user": { 2024-03-26T08:40:03.638 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "smtp_user", 2024-03-26T08:40:03.638 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.638 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.638 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.638 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.638 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.638 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.638 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.638 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "User to authenticate as", 2024-03-26T08:40:03.638 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.638 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.639 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.639 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.639 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.639 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "balancer", 2024-03-26T08:40:03.639 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.639 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.639 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.639 INFO:teuthology.orchestra.run.smithi043.stdout: "active": { 2024-03-26T08:40:03.639 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "active", 2024-03-26T08:40:03.639 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.639 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.639 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.639 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically balance PGs across cluster", 2024-03-26T08:40:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_time": { 2024-03-26T08:40:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_time", 2024-03-26T08:40:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0000", 2024-03-26T08:40:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.641 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.641 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "beginning time of day to automatically balance", 2024-03-26T08:40:03.641 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T08:40:03.641 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.641 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.641 INFO:teuthology.orchestra.run.smithi043.stdout: "begin_weekday": { 2024-03-26T08:40:03.641 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "begin_weekday", 2024-03-26T08:40:03.641 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-03-26T08:40:03.641 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.641 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.641 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-03-26T08:40:03.641 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-03-26T08:40:03.641 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-03-26T08:40:03.641 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.642 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-26T08:40:03.642 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T08:40:03.642 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.642 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.642 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_max_iterations": { 2024-03-26T08:40:03.642 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_max_iterations", 2024-03-26T08:40:03.642 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-03-26T08:40:03.642 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.642 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.642 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "25", 2024-03-26T08:40:03.642 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-03-26T08:40:03.642 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "250", 2024-03-26T08:40:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-26T08:40:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_metrics": { 2024-03-26T08:40:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_metrics", 2024-03-26T08:40:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "pgs,objects,bytes", 2024-03-26T08:40:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.644 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-26T08:40:03.644 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-26T08:40:03.644 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.644 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.644 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_compat_step": { 2024-03-26T08:40:03.644 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crush_compat_step", 2024-03-26T08:40:03.644 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-03-26T08:40:03.644 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.644 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.644 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.5", 2024-03-26T08:40:03.644 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0.001", 2024-03-26T08:40:03.644 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "0.999", 2024-03-26T08:40:03.644 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.645 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "aggressiveness of optimization", 2024-03-26T08:40:03.645 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-26T08:40:03.645 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.645 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.645 INFO:teuthology.orchestra.run.smithi043.stdout: "end_time": { 2024-03-26T08:40:03.645 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_time", 2024-03-26T08:40:03.645 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.645 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.645 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.645 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2359", 2024-03-26T08:40:03.645 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.645 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.645 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.645 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ending time of day to automatically balance", 2024-03-26T08:40:03.646 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T08:40:03.646 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.646 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.646 INFO:teuthology.orchestra.run.smithi043.stdout: "end_weekday": { 2024-03-26T08:40:03.646 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "end_weekday", 2024-03-26T08:40:03.646 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-03-26T08:40:03.646 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.646 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.646 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-03-26T08:40:03.646 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "0", 2024-03-26T08:40:03.646 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "6", 2024-03-26T08:40:03.646 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.646 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-26T08:40:03.647 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T08:40:03.647 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.647 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.647 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.647 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.647 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.647 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.647 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.647 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.647 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.647 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.647 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.647 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.648 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.648 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.648 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.648 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.648 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.648 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.648 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.648 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.648 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.648 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.648 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.648 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.648 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.648 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.648 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.649 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.649 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.649 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.649 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.649 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.649 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.649 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.649 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.649 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.649 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.649 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.649 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.649 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.650 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.650 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.650 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.650 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.650 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.650 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.650 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.650 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.650 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.650 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.650 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.650 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.650 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.650 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.651 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.651 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.651 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.651 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.651 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.651 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.651 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.651 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.651 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.651 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.651 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.651 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.651 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.651 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.652 INFO:teuthology.orchestra.run.smithi043.stdout: "min_score": { 2024-03-26T08:40:03.652 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_score", 2024-03-26T08:40:03.652 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-03-26T08:40:03.652 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.652 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.652 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-03-26T08:40:03.652 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.652 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.652 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.652 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-26T08:40:03.652 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.652 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.652 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-03-26T08:40:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-03-26T08:40:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "upmap", 2024-03-26T08:40:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: "crush-compat", 2024-03-26T08:40:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: "none", 2024-03-26T08:40:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: "read", 2024-03-26T08:40:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap", 2024-03-26T08:40:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap-read" 2024-03-26T08:40:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.654 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Balancer mode", 2024-03-26T08:40:03.654 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.654 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.654 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.654 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ids": { 2024-03-26T08:40:03.654 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_ids", 2024-03-26T08:40:03.654 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.654 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.654 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.654 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.654 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.654 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.654 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.655 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-26T08:40:03.655 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.655 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.655 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.655 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-03-26T08:40:03.655 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-03-26T08:40:03.655 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.655 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.655 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.655 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-03-26T08:40:03.655 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.655 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.655 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.655 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-26T08:40:03.656 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.656 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.656 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.656 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_deviation": { 2024-03-26T08:40:03.656 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_deviation", 2024-03-26T08:40:03.656 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.656 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.656 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.656 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-03-26T08:40:03.656 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-03-26T08:40:03.656 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.656 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.656 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-26T08:40:03.657 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-26T08:40:03.657 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.657 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.657 INFO:teuthology.orchestra.run.smithi043.stdout: "upmap_max_optimizations": { 2024-03-26T08:40:03.657 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "upmap_max_optimizations", 2024-03-26T08:40:03.657 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-03-26T08:40:03.657 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.657 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.657 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-03-26T08:40:03.657 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.657 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.657 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.657 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-26T08:40:03.657 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.658 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.658 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.658 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.658 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.658 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm", 2024-03-26T08:40:03.658 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.658 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.658 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.658 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_down_multiplier": { 2024-03-26T08:40:03.658 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_down_multiplier", 2024-03-26T08:40:03.658 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-03-26T08:40:03.658 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.659 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.659 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-03-26T08:40:03.659 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.659 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.659 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.659 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-26T08:40:03.659 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.659 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.659 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.659 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_refresh_rate": { 2024-03-26T08:40:03.659 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_refresh_rate", 2024-03-26T08:40:03.659 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.659 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.660 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.660 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "20", 2024-03-26T08:40:03.660 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.660 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.660 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.660 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-26T08:40:03.660 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.660 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.660 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.660 INFO:teuthology.orchestra.run.smithi043.stdout: "agent_starting_port": { 2024-03-26T08:40:03.660 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "agent_starting_port", 2024-03-26T08:40:03.660 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.660 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.661 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.661 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4721", 2024-03-26T08:40:03.661 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.661 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.661 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.661 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-26T08:40:03.661 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.661 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.661 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.661 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_ptrace": { 2024-03-26T08:40:03.661 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_ptrace", 2024-03-26T08:40:03.661 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.661 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.661 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.661 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.662 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.662 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.662 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.662 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-26T08:40:03.662 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-26T08:40:03.662 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.662 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.662 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_interval": { 2024-03-26T08:40:03.662 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_interval", 2024-03-26T08:40:03.662 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.662 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.662 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.662 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-03-26T08:40:03.663 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.663 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.663 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.663 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-26T08:40:03.663 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.663 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.663 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.663 INFO:teuthology.orchestra.run.smithi043.stdout: "autotune_memory_target_ratio": { 2024-03-26T08:40:03.663 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "autotune_memory_target_ratio", 2024-03-26T08:40:03.663 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-03-26T08:40:03.663 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.663 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.663 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0.7", 2024-03-26T08:40:03.664 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.664 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.664 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.664 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-26T08:40:03.664 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.664 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.664 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.664 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm_log_destination": { 2024-03-26T08:40:03.664 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cephadm_log_destination", 2024-03-26T08:40:03.664 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.664 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.664 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.664 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.664 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.665 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.665 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.665 INFO:teuthology.orchestra.run.smithi043.stdout: "file", 2024-03-26T08:40:03.665 INFO:teuthology.orchestra.run.smithi043.stdout: "file,syslog", 2024-03-26T08:40:03.665 INFO:teuthology.orchestra.run.smithi043.stdout: "syslog" 2024-03-26T08:40:03.665 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.665 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-26T08:40:03.665 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.665 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.665 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.665 INFO:teuthology.orchestra.run.smithi043.stdout: "cgroups_split": { 2024-03-26T08:40:03.665 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cgroups_split", 2024-03-26T08:40:03.665 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.665 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.666 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.666 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.666 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.666 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.666 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.666 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-26T08:40:03.666 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.666 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.666 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.666 INFO:teuthology.orchestra.run.smithi043.stdout: "config_checks_enabled": { 2024-03-26T08:40:03.666 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_checks_enabled", 2024-03-26T08:40:03.666 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.666 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.667 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.667 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.667 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.667 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.667 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.667 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-26T08:40:03.667 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.667 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.667 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.667 INFO:teuthology.orchestra.run.smithi043.stdout: "config_dashboard": { 2024-03-26T08:40:03.667 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_dashboard", 2024-03-26T08:40:03.667 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.667 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.667 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.668 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.668 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.668 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.668 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.668 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-26T08:40:03.668 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.668 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.668 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.668 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_alertmanager": { 2024-03-26T08:40:03.668 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_alertmanager", 2024-03-26T08:40:03.668 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.668 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.668 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.668 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-26T08:40:03.669 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.669 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.669 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.669 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-03-26T08:40:03.669 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.669 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.669 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.669 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_base": { 2024-03-26T08:40:03.669 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_base", 2024-03-26T08:40:03.669 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.669 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.669 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.669 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-26T08:40:03.669 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.670 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.670 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.670 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Container image name, without the tag", 2024-03-26T08:40:03.670 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.670 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.670 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.670 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_elasticsearch": { 2024-03-26T08:40:03.670 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_elasticsearch", 2024-03-26T08:40:03.670 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.670 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.670 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.670 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-26T08:40:03.670 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "elasticsearch container image", 2024-03-26T08:40:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_grafana": { 2024-03-26T08:40:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_grafana", 2024-03-26T08:40:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-26T08:40:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.672 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.672 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-03-26T08:40:03.672 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.672 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.672 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.672 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_haproxy": { 2024-03-26T08:40:03.672 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_haproxy", 2024-03-26T08:40:03.672 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.672 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.672 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.672 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-26T08:40:03.672 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.672 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.673 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.673 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "HAproxy container image", 2024-03-26T08:40:03.673 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.673 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.673 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.673 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_agent": { 2024-03-26T08:40:03.673 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_agent", 2024-03-26T08:40:03.673 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.673 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.673 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.673 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-26T08:40:03.673 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.673 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.674 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.674 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger agent container image", 2024-03-26T08:40:03.674 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.674 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.674 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.674 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_collector": { 2024-03-26T08:40:03.674 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_collector", 2024-03-26T08:40:03.674 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.674 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.674 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.674 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-26T08:40:03.674 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.674 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.674 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.675 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger collector container image", 2024-03-26T08:40:03.675 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.675 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.675 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.675 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_jaeger_query": { 2024-03-26T08:40:03.675 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_jaeger_query", 2024-03-26T08:40:03.675 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.675 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.675 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.675 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-26T08:40:03.675 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.675 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.675 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.676 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Jaeger query container image", 2024-03-26T08:40:03.676 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.676 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.676 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.676 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_keepalived": { 2024-03-26T08:40:03.676 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_keepalived", 2024-03-26T08:40:03.676 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.676 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.676 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.676 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-26T08:40:03.676 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.676 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.676 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.676 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Keepalived container image", 2024-03-26T08:40:03.677 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.677 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.677 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.677 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_loki": { 2024-03-26T08:40:03.677 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_loki", 2024-03-26T08:40:03.677 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.677 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.677 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.677 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-26T08:40:03.677 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.677 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.677 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.677 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Loki container image", 2024-03-26T08:40:03.678 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.678 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.678 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.678 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_node_exporter": { 2024-03-26T08:40:03.678 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_node_exporter", 2024-03-26T08:40:03.678 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.678 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.678 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.678 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-26T08:40:03.678 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.678 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.678 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.678 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-03-26T08:40:03.678 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.679 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.679 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.679 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_nvmeof": { 2024-03-26T08:40:03.679 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_nvmeof", 2024-03-26T08:40:03.679 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.679 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.679 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.679 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-26T08:40:03.679 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.679 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.679 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.679 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Nvme-of container image", 2024-03-26T08:40:03.679 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.680 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.680 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.680 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_prometheus": { 2024-03-26T08:40:03.680 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_prometheus", 2024-03-26T08:40:03.680 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.680 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.680 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.680 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-26T08:40:03.680 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.680 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.680 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.680 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Prometheus container image", 2024-03-26T08:40:03.680 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.680 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.681 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.681 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_promtail": { 2024-03-26T08:40:03.681 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_promtail", 2024-03-26T08:40:03.681 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.681 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.681 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.681 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-26T08:40:03.681 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.681 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.681 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.681 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Promtail container image", 2024-03-26T08:40:03.681 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.681 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.681 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.682 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image_snmp_gateway": { 2024-03-26T08:40:03.682 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_image_snmp_gateway", 2024-03-26T08:40:03.682 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.682 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.682 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.682 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-26T08:40:03.682 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.682 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.682 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.682 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "SNMP Gateway container image", 2024-03-26T08:40:03.682 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.682 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.682 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.683 INFO:teuthology.orchestra.run.smithi043.stdout: "container_init": { 2024-03-26T08:40:03.683 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "container_init", 2024-03-26T08:40:03.683 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.683 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.683 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.683 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.683 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.683 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.683 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.683 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Run podman/docker with `--init`", 2024-03-26T08:40:03.683 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.683 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.683 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.684 INFO:teuthology.orchestra.run.smithi043.stdout: "daemon_cache_timeout": { 2024-03-26T08:40:03.684 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "daemon_cache_timeout", 2024-03-26T08:40:03.684 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.684 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.684 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.684 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-03-26T08:40:03.684 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.684 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.684 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.684 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-26T08:40:03.684 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.684 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.684 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.685 INFO:teuthology.orchestra.run.smithi043.stdout: "default_cephadm_command_timeout": { 2024-03-26T08:40:03.685 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_cephadm_command_timeout", 2024-03-26T08:40:03.685 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.685 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.685 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.685 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "900", 2024-03-26T08:40:03.685 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.685 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.685 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.685 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-26T08:40:03.685 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.685 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.685 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.686 INFO:teuthology.orchestra.run.smithi043.stdout: "default_registry": { 2024-03-26T08:40:03.686 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "default_registry", 2024-03-26T08:40:03.686 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.686 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.686 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.686 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "docker.io", 2024-03-26T08:40:03.686 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.686 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.686 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.686 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-26T08:40:03.686 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.686 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.686 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.687 INFO:teuthology.orchestra.run.smithi043.stdout: "device_cache_timeout": { 2024-03-26T08:40:03.687 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_cache_timeout", 2024-03-26T08:40:03.687 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.687 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.687 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.687 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1800", 2024-03-26T08:40:03.687 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.687 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.687 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.687 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache device inventory", 2024-03-26T08:40:03.687 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.687 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.687 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.687 INFO:teuthology.orchestra.run.smithi043.stdout: "device_enhanced_scan": { 2024-03-26T08:40:03.688 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_enhanced_scan", 2024-03-26T08:40:03.688 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.688 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.688 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.688 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.688 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.688 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.688 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.688 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-26T08:40:03.688 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.688 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.688 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.688 INFO:teuthology.orchestra.run.smithi043.stdout: "facts_cache_timeout": { 2024-03-26T08:40:03.688 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "facts_cache_timeout", 2024-03-26T08:40:03.689 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.689 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.689 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.689 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-03-26T08:40:03.689 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.689 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.689 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.689 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "seconds to cache host facts data", 2024-03-26T08:40:03.689 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.689 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.689 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.689 INFO:teuthology.orchestra.run.smithi043.stdout: "host_check_interval": { 2024-03-26T08:40:03.689 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "host_check_interval", 2024-03-26T08:40:03.689 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.690 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.690 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.690 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-03-26T08:40:03.690 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.690 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.690 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.690 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to perform a host check", 2024-03-26T08:40:03.690 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.690 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.690 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.690 INFO:teuthology.orchestra.run.smithi043.stdout: "hw_monitoring": { 2024-03-26T08:40:03.690 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hw_monitoring", 2024-03-26T08:40:03.690 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.690 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.691 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.691 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.691 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.691 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.691 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.691 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-26T08:40:03.691 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.691 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.691 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.691 INFO:teuthology.orchestra.run.smithi043.stdout: "inventory_list_all": { 2024-03-26T08:40:03.691 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "inventory_list_all", 2024-03-26T08:40:03.691 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.691 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.691 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-26T08:40:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: "log_refresh_metadata": { 2024-03-26T08:40:03.694 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_refresh_metadata", 2024-03-26T08:40:03.694 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.694 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.694 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.694 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.694 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.694 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.694 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.694 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-26T08:40:03.694 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.694 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.694 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.694 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.694 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.695 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.695 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.695 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.695 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.695 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.695 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.695 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.695 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-26T08:40:03.695 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.695 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.695 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.695 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.695 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.698 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.698 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-26T08:40:03.698 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-26T08:40:03.698 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.698 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.698 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.698 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.698 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.698 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.698 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.698 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-26T08:40:03.698 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.698 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.699 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.699 INFO:teuthology.orchestra.run.smithi043.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-26T08:40:03.699 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-26T08:40:03.699 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.699 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.699 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.699 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "*", 2024-03-26T08:40:03.699 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.699 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.699 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.699 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-26T08:40:03.699 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.699 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.699 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.700 INFO:teuthology.orchestra.run.smithi043.stdout: "max_count_per_host": { 2024-03-26T08:40:03.700 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_count_per_host", 2024-03-26T08:40:03.700 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.700 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.700 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.700 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-03-26T08:40:03.700 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.700 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.700 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.700 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of daemons per service per host", 2024-03-26T08:40:03.700 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.700 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.700 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.701 INFO:teuthology.orchestra.run.smithi043.stdout: "max_osd_draining_count": { 2024-03-26T08:40:03.701 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_osd_draining_count", 2024-03-26T08:40:03.701 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.701 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.701 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.701 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-03-26T08:40:03.701 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.701 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.701 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.701 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-26T08:40:03.701 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.701 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.701 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.701 INFO:teuthology.orchestra.run.smithi043.stdout: "migration_current": { 2024-03-26T08:40:03.702 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "migration_current", 2024-03-26T08:40:03.702 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.702 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.702 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.702 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.702 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.702 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.702 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.702 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "internal - do not modify", 2024-03-26T08:40:03.702 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.702 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.702 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.702 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": { 2024-03-26T08:40:03.703 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mode", 2024-03-26T08:40:03.703 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.703 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.703 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.703 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "root", 2024-03-26T08:40:03.703 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.703 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.703 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.703 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm-package", 2024-03-26T08:40:03.703 INFO:teuthology.orchestra.run.smithi043.stdout: "root" 2024-03-26T08:40:03.703 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.703 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "mode for remote execution of cephadm", 2024-03-26T08:40:03.703 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.703 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.703 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.704 INFO:teuthology.orchestra.run.smithi043.stdout: "oob_default_addr": { 2024-03-26T08:40:03.704 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "oob_default_addr", 2024-03-26T08:40:03.704 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.704 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.704 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.704 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "169.254.1.1", 2024-03-26T08:40:03.704 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.704 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.704 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.704 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-26T08:40:03.704 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.704 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.704 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.705 INFO:teuthology.orchestra.run.smithi043.stdout: "prometheus_alerts_path": { 2024-03-26T08:40:03.705 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus_alerts_path", 2024-03-26T08:40:03.705 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.705 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.705 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.705 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-26T08:40:03.705 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.705 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.705 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.705 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-26T08:40:03.705 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.705 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.705 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.705 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_insecure": { 2024-03-26T08:40:03.706 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_insecure", 2024-03-26T08:40:03.706 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.706 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.706 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.706 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.706 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.706 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.706 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.706 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-26T08:40:03.706 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.706 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.706 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.706 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_password": { 2024-03-26T08:40:03.706 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_password", 2024-03-26T08:40:03.707 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.707 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.707 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.707 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.707 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.707 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.707 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.707 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-26T08:40:03.707 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.707 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.707 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.707 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_url": { 2024-03-26T08:40:03.707 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_url", 2024-03-26T08:40:03.707 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.708 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.708 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.708 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.708 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.708 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.708 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.708 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-26T08:40:03.708 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.708 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.708 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.708 INFO:teuthology.orchestra.run.smithi043.stdout: "registry_username": { 2024-03-26T08:40:03.708 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "registry_username", 2024-03-26T08:40:03.708 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.708 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.709 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.709 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.709 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.709 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.709 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.709 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-26T08:40:03.709 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.709 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.709 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.709 INFO:teuthology.orchestra.run.smithi043.stdout: "secure_monitoring_stack": { 2024-03-26T08:40:03.709 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "secure_monitoring_stack", 2024-03-26T08:40:03.709 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.709 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.709 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-26T08:40:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: "service_discovery_port": { 2024-03-26T08:40:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "service_discovery_port", 2024-03-26T08:40:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8765", 2024-03-26T08:40:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "cephadm service discovery port", 2024-03-26T08:40:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "ssh_config_file": { 2024-03-26T08:40:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssh_config_file", 2024-03-26T08:40:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-26T08:40:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: "use_agent": { 2024-03-26T08:40:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_agent", 2024-03-26T08:40:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-26T08:40:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "use_repo_digest": { 2024-03-26T08:40:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "use_repo_digest", 2024-03-26T08:40:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-26T08:40:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_failed_host_check": { 2024-03-26T08:40:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_failed_host_check", 2024-03-26T08:40:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if the host check fails", 2024-03-26T08:40:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_daemons": { 2024-03-26T08:40:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_daemons", 2024-03-26T08:40:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-26T08:40:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_on_stray_hosts": { 2024-03-26T08:40:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_on_stray_hosts", 2024-03-26T08:40:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-26T08:40:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crash", 2024-03-26T08:40:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "retain_interval": { 2024-03-26T08:40:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retain_interval", 2024-03-26T08:40:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "31536000", 2024-03-26T08:40:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-26T08:40:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_recent_interval": { 2024-03-26T08:40:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_recent_interval", 2024-03-26T08:40:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1209600", 2024-03-26T08:40:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-26T08:40:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dashboard", 2024-03-26T08:40:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-26T08:40:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-26T08:40:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-03-26T08:40:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_HOST": { 2024-03-26T08:40:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-26T08:40:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-26T08:40:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-26T08:40:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_ENABLED": { 2024-03-26T08:40:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_ENABLED", 2024-03-26T08:40:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-26T08:40:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-26T08:40:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "ENABLE_BROWSABLE_API": { 2024-03-26T08:40:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-26T08:40:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-26T08:40:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-26T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-26T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-26T08:40:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-26T08:40:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-26T08:40:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-26T08:40:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-26T08:40:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-26T08:40:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-26T08:40:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-26T08:40:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-26T08:40:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-26T08:40:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-26T08:40:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-26T08:40:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-26T08:40:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_PASSWORD": { 2024-03-26T08:40:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-26T08:40:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-03-26T08:40:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-26T08:40:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-26T08:40:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.740 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_URL": { 2024-03-26T08:40:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_URL", 2024-03-26T08:40:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_API_USERNAME": { 2024-03-26T08:40:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-26T08:40:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-03-26T08:40:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-26T08:40:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-26T08:40:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-26T08:40:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-26T08:40:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-26T08:40:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-26T08:40:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-26T08:40:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-26T08:40:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: "MULTICLUSTER_CONFIG": { 2024-03-26T08:40:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-26T08:40:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "{}", 2024-03-26T08:40:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_HOST": { 2024-03-26T08:40:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-26T08:40:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.750 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-26T08:40:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-26T08:40:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.751 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-26T08:40:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-26T08:40:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.753 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.753 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.753 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.753 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.753 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.753 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-26T08:40:03.753 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-26T08:40:03.753 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.753 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.753 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.753 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.753 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.753 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.753 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.754 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.754 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.754 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.754 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.754 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-26T08:40:03.754 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-26T08:40:03.754 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.754 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.754 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.754 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.754 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.754 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.754 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-26T08:40:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-26T08:40:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.755 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-26T08:40:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-26T08:40:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.756 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.757 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.757 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-26T08:40:03.757 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-26T08:40:03.757 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.757 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.757 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.757 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.757 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.757 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.757 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.757 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.757 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.757 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.757 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.758 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-26T08:40:03.758 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-26T08:40:03.758 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.758 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.758 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.758 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.758 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.758 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.758 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.758 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.758 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.758 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.758 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.759 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_ENABLED": { 2024-03-26T08:40:03.759 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-26T08:40:03.759 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.759 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.759 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.759 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.759 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.759 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.759 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.759 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.759 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.759 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.759 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.760 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-26T08:40:03.760 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-26T08:40:03.760 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.760 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.760 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.760 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-26T08:40:03.760 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.760 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.760 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.760 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.760 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.760 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.760 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-26T08:40:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-26T08:40:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-03-26T08:40:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.762 INFO:teuthology.orchestra.run.smithi043.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-26T08:40:03.762 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-26T08:40:03.762 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.762 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.762 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.762 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8", 2024-03-26T08:40:03.762 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.762 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.762 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.762 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.762 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.762 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.762 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.763 INFO:teuthology.orchestra.run.smithi043.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-26T08:40:03.763 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-26T08:40:03.763 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.763 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.763 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.763 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "45", 2024-03-26T08:40:03.763 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.763 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.763 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.763 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.763 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.763 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.763 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.764 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ACCESS_KEY": { 2024-03-26T08:40:03.764 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-26T08:40:03.764 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.764 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.764 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.764 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.764 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.764 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.764 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.764 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.764 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.764 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.764 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.765 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-26T08:40:03.765 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-26T08:40:03.765 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.765 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.765 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.765 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "admin", 2024-03-26T08:40:03.765 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.765 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.765 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.765 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.765 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.765 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.765 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SECRET_KEY": { 2024-03-26T08:40:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-26T08:40:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.766 INFO:teuthology.orchestra.run.smithi043.stdout: "RGW_API_SSL_VERIFY": { 2024-03-26T08:40:03.767 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-26T08:40:03.767 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.767 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.767 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.767 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.767 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.767 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.767 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.767 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.767 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.767 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.767 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.768 INFO:teuthology.orchestra.run.smithi043.stdout: "UNSAFE_TLS_v1_2": { 2024-03-26T08:40:03.768 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-26T08:40:03.768 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.768 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.768 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.768 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.768 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.768 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.768 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.768 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.768 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.768 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.768 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.768 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-26T08:40:03.769 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-26T08:40:03.769 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.769 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.769 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.769 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-03-26T08:40:03.769 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.769 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.769 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.769 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.769 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.769 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.769 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.769 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-26T08:40:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-26T08:40:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-03-26T08:40:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-26T08:40:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-26T08:40:03.770 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-03-26T08:40:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: "cross_origin_url": { 2024-03-26T08:40:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cross_origin_url", 2024-03-26T08:40:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.771 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.772 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.772 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.772 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.772 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.772 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.772 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.772 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.772 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.772 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.772 INFO:teuthology.orchestra.run.smithi043.stdout: "crt_file": { 2024-03-26T08:40:03.772 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "crt_file", 2024-03-26T08:40:03.772 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.772 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.772 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.773 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.773 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.773 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.773 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.773 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.773 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.773 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.773 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.773 INFO:teuthology.orchestra.run.smithi043.stdout: "debug": { 2024-03-26T08:40:03.773 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "debug", 2024-03-26T08:40:03.773 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.773 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.773 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Enable/disable debug options", 2024-03-26T08:40:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: "jwt_token_ttl": { 2024-03-26T08:40:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "jwt_token_ttl", 2024-03-26T08:40:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.774 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.775 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "28800", 2024-03-26T08:40:03.775 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.775 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.775 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.775 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.775 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.775 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.775 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.775 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-03-26T08:40:03.775 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-03-26T08:40:03.775 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.775 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.775 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.776 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.776 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.776 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.776 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.776 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.776 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.776 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.776 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.776 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.776 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.776 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.776 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.776 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.776 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.777 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.777 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.777 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.777 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.777 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.777 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.777 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.777 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.777 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.777 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.777 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.777 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.777 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.777 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.779 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.780 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.781 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.781 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.781 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.781 INFO:teuthology.orchestra.run.smithi043.stdout: "motd": { 2024-03-26T08:40:03.781 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "motd", 2024-03-26T08:40:03.781 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.781 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.781 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.781 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.781 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.781 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.781 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.781 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "The message of the day", 2024-03-26T08:40:03.781 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.782 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.782 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.782 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect_resolve_ip_addr": { 2024-03-26T08:40:03.782 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "redirect_resolve_ip_addr", 2024-03-26T08:40:03.782 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.782 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.782 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.782 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.782 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.782 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.782 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.782 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.782 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.783 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.783 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.783 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-03-26T08:40:03.783 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-03-26T08:40:03.783 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.783 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.783 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.783 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-03-26T08:40:03.783 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.783 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.783 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.783 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.783 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.783 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-03-26T08:40:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-03-26T08:40:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8080", 2024-03-26T08:40:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.784 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.785 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.785 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-03-26T08:40:03.785 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-03-26T08:40:03.785 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.785 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.785 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.785 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.785 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.785 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.785 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.785 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.785 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.785 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.786 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.786 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl_server_port": { 2024-03-26T08:40:03.786 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl_server_port", 2024-03-26T08:40:03.786 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.786 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.786 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.786 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8443", 2024-03-26T08:40:03.786 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.786 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.786 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.786 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.786 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.786 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-03-26T08:40:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-03-26T08:40:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "redirect", 2024-03-26T08:40:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: "redirect" 2024-03-26T08:40:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.787 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.788 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.788 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.788 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.788 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-03-26T08:40:03.788 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-03-26T08:40:03.788 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.788 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.788 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.788 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-03-26T08:40:03.788 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-03-26T08:40:03.788 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-03-26T08:40:03.788 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.788 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.789 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.789 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.789 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.789 INFO:teuthology.orchestra.run.smithi043.stdout: "url_prefix": { 2024-03-26T08:40:03.789 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url_prefix", 2024-03-26T08:40:03.789 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.789 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.789 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.789 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.789 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.789 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.789 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.789 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.789 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.790 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.790 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.790 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.790 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.790 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "devicehealth", 2024-03-26T08:40:03.790 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.790 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.790 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.790 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_monitoring": { 2024-03-26T08:40:03.790 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_monitoring", 2024-03-26T08:40:03.790 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.790 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "monitor device health metrics", 2024-03-26T08:40:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.791 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.792 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.792 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.792 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.792 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.792 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.792 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.792 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.792 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.792 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.792 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.792 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.792 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.792 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.792 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.793 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.793 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.793 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.793 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.793 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.793 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.793 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.793 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.793 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.793 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.793 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.793 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.793 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.794 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.794 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.794 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.794 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.794 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.794 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.794 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.794 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.794 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.794 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.794 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.794 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.795 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.796 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.796 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.796 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.796 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.796 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.796 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.796 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.796 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.796 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.796 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.796 INFO:teuthology.orchestra.run.smithi043.stdout: "mark_out_threshold": { 2024-03-26T08:40:03.796 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mark_out_threshold", 2024-03-26T08:40:03.796 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.797 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.797 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.797 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "2419200", 2024-03-26T08:40:03.797 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.797 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.797 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.797 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-26T08:40:03.797 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.797 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.797 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.797 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": { 2024-03-26T08:40:03.797 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pool_name", 2024-03-26T08:40:03.797 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.798 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.798 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.798 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "device_health_metrics", 2024-03-26T08:40:03.798 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.798 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.798 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.798 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-26T08:40:03.798 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.798 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.798 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.798 INFO:teuthology.orchestra.run.smithi043.stdout: "retention_period": { 2024-03-26T08:40:03.798 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "retention_period", 2024-03-26T08:40:03.798 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.799 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.799 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.799 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15552000", 2024-03-26T08:40:03.799 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.799 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.799 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.799 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long to retain device health metrics", 2024-03-26T08:40:03.799 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.799 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.799 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.799 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_frequency": { 2024-03-26T08:40:03.799 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_frequency", 2024-03-26T08:40:03.800 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.800 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.800 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.800 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-03-26T08:40:03.800 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.800 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.800 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.800 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-26T08:40:03.800 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.800 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.800 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.800 INFO:teuthology.orchestra.run.smithi043.stdout: "self_heal": { 2024-03-26T08:40:03.800 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "self_heal", 2024-03-26T08:40:03.800 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.801 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.801 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.801 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.801 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.801 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.801 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.801 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-26T08:40:03.801 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.801 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.801 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.801 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-03-26T08:40:03.801 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-03-26T08:40:03.801 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.801 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.802 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.802 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-03-26T08:40:03.802 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.802 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.802 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.802 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how frequently to wake up and check device health", 2024-03-26T08:40:03.802 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.802 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.802 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.802 INFO:teuthology.orchestra.run.smithi043.stdout: "warn_threshold": { 2024-03-26T08:40:03.802 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "warn_threshold", 2024-03-26T08:40:03.802 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7257600", 2024-03-26T08:40:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-26T08:40:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.803 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "diskprediction_local", 2024-03-26T08:40:03.804 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.804 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.804 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.804 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.804 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.804 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.804 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.804 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.804 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.804 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.804 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.804 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.804 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.804 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.805 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.806 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.806 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.806 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.806 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.806 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.806 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.806 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.806 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.806 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.806 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.806 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.806 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.806 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.807 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.807 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.807 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.807 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.807 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.807 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.807 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.807 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.807 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.807 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.807 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.807 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.807 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.807 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.807 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.808 INFO:teuthology.orchestra.run.smithi043.stdout: "predict_interval": { 2024-03-26T08:40:03.809 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predict_interval", 2024-03-26T08:40:03.809 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.809 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.809 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.809 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "86400", 2024-03-26T08:40:03.809 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.809 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.809 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.809 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.809 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.809 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.809 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.809 INFO:teuthology.orchestra.run.smithi043.stdout: "predictor_model": { 2024-03-26T08:40:03.810 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "predictor_model", 2024-03-26T08:40:03.810 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.810 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.810 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.810 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "prophetstor", 2024-03-26T08:40:03.810 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.810 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.810 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.810 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.810 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.810 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.811 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.811 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-03-26T08:40:03.811 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-03-26T08:40:03.811 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.811 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.811 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.811 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "600", 2024-03-26T08:40:03.811 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.811 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.811 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.811 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.811 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.811 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.812 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.812 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.812 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.812 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "influx", 2024-03-26T08:40:03.812 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": false, 2024-03-26T08:40:03.812 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "influxdb python module not found", 2024-03-26T08:40:03.812 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.812 INFO:teuthology.orchestra.run.smithi043.stdout: "batch_size": { 2024-03-26T08:40:03.812 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "batch_size", 2024-03-26T08:40:03.812 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.812 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.812 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.813 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5000", 2024-03-26T08:40:03.813 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.813 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.813 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.813 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-26T08:40:03.813 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.813 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.813 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.813 INFO:teuthology.orchestra.run.smithi043.stdout: "database": { 2024-03-26T08:40:03.813 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "database", 2024-03-26T08:40:03.813 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.813 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.813 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.814 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "ceph", 2024-03-26T08:40:03.814 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.814 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.814 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.814 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-26T08:40:03.814 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.814 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.814 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.814 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": { 2024-03-26T08:40:03.814 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "hostname", 2024-03-26T08:40:03.814 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.814 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.814 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.815 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.815 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.815 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.815 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.815 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server hostname", 2024-03-26T08:40:03.815 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.815 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.815 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.815 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-03-26T08:40:03.815 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-03-26T08:40:03.815 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.815 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.815 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.816 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "30", 2024-03-26T08:40:03.816 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "5", 2024-03-26T08:40:03.816 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.816 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.816 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-26T08:40:03.816 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.816 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.816 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.816 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.816 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.816 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.816 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.816 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.816 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.817 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.817 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.817 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.817 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.817 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.817 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.817 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.817 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.817 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.817 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.817 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.817 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.817 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.817 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.818 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.818 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.818 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.818 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.818 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.818 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.818 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.818 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.818 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.818 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.818 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.818 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.818 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.819 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.819 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.819 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.819 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.819 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.819 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.819 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.819 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.819 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.819 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.819 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.819 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.819 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.819 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.819 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.820 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.820 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.820 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.820 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.820 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.820 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.820 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.820 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.820 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.820 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.820 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.820 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.820 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.820 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.821 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.821 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.821 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.821 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.821 INFO:teuthology.orchestra.run.smithi043.stdout: "password": { 2024-03-26T08:40:03.821 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "password", 2024-03-26T08:40:03.821 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.821 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.821 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.821 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.821 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.821 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.821 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.821 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "password of InfluxDB server user", 2024-03-26T08:40:03.822 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.822 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.822 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.822 INFO:teuthology.orchestra.run.smithi043.stdout: "port": { 2024-03-26T08:40:03.822 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "port", 2024-03-26T08:40:03.822 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.822 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.822 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.822 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "8086", 2024-03-26T08:40:03.822 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.822 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.822 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.822 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "InfluxDB server port", 2024-03-26T08:40:03.822 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.823 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.823 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.823 INFO:teuthology.orchestra.run.smithi043.stdout: "ssl": { 2024-03-26T08:40:03.823 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ssl", 2024-03-26T08:40:03.823 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.823 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.823 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.823 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "false", 2024-03-26T08:40:03.823 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.823 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.823 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.823 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T08:40:03.823 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.823 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.824 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.824 INFO:teuthology.orchestra.run.smithi043.stdout: "threads": { 2024-03-26T08:40:03.824 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threads", 2024-03-26T08:40:03.824 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.824 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.824 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.824 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-03-26T08:40:03.824 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-03-26T08:40:03.824 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "32", 2024-03-26T08:40:03.824 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.824 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-26T08:40:03.824 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.824 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.824 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.825 INFO:teuthology.orchestra.run.smithi043.stdout: "username": { 2024-03-26T08:40:03.825 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "username", 2024-03-26T08:40:03.825 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.825 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.825 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.825 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.825 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.825 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.825 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.825 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "username of InfluxDB server user", 2024-03-26T08:40:03.825 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.825 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.825 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.825 INFO:teuthology.orchestra.run.smithi043.stdout: "verify_ssl": { 2024-03-26T08:40:03.826 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "verify_ssl", 2024-03-26T08:40:03.826 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.826 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.826 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.826 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "true", 2024-03-26T08:40:03.826 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.826 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.826 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.826 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T08:40:03.826 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.826 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.826 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.826 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.827 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.827 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "insights", 2024-03-26T08:40:03.827 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.827 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.827 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.827 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.827 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.827 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.827 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.827 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.827 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.827 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.827 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.828 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.828 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.828 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.828 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.828 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.828 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.828 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.828 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.828 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.828 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.828 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.828 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.828 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.828 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.829 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.829 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.829 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.829 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.829 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.829 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.829 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.829 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.829 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.829 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.829 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.829 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.829 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.829 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.830 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.830 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.830 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.830 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.830 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.830 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.830 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.830 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.830 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.830 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.830 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.830 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.830 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.830 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.830 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.831 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.831 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.831 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.831 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.831 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.831 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.831 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.831 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.831 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.831 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.831 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.831 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.831 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.831 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.832 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.832 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.832 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.832 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "iostat", 2024-03-26T08:40:03.832 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.832 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.832 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.832 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.832 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.832 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.832 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.832 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.832 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.833 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.833 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.833 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.833 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.833 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.833 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.833 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.833 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.833 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.833 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.833 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.833 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.833 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.833 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.834 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.834 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.834 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.834 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.834 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.834 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.834 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.834 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.834 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.834 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.834 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.834 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.834 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.835 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.835 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.835 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.835 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.835 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.835 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.835 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.835 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.835 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.835 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.835 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.835 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.835 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.835 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.835 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.836 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.836 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.836 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.836 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.836 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.836 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.836 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.836 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.836 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.836 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.836 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.836 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.836 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.836 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.836 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.837 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.837 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.837 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.837 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.837 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.837 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "k8sevents", 2024-03-26T08:40:03.837 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.837 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.837 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.837 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_event_retention_days": { 2024-03-26T08:40:03.837 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "ceph_event_retention_days", 2024-03-26T08:40:03.837 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.837 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.838 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.838 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "7", 2024-03-26T08:40:03.838 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.838 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.838 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.838 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-26T08:40:03.838 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.838 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.838 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.838 INFO:teuthology.orchestra.run.smithi043.stdout: "config_check_secs": { 2024-03-26T08:40:03.838 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "config_check_secs", 2024-03-26T08:40:03.838 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.838 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.838 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.839 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-03-26T08:40:03.839 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "10", 2024-03-26T08:40:03.839 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.839 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.839 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-26T08:40:03.839 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.839 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.839 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.839 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.839 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.839 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.839 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.839 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.839 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.840 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.840 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.840 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.840 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.840 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.840 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.840 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.840 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.840 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.840 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.840 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.840 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.840 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.840 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.841 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.841 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.841 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.841 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.841 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.841 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.841 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.841 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.841 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.841 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.841 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.841 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.841 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.842 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.842 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.842 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.842 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.842 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.842 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.842 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.842 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.842 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.842 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.842 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.842 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.843 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.843 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.843 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.843 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.843 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.843 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.843 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.843 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.843 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.843 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.843 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.843 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.843 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.843 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.843 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.844 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.844 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.844 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.844 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.844 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.844 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.844 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.844 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.844 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "localpool", 2024-03-26T08:40:03.844 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.844 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.844 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.844 INFO:teuthology.orchestra.run.smithi043.stdout: "failure_domain": { 2024-03-26T08:40:03.844 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "failure_domain", 2024-03-26T08:40:03.844 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.845 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.845 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.845 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "host", 2024-03-26T08:40:03.845 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.845 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.845 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.845 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "failure domain for any created local pool", 2024-03-26T08:40:03.845 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-26T08:40:03.845 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.845 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.845 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.845 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.845 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.845 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.845 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.846 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.846 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.846 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.846 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.846 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.846 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.846 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.846 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.846 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.846 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.846 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.846 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.846 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.846 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.847 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.847 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.847 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.847 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.847 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.847 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.847 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.847 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.847 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.847 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.847 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.847 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.847 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.847 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.847 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.848 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.848 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.848 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.848 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.848 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.848 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.848 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.848 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.848 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.848 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.848 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.848 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.848 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.848 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.849 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.849 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.849 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.849 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.849 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.849 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.849 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.849 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.849 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.849 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.849 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.849 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.849 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.849 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.850 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.850 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.850 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.850 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.850 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": { 2024-03-26T08:40:03.850 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "min_size", 2024-03-26T08:40:03.850 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.850 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.850 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.850 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.850 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.850 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.851 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.851 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default min_size for any created local pool", 2024-03-26T08:40:03.851 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-26T08:40:03.851 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.851 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.851 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rep": { 2024-03-26T08:40:03.851 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "num_rep", 2024-03-26T08:40:03.851 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.851 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.851 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.851 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3", 2024-03-26T08:40:03.851 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.851 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.851 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.852 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default replica count for any created local pool", 2024-03-26T08:40:03.852 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.852 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.852 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.852 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": { 2024-03-26T08:40:03.852 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_num", 2024-03-26T08:40:03.852 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.852 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.852 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.852 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "128", 2024-03-26T08:40:03.852 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.852 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.852 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.852 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "default pg_num for any created local pool", 2024-03-26T08:40:03.853 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.853 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.853 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.853 INFO:teuthology.orchestra.run.smithi043.stdout: "prefix": { 2024-03-26T08:40:03.853 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prefix", 2024-03-26T08:40:03.853 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.853 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.853 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.853 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.853 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.853 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.853 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.853 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "name prefix for any created local pool", 2024-03-26T08:40:03.853 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.854 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.854 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.854 INFO:teuthology.orchestra.run.smithi043.stdout: "subtree": { 2024-03-26T08:40:03.854 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "subtree", 2024-03-26T08:40:03.854 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.854 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.854 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.854 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "rack", 2024-03-26T08:40:03.854 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.854 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.854 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.854 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-26T08:40:03.854 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-26T08:40:03.855 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.855 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.855 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.855 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.855 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mds_autoscaler", 2024-03-26T08:40:03.855 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.855 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.855 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.855 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.855 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.855 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.855 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.855 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.856 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.856 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.856 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.856 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.856 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.856 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.856 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.856 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.856 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.856 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.856 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.856 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.856 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.856 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.856 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.857 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.857 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.857 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.857 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.857 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.857 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.857 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.857 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.857 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.857 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.857 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.857 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.857 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.858 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.858 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.858 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.858 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.858 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.858 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.858 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.858 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.858 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.858 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.858 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.858 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.858 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.858 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.859 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.859 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.859 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.859 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.859 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.859 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.859 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.859 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.859 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.859 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.859 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.859 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.859 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.859 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.860 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.860 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.860 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.860 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.860 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.860 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.860 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.860 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirroring", 2024-03-26T08:40:03.860 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.860 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.860 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.860 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.860 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.861 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.861 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.861 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.861 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.861 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.861 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.861 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.861 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.861 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.861 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.861 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.861 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.861 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.861 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.861 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.862 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.862 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.862 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.862 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.862 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.862 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.862 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.862 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.862 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.862 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.862 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.862 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.862 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.863 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.863 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.863 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.863 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.863 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.863 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.863 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.863 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.863 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.863 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.863 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.863 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.863 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.863 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.864 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.864 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.864 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.864 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.864 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.864 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.864 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.864 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.864 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.864 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.864 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.864 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.864 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.864 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.864 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.865 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.865 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.865 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.865 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.865 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.865 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.865 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.865 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.865 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.865 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "nfs", 2024-03-26T08:40:03.865 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.865 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.865 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.866 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.866 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.866 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.866 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.866 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.866 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.866 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.866 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.866 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.866 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.866 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.866 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.866 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.866 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.867 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.868 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.868 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.868 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.868 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.868 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.868 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.868 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.868 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.868 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.868 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.868 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.868 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.868 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.868 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.869 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.869 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.869 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.869 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.869 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.869 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.869 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.869 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.869 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.869 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.869 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.869 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.869 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.869 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.870 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.870 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.870 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.870 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.870 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.870 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.870 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.870 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.870 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.870 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.870 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.870 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.871 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-03-26T08:40:03.871 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.871 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.871 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.871 INFO:teuthology.orchestra.run.smithi043.stdout: "fail_fs": { 2024-03-26T08:40:03.871 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "fail_fs", 2024-03-26T08:40:03.871 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.871 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.871 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.871 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.871 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.871 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.871 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.871 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-26T08:40:03.871 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.872 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.872 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.872 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.872 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.872 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.872 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.872 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.872 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.872 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.872 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.872 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.872 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.872 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.872 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.872 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.873 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.873 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.873 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.873 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.873 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.873 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.873 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.873 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.873 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.873 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.873 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.873 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.873 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.873 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.874 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.874 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.874 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.874 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.874 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.874 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.874 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.874 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.874 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.874 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.874 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.874 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.875 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.875 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.875 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.875 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.875 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.875 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.875 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.875 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.875 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.875 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.875 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.875 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.875 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.875 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.876 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.876 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.876 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.876 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.876 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.876 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.876 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.876 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.876 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.876 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.876 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.876 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.876 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.877 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator": { 2024-03-26T08:40:03.877 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "orchestrator", 2024-03-26T08:40:03.877 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.877 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.877 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.877 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.877 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.877 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.877 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.877 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm", 2024-03-26T08:40:03.877 INFO:teuthology.orchestra.run.smithi043.stdout: "rook", 2024-03-26T08:40:03.877 INFO:teuthology.orchestra.run.smithi043.stdout: "test_orchestrator" 2024-03-26T08:40:03.877 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.877 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Orchestrator backend", 2024-03-26T08:40:03.877 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.878 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.878 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.878 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.878 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.878 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_perf_query", 2024-03-26T08:40:03.878 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.878 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.878 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.878 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.878 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.878 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.878 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.879 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.879 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.879 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.879 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.879 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.879 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.879 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.879 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.879 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.879 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.879 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.879 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.879 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.879 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.880 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.880 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.880 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.880 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.880 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.880 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.880 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.880 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.880 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.880 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.880 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.880 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.880 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.881 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.881 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.881 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.881 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.881 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.881 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.881 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.881 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.881 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.881 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.881 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.881 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.881 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.881 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.882 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.882 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.882 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.882 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.882 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.882 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.882 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.882 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.882 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.882 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.882 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.882 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.882 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.883 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.883 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.883 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.883 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.883 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.883 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.883 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.883 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.883 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.883 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.883 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "osd_support", 2024-03-26T08:40:03.883 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.884 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.884 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.884 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.884 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.884 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.884 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.884 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.884 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.884 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.884 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.884 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.884 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.884 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.884 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.884 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.884 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.885 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.885 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.885 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.885 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.885 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.885 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.885 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.885 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.885 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.885 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.885 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.885 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.885 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.885 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.886 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.886 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.886 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.886 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.886 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.886 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.886 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.886 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.886 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.886 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.886 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.886 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.886 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.886 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.887 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.887 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.887 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.887 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.887 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.887 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.887 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.887 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.887 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.887 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.887 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.887 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.887 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.887 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.888 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.888 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.888 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.888 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.888 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.888 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.888 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.888 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.888 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.888 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.888 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.888 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.888 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "pg_autoscaler", 2024-03-26T08:40:03.888 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.889 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.889 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.889 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.889 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.889 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.889 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.889 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.889 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.889 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.889 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.889 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.889 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.889 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.889 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.889 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.890 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.890 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.890 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.890 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.890 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.890 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.890 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.890 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.890 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.890 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.890 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.890 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.890 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.891 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.891 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.891 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.891 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.891 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.891 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.891 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.891 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.891 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.891 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.891 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.891 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.891 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.892 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.892 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.892 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.892 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.892 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.892 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.892 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.892 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.892 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.892 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.892 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.892 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.892 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.892 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.893 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.893 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.893 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.893 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.893 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.893 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.893 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.893 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.893 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.893 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.893 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.893 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.893 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.894 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-03-26T08:40:03.894 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-03-26T08:40:03.894 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.894 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.894 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.894 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-03-26T08:40:03.894 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.894 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.894 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.894 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.894 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.894 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.894 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.894 INFO:teuthology.orchestra.run.smithi043.stdout: "threshold": { 2024-03-26T08:40:03.895 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "threshold", 2024-03-26T08:40:03.895 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-03-26T08:40:03.895 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.895 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.895 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "3.0", 2024-03-26T08:40:03.895 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1.0", 2024-03-26T08:40:03.895 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.895 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.895 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "scaling threshold", 2024-03-26T08:40:03.895 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-26T08:40:03.895 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.895 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.895 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.895 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.896 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "progress", 2024-03-26T08:40:03.896 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.896 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.896 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.896 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_pg_recovery_event": { 2024-03-26T08:40:03.896 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_pg_recovery_event", 2024-03-26T08:40:03.896 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.896 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.896 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.896 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.896 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.896 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.896 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.897 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-26T08:40:03.897 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.897 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.897 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.897 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-03-26T08:40:03.897 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-03-26T08:40:03.897 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.897 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.897 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.897 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.897 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.897 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.897 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.897 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.898 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.898 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.898 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.898 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.898 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.898 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.898 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.898 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.898 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.898 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.898 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.898 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.898 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.899 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.899 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.899 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.899 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.899 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.899 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.899 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.899 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.899 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.899 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.899 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.899 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.899 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.900 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.900 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.900 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.900 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.900 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.900 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.900 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.900 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.900 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.900 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.900 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.900 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.900 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.900 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.901 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.901 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.901 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.901 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.901 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.901 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.901 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.901 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.901 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.901 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.901 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.901 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.901 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.901 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.901 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.902 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.902 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.902 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.902 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.902 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.902 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.902 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.902 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.902 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.902 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.902 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.902 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.902 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.903 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.903 INFO:teuthology.orchestra.run.smithi043.stdout: "max_completed_events": { 2024-03-26T08:40:03.903 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_completed_events", 2024-03-26T08:40:03.903 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.903 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.903 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.903 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "50", 2024-03-26T08:40:03.903 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.903 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.903 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.903 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "number of past completed events to remember", 2024-03-26T08:40:03.903 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.903 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.904 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.904 INFO:teuthology.orchestra.run.smithi043.stdout: "sleep_interval": { 2024-03-26T08:40:03.904 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "sleep_interval", 2024-03-26T08:40:03.904 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.904 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.904 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.904 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "5", 2024-03-26T08:40:03.904 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.904 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.904 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.904 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "how long the module is going to sleep", 2024-03-26T08:40:03.904 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.904 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.904 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.905 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.905 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.905 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "prometheus", 2024-03-26T08:40:03.905 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.905 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.905 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.905 INFO:teuthology.orchestra.run.smithi043.stdout: "cache": { 2024-03-26T08:40:03.905 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "cache", 2024-03-26T08:40:03.905 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.905 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.905 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.905 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.906 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.906 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.906 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.906 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.906 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.906 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.906 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.906 INFO:teuthology.orchestra.run.smithi043.stdout: "exclude_perf_counters": { 2024-03-26T08:40:03.906 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "exclude_perf_counters", 2024-03-26T08:40:03.906 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.906 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.906 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.906 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.907 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.907 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.907 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.907 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-26T08:40:03.907 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-26T08:40:03.907 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.907 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.907 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.907 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.907 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.907 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.907 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.908 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.908 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.908 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.908 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.908 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.908 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.908 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.908 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.908 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.908 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.908 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.908 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.908 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.908 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.908 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.909 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.909 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.909 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.909 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.909 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.909 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.909 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.909 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.909 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.909 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.909 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.909 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.909 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.910 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.910 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.910 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.910 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.910 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.910 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.910 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.910 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.910 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.910 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.910 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.910 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.910 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.910 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.911 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.911 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.911 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.911 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.911 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.911 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.911 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.911 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.911 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.911 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.911 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.911 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.911 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.912 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.912 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.912 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.912 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.912 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.912 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.912 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools": { 2024-03-26T08:40:03.912 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools", 2024-03-26T08:40:03.912 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.912 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.912 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.912 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.912 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.913 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.913 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.913 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.913 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.913 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.913 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.913 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-26T08:40:03.913 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-26T08:40:03.913 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.913 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.913 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.913 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "300", 2024-03-26T08:40:03.913 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.913 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.914 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.914 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.914 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.914 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.914 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.914 INFO:teuthology.orchestra.run.smithi043.stdout: "scrape_interval": { 2024-03-26T08:40:03.914 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "scrape_interval", 2024-03-26T08:40:03.914 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-03-26T08:40:03.914 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.914 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.914 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15.0", 2024-03-26T08:40:03.914 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.915 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.915 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.915 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.915 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.915 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.915 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.915 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-03-26T08:40:03.915 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-03-26T08:40:03.915 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.915 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.915 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.915 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "::", 2024-03-26T08:40:03.915 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.916 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.916 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.916 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-26T08:40:03.916 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.916 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.916 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.916 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-03-26T08:40:03.916 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-03-26T08:40:03.916 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.916 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.916 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.916 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "9283", 2024-03-26T08:40:03.916 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.916 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.917 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.917 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-26T08:40:03.917 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.917 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.917 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.917 INFO:teuthology.orchestra.run.smithi043.stdout: "stale_cache_strategy": { 2024-03-26T08:40:03.917 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stale_cache_strategy", 2024-03-26T08:40:03.917 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.917 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.917 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.917 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "log", 2024-03-26T08:40:03.917 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.917 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.918 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.918 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.918 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.918 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.918 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.918 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_behaviour": { 2024-03-26T08:40:03.918 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_behaviour", 2024-03-26T08:40:03.918 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.918 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.918 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.918 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "default", 2024-03-26T08:40:03.918 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.918 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.918 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: "default", 2024-03-26T08:40:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: "error" 2024-03-26T08:40:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: "standby_error_status_code": { 2024-03-26T08:40:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "standby_error_status_code", 2024-03-26T08:40:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "500", 2024-03-26T08:40:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "400", 2024-03-26T08:40:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "599", 2024-03-26T08:40:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_support", 2024-03-26T08:40:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.920 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.921 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.921 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.921 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.921 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.921 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.921 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.921 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.921 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.921 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.921 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.921 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.921 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.921 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.921 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.922 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.923 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.923 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.923 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.923 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.923 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.923 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.923 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.923 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.923 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.923 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.923 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.923 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.923 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.925 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.925 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.925 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.925 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.925 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.925 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.925 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.925 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.925 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.925 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.925 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.925 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.925 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_snap_create": { 2024-03-26T08:40:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_snap_create", 2024-03-26T08:40:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10", 2024-03-26T08:40:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.926 INFO:teuthology.orchestra.run.smithi043.stdout: "mirror_snapshot_schedule": { 2024-03-26T08:40:03.927 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mirror_snapshot_schedule", 2024-03-26T08:40:03.927 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.927 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.927 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.927 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.927 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.927 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.927 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.927 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.927 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.927 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.927 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.927 INFO:teuthology.orchestra.run.smithi043.stdout: "trash_purge_schedule": { 2024-03-26T08:40:03.928 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "trash_purge_schedule", 2024-03-26T08:40:03.928 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.928 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.928 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.928 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.928 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.928 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.928 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.928 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.928 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.928 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.928 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.928 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.928 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "restful", 2024-03-26T08:40:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_auth": { 2024-03-26T08:40:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enable_auth", 2024-03-26T08:40:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.929 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.930 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.930 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.930 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.930 INFO:teuthology.orchestra.run.smithi043.stdout: "key_file": { 2024-03-26T08:40:03.930 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "key_file", 2024-03-26T08:40:03.930 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.930 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.930 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.930 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.930 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.930 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.930 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.930 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.932 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.932 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.932 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.932 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.932 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.932 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.932 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.932 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.932 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.932 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.932 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.932 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.932 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.932 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.933 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.933 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.933 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.933 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.933 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.933 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.933 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.933 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.933 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.933 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.933 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.933 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.933 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.933 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.934 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.934 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.934 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.934 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.934 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.934 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.934 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.934 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.934 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.934 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.934 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.934 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.934 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.934 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.936 INFO:teuthology.orchestra.run.smithi043.stdout: "server_addr": { 2024-03-26T08:40:03.936 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_addr", 2024-03-26T08:40:03.936 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.936 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.936 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.936 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.936 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.936 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.936 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.936 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.936 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.936 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.936 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.936 INFO:teuthology.orchestra.run.smithi043.stdout: "server_port": { 2024-03-26T08:40:03.937 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "server_port", 2024-03-26T08:40:03.937 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.937 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.937 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.937 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.937 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.937 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.937 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.937 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.937 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.937 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.938 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.938 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.938 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.938 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rgw", 2024-03-26T08:40:03.938 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.938 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.939 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.939 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.939 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.939 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.939 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.939 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.939 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.939 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.940 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.940 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.940 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.940 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.940 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.940 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.940 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.940 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.941 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.941 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.941 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.941 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.941 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.941 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.942 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.942 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.942 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.942 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.942 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.942 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.942 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.945 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.945 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.945 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.945 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.945 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.946 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.946 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.946 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.946 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.946 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.946 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.946 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.946 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.946 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.946 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.946 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.946 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.947 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.947 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.947 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.947 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.947 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.947 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.947 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.947 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.948 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.948 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.948 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.948 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.948 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.948 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.948 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.948 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.948 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.948 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.948 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.948 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.948 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.948 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.949 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.949 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rook", 2024-03-26T08:40:03.949 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.949 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.949 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.949 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.949 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.949 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.949 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.949 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.949 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.949 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.950 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.950 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.950 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.950 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.950 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.950 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.950 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.950 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.950 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.950 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.950 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.950 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.950 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.951 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.952 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.952 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.952 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.952 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.952 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.952 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.952 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.952 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.952 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.952 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.952 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.952 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.952 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.952 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.953 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.953 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.953 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.953 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.953 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.953 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.953 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.953 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.953 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.953 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.953 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.953 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.953 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.953 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.954 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.954 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.954 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.954 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.954 INFO:teuthology.orchestra.run.smithi043.stdout: "storage_class": { 2024-03-26T08:40:03.954 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "storage_class", 2024-03-26T08:40:03.954 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.954 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.954 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.954 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "local", 2024-03-26T08:40:03.954 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.954 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.954 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.955 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-26T08:40:03.955 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.955 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.955 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.955 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.955 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.955 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "selftest", 2024-03-26T08:40:03.955 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.955 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.955 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.955 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.955 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.956 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.957 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.958 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.958 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.958 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.958 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.958 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.958 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.958 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.958 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.958 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.958 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.958 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.958 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.958 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.959 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.960 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.960 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.960 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.960 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.960 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.960 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.960 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.960 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.960 INFO:teuthology.orchestra.run.smithi043.stdout: "roption1": { 2024-03-26T08:40:03.960 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption1", 2024-03-26T08:40:03.960 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.960 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.960 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.960 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.961 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.961 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.961 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.961 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.961 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.961 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.961 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.961 INFO:teuthology.orchestra.run.smithi043.stdout: "roption2": { 2024-03-26T08:40:03.961 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "roption2", 2024-03-26T08:40:03.961 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.961 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.961 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.961 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "xyz", 2024-03-26T08:40:03.962 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.962 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.962 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.962 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.962 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.962 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.962 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.962 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption1": { 2024-03-26T08:40:03.962 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption1", 2024-03-26T08:40:03.962 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.962 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.962 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption2": { 2024-03-26T08:40:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption2", 2024-03-26T08:40:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.963 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.964 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.964 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.964 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.964 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.964 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.964 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.964 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.964 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption3": { 2024-03-26T08:40:03.964 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption3", 2024-03-26T08:40:03.964 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "float", 2024-03-26T08:40:03.964 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.964 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.964 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.965 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.965 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.965 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.965 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.965 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.965 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.965 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.965 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption4": { 2024-03-26T08:40:03.965 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption4", 2024-03-26T08:40:03.965 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.965 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.965 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.965 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.966 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.966 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.966 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.966 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.966 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.966 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.966 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.966 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption5": { 2024-03-26T08:40:03.966 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption5", 2024-03-26T08:40:03.966 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.966 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.966 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.966 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.967 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.967 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.967 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.967 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.967 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.967 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.967 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.967 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption6": { 2024-03-26T08:40:03.967 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption6", 2024-03-26T08:40:03.967 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.967 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.967 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.967 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:03.968 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.968 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.968 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.968 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.968 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.968 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.968 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.968 INFO:teuthology.orchestra.run.smithi043.stdout: "rwoption7": { 2024-03-26T08:40:03.968 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rwoption7", 2024-03-26T08:40:03.968 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:03.968 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.968 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.969 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.969 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "1", 2024-03-26T08:40:03.969 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "42", 2024-03-26T08:40:03.969 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.969 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.969 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.969 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.969 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.969 INFO:teuthology.orchestra.run.smithi043.stdout: "testkey": { 2024-03-26T08:40:03.969 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testkey", 2024-03-26T08:40:03.969 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.970 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.970 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.970 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.970 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.970 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.970 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.970 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.970 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.970 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.970 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.970 INFO:teuthology.orchestra.run.smithi043.stdout: "testlkey": { 2024-03-26T08:40:03.971 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testlkey", 2024-03-26T08:40:03.971 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.971 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.971 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.971 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.971 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.971 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.971 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.971 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.971 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.971 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.971 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.972 INFO:teuthology.orchestra.run.smithi043.stdout: "testnewline": { 2024-03-26T08:40:03.972 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "testnewline", 2024-03-26T08:40:03.972 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.972 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.972 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.972 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.972 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.972 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.972 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.972 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.972 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.972 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.972 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.973 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.973 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.973 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snap_schedule", 2024-03-26T08:40:03.973 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.973 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.973 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.973 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_m_granularity": { 2024-03-26T08:40:03.973 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "allow_m_granularity", 2024-03-26T08:40:03.973 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.973 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.973 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.973 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.974 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.974 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.974 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.974 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "allow minute scheduled snapshots", 2024-03-26T08:40:03.974 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.974 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.974 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.974 INFO:teuthology.orchestra.run.smithi043.stdout: "dump_on_update": { 2024-03-26T08:40:03.974 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "dump_on_update", 2024-03-26T08:40:03.974 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.974 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.974 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.975 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.975 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.975 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.975 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.975 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "dump database to debug log on update", 2024-03-26T08:40:03.975 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.975 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.975 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.975 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.975 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.975 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.975 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.975 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.976 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.976 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.976 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.976 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.976 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.976 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.976 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.976 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.976 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.976 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.976 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.977 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.977 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.977 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.977 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.977 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.977 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.977 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.977 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.977 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.977 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.977 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.977 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.978 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.978 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.978 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.978 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.978 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.978 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.978 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.978 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.978 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.978 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.978 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.979 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.979 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.979 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.979 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.979 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.979 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.979 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.979 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.979 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.979 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.979 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.979 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.979 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.979 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.980 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.980 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.980 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.980 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.980 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.980 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.980 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.980 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.980 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.980 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.980 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.980 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.980 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.981 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.981 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.981 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "stats", 2024-03-26T08:40:03.981 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.981 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.981 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.981 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.981 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.981 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.981 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.982 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.982 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.982 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.982 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.982 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.982 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.982 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.982 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.982 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.982 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.982 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.982 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.982 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.982 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.983 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.984 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.984 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.984 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.984 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.984 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.984 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.984 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.984 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.984 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.984 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.984 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.984 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.985 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.985 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.985 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.985 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.985 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.985 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.985 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.985 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.985 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.985 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.985 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.985 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.985 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "status", 2024-03-26T08:40:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.987 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.988 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.988 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.988 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.988 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.988 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.988 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.988 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.988 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.988 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.988 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.988 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.988 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.988 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.988 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.989 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.989 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.989 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.989 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.989 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.989 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.989 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.989 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.989 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.989 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.989 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.989 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.990 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.990 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.990 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.990 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.990 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.990 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.990 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.990 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.990 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.990 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.990 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.990 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.990 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.991 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.991 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.991 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.991 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.991 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.991 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.991 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.991 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.991 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.991 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.991 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.991 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.991 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.991 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.992 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.992 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.992 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.992 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.992 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.992 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telegraf", 2024-03-26T08:40:03.992 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:03.992 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:03.992 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:03.992 INFO:teuthology.orchestra.run.smithi043.stdout: "address": { 2024-03-26T08:40:03.992 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "address", 2024-03-26T08:40:03.992 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.993 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.993 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.993 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-26T08:40:03.993 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.993 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.993 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.993 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.993 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.993 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.993 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.993 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-03-26T08:40:03.993 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-03-26T08:40:03.993 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "15", 2024-03-26T08:40:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.994 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.995 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.996 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.996 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:03.996 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:03.996 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.996 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.996 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.996 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.996 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.996 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.996 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.996 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.996 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.996 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:03.997 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:03.998 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:03.998 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:03.998 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:03.998 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.998 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.998 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.998 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.998 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:03.998 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:03.998 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:03.998 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:03.998 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:03.999 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:03.999 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:03.999 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:03.999 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:03.999 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:03.999 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:03.999 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:03.999 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:03.999 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.999 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:03.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:03.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:03.999 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "telemetry", 2024-03-26T08:40:03.999 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:04.000 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:04.000 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:04.000 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_basic": { 2024-03-26T08:40:04.000 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_basic", 2024-03-26T08:40:04.000 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:04.000 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.000 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:04.000 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:04.000 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.000 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.000 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.000 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-26T08:40:04.000 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_crash": { 2024-03-26T08:40:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_crash", 2024-03-26T08:40:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-26T08:40:04.001 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_device": { 2024-03-26T08:40:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_device", 2024-03-26T08:40:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-26T08:40:04.003 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.003 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.003 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.003 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_ident": { 2024-03-26T08:40:04.003 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_ident", 2024-03-26T08:40:04.003 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:04.003 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.003 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:04.003 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:04.003 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.003 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.003 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.003 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-26T08:40:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: "channel_perf": { 2024-03-26T08:40:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "channel_perf", 2024-03-26T08:40:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-26T08:40:04.004 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.005 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.005 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.005 INFO:teuthology.orchestra.run.smithi043.stdout: "contact": { 2024-03-26T08:40:04.005 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "contact", 2024-03-26T08:40:04.005 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:04.005 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.005 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:04.005 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:04.005 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.005 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.005 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.005 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.005 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.005 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: "description": { 2024-03-26T08:40:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "description", 2024-03-26T08:40:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: "device_url": { 2024-03-26T08:40:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "device_url", 2024-03-26T08:40:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-26T08:40:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.008 INFO:teuthology.orchestra.run.smithi043.stdout: "enabled": { 2024-03-26T08:40:04.008 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "enabled", 2024-03-26T08:40:04.008 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:04.008 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.008 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:04.008 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:04.008 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.008 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.008 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.008 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.008 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.008 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.008 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.008 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-03-26T08:40:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-03-26T08:40:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "24", 2024-03-26T08:40:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "8", 2024-03-26T08:40:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: "last_opt_revision": { 2024-03-26T08:40:04.009 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "last_opt_revision", 2024-03-26T08:40:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "1", 2024-03-26T08:40:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard": { 2024-03-26T08:40:04.010 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard", 2024-03-26T08:40:04.011 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:04.011 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.011 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:04.011 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:04.011 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.011 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.011 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.011 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.011 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.011 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.011 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.011 INFO:teuthology.orchestra.run.smithi043.stdout: "leaderboard_description": { 2024-03-26T08:40:04.011 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "leaderboard_description", 2024-03-26T08:40:04.012 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:04.012 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.012 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:04.012 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:04.012 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.012 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.012 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.012 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.012 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.012 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.012 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.012 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:04.012 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:04.012 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:04.013 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.014 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:04.015 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:04.016 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:04.016 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:04.016 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:04.016 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:04.016 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:04.016 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:04.016 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.016 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.016 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.016 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.016 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:04.016 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:04.017 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:04.017 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.017 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:04.017 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:04.017 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.017 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.017 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.017 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.017 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.017 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.017 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.017 INFO:teuthology.orchestra.run.smithi043.stdout: "organization": { 2024-03-26T08:40:04.018 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "organization", 2024-03-26T08:40:04.018 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:04.018 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.018 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:04.018 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:04.018 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.018 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.018 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.018 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.018 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.018 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.018 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.018 INFO:teuthology.orchestra.run.smithi043.stdout: "proxy": { 2024-03-26T08:40:04.019 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "proxy", 2024-03-26T08:40:04.019 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:04.019 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.019 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:04.019 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:04.019 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.019 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.019 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.019 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.019 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.019 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.019 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.019 INFO:teuthology.orchestra.run.smithi043.stdout: "url": { 2024-03-26T08:40:04.019 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "url", 2024-03-26T08:40:04.020 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:04.020 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.020 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:04.020 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-26T08:40:04.020 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.020 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.020 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.020 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.020 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.020 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.020 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.020 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:04.020 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:04.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:04.021 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "test_orchestrator", 2024-03-26T08:40:04.021 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:04.021 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:04.021 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:04.021 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:04.021 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:04.021 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:04.021 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.021 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:04.021 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:04.021 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.021 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.021 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:04.021 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:04.021 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:04.022 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:04.022 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:04.022 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:04.022 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:04.022 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:04.022 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.022 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.022 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.022 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.022 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:04.022 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:04.022 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:04.022 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.023 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:04.023 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:04.023 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.023 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.023 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.023 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.023 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.023 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.023 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.023 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:04.023 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:04.023 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:04.023 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.023 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:04.024 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:04.024 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.024 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.024 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:04.024 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:04.024 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:04.024 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:04.024 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:04.024 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:04.024 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:04.024 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:04.024 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.024 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.024 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.024 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.025 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:04.025 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:04.025 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:04.025 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.025 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:04.025 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:04.025 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.025 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.025 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.025 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.025 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.025 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.025 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.025 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:04.026 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:04.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:04.026 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "volumes", 2024-03-26T08:40:04.026 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:04.026 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:04.026 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:04.026 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:04.026 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:04.026 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:04.026 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.026 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:04.026 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:04.026 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.027 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.027 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:04.027 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:04.027 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:04.027 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:04.027 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:04.027 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:04.027 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:04.027 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:04.027 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.027 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.027 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.027 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.028 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:04.028 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:04.028 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:04.028 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.028 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:04.028 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:04.028 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.028 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.028 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.028 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.028 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.028 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.028 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.028 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:04.029 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:04.029 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:04.029 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.029 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:04.029 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:04.029 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.029 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.029 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:04.029 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:04.029 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:04.029 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:04.029 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:04.029 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:04.029 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:04.029 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:04.030 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.030 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.030 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.030 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.030 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:04.030 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:04.030 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:04.030 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.030 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:04.030 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:04.030 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.030 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.030 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.031 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.031 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.031 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.031 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.031 INFO:teuthology.orchestra.run.smithi043.stdout: "max_concurrent_clones": { 2024-03-26T08:40:04.031 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "max_concurrent_clones", 2024-03-26T08:40:04.031 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:04.031 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.031 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:04.031 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "4", 2024-03-26T08:40:04.031 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.031 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.031 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.032 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-26T08:40:04.032 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.032 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.032 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.032 INFO:teuthology.orchestra.run.smithi043.stdout: "periodic_async_work": { 2024-03-26T08:40:04.032 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "periodic_async_work", 2024-03-26T08:40:04.032 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:04.032 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.032 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:04.032 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:04.032 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.032 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.032 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.033 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Periodically check for async work", 2024-03-26T08:40:04.033 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.033 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.033 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.033 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_delay": { 2024-03-26T08:40:04.033 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_delay", 2024-03-26T08:40:04.033 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:04.033 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.033 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:04.033 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "0", 2024-03-26T08:40:04.033 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.033 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.033 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.034 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-26T08:40:04.034 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.034 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.034 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.034 INFO:teuthology.orchestra.run.smithi043.stdout: "snapshot_clone_no_wait": { 2024-03-26T08:40:04.034 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "snapshot_clone_no_wait", 2024-03-26T08:40:04.034 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:04.034 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.034 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:04.034 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "True", 2024-03-26T08:40:04.034 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.034 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.034 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.034 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-26T08:40:04.035 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.035 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.035 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.035 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:04.035 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:04.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:04.035 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix", 2024-03-26T08:40:04.035 INFO:teuthology.orchestra.run.smithi043.stdout: "can_run": true, 2024-03-26T08:40:04.035 INFO:teuthology.orchestra.run.smithi043.stdout: "error_string": "", 2024-03-26T08:40:04.035 INFO:teuthology.orchestra.run.smithi043.stdout: "module_options": { 2024-03-26T08:40:04.035 INFO:teuthology.orchestra.run.smithi043.stdout: "discovery_interval": { 2024-03-26T08:40:04.035 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "discovery_interval", 2024-03-26T08:40:04.035 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "uint", 2024-03-26T08:40:04.036 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.036 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:04.036 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "100", 2024-03-26T08:40:04.036 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.036 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.036 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.036 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.036 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.036 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.036 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.036 INFO:teuthology.orchestra.run.smithi043.stdout: "identifier": { 2024-03-26T08:40:04.036 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "identifier", 2024-03-26T08:40:04.036 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:04.036 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.037 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:04.037 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:04.037 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.037 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.037 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.037 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.037 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.037 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.037 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.037 INFO:teuthology.orchestra.run.smithi043.stdout: "interval": { 2024-03-26T08:40:04.037 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "interval", 2024-03-26T08:40:04.037 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "secs", 2024-03-26T08:40:04.037 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.038 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:04.038 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "60", 2024-03-26T08:40:04.038 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.038 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.038 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.038 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.038 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.038 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.038 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.038 INFO:teuthology.orchestra.run.smithi043.stdout: "log_level": { 2024-03-26T08:40:04.038 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_level", 2024-03-26T08:40:04.038 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:04.038 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.038 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:04.039 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:04.039 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.039 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.039 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:04.039 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:04.039 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:04.039 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:04.039 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:04.039 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:04.039 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:04.039 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:04.039 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.039 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.039 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.039 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.040 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster": { 2024-03-26T08:40:04.040 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster", 2024-03-26T08:40:04.040 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:04.040 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.040 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:04.040 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:04.040 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.040 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.040 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.040 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.040 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.040 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.041 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.041 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_cluster_level": { 2024-03-26T08:40:04.041 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_cluster_level", 2024-03-26T08:40:04.041 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:04.041 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.041 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:04.041 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "info", 2024-03-26T08:40:04.041 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.041 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.041 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [ 2024-03-26T08:40:04.041 INFO:teuthology.orchestra.run.smithi043.stdout: "", 2024-03-26T08:40:04.041 INFO:teuthology.orchestra.run.smithi043.stdout: "critical", 2024-03-26T08:40:04.041 INFO:teuthology.orchestra.run.smithi043.stdout: "debug", 2024-03-26T08:40:04.042 INFO:teuthology.orchestra.run.smithi043.stdout: "error", 2024-03-26T08:40:04.042 INFO:teuthology.orchestra.run.smithi043.stdout: "info", 2024-03-26T08:40:04.042 INFO:teuthology.orchestra.run.smithi043.stdout: "warning" 2024-03-26T08:40:04.042 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:04.042 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.042 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.042 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.042 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.042 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": { 2024-03-26T08:40:04.042 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "log_to_file", 2024-03-26T08:40:04.042 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "bool", 2024-03-26T08:40:04.042 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.042 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-03-26T08:40:04.043 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "False", 2024-03-26T08:40:04.043 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.043 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.043 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.043 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.043 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.043 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.043 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.043 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_host": { 2024-03-26T08:40:04.043 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_host", 2024-03-26T08:40:04.043 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:04.043 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.043 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:04.043 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "", 2024-03-26T08:40:04.044 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.044 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.044 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.044 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.044 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.044 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.044 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.044 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_port": { 2024-03-26T08:40:04.044 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_port", 2024-03-26T08:40:04.044 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "int", 2024-03-26T08:40:04.044 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.044 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:04.044 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "10051", 2024-03-26T08:40:04.044 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.045 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.045 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.045 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.045 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.045 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.045 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.045 INFO:teuthology.orchestra.run.smithi043.stdout: "zabbix_sender": { 2024-03-26T08:40:04.045 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "zabbix_sender", 2024-03-26T08:40:04.045 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "str", 2024-03-26T08:40:04.045 INFO:teuthology.orchestra.run.smithi043.stdout: "level": "advanced", 2024-03-26T08:40:04.045 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 0, 2024-03-26T08:40:04.045 INFO:teuthology.orchestra.run.smithi043.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-26T08:40:04.045 INFO:teuthology.orchestra.run.smithi043.stdout: "min": "", 2024-03-26T08:40:04.046 INFO:teuthology.orchestra.run.smithi043.stdout: "max": "", 2024-03-26T08:40:04.046 INFO:teuthology.orchestra.run.smithi043.stdout: "enum_allowed": [], 2024-03-26T08:40:04.046 INFO:teuthology.orchestra.run.smithi043.stdout: "desc": "", 2024-03-26T08:40:04.046 INFO:teuthology.orchestra.run.smithi043.stdout: "long_desc": "", 2024-03-26T08:40:04.046 INFO:teuthology.orchestra.run.smithi043.stdout: "tags": [], 2024-03-26T08:40:04.046 INFO:teuthology.orchestra.run.smithi043.stdout: "see_also": [] 2024-03-26T08:40:04.046 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:04.046 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:04.046 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:04.046 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:04.046 INFO:teuthology.orchestra.run.smithi043.stdout: "services": { 2024-03-26T08:40:04.046 INFO:teuthology.orchestra.run.smithi043.stdout: "dashboard": "https://172.21.15.43:8443/" 2024-03-26T08:40:04.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.046 INFO:teuthology.orchestra.run.smithi043.stdout: "always_on_modules": { 2024-03-26T08:40:04.047 INFO:teuthology.orchestra.run.smithi043.stdout: "octopus": [ 2024-03-26T08:40:04.047 INFO:teuthology.orchestra.run.smithi043.stdout: "balancer", 2024-03-26T08:40:04.047 INFO:teuthology.orchestra.run.smithi043.stdout: "crash", 2024-03-26T08:40:04.047 INFO:teuthology.orchestra.run.smithi043.stdout: "devicehealth", 2024-03-26T08:40:04.047 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator", 2024-03-26T08:40:04.047 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscaler", 2024-03-26T08:40:04.047 INFO:teuthology.orchestra.run.smithi043.stdout: "progress", 2024-03-26T08:40:04.047 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_support", 2024-03-26T08:40:04.047 INFO:teuthology.orchestra.run.smithi043.stdout: "status", 2024-03-26T08:40:04.047 INFO:teuthology.orchestra.run.smithi043.stdout: "telemetry", 2024-03-26T08:40:04.047 INFO:teuthology.orchestra.run.smithi043.stdout: "volumes" 2024-03-26T08:40:04.047 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:04.047 INFO:teuthology.orchestra.run.smithi043.stdout: "pacific": [ 2024-03-26T08:40:04.047 INFO:teuthology.orchestra.run.smithi043.stdout: "balancer", 2024-03-26T08:40:04.048 INFO:teuthology.orchestra.run.smithi043.stdout: "crash", 2024-03-26T08:40:04.048 INFO:teuthology.orchestra.run.smithi043.stdout: "devicehealth", 2024-03-26T08:40:04.048 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator", 2024-03-26T08:40:04.048 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscaler", 2024-03-26T08:40:04.048 INFO:teuthology.orchestra.run.smithi043.stdout: "progress", 2024-03-26T08:40:04.048 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_support", 2024-03-26T08:40:04.048 INFO:teuthology.orchestra.run.smithi043.stdout: "status", 2024-03-26T08:40:04.048 INFO:teuthology.orchestra.run.smithi043.stdout: "telemetry", 2024-03-26T08:40:04.048 INFO:teuthology.orchestra.run.smithi043.stdout: "volumes" 2024-03-26T08:40:04.048 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:04.048 INFO:teuthology.orchestra.run.smithi043.stdout: "quincy": [ 2024-03-26T08:40:04.048 INFO:teuthology.orchestra.run.smithi043.stdout: "balancer", 2024-03-26T08:40:04.048 INFO:teuthology.orchestra.run.smithi043.stdout: "crash", 2024-03-26T08:40:04.048 INFO:teuthology.orchestra.run.smithi043.stdout: "devicehealth", 2024-03-26T08:40:04.048 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator", 2024-03-26T08:40:04.049 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscaler", 2024-03-26T08:40:04.049 INFO:teuthology.orchestra.run.smithi043.stdout: "progress", 2024-03-26T08:40:04.049 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_support", 2024-03-26T08:40:04.049 INFO:teuthology.orchestra.run.smithi043.stdout: "status", 2024-03-26T08:40:04.049 INFO:teuthology.orchestra.run.smithi043.stdout: "telemetry", 2024-03-26T08:40:04.049 INFO:teuthology.orchestra.run.smithi043.stdout: "volumes" 2024-03-26T08:40:04.049 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:04.049 INFO:teuthology.orchestra.run.smithi043.stdout: "reef": [ 2024-03-26T08:40:04.049 INFO:teuthology.orchestra.run.smithi043.stdout: "balancer", 2024-03-26T08:40:04.049 INFO:teuthology.orchestra.run.smithi043.stdout: "crash", 2024-03-26T08:40:04.049 INFO:teuthology.orchestra.run.smithi043.stdout: "devicehealth", 2024-03-26T08:40:04.049 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator", 2024-03-26T08:40:04.049 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscaler", 2024-03-26T08:40:04.049 INFO:teuthology.orchestra.run.smithi043.stdout: "progress", 2024-03-26T08:40:04.049 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_support", 2024-03-26T08:40:04.050 INFO:teuthology.orchestra.run.smithi043.stdout: "status", 2024-03-26T08:40:04.050 INFO:teuthology.orchestra.run.smithi043.stdout: "telemetry", 2024-03-26T08:40:04.050 INFO:teuthology.orchestra.run.smithi043.stdout: "volumes" 2024-03-26T08:40:04.050 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-26T08:40:04.050 INFO:teuthology.orchestra.run.smithi043.stdout: "squid": [ 2024-03-26T08:40:04.050 INFO:teuthology.orchestra.run.smithi043.stdout: "balancer", 2024-03-26T08:40:04.050 INFO:teuthology.orchestra.run.smithi043.stdout: "crash", 2024-03-26T08:40:04.050 INFO:teuthology.orchestra.run.smithi043.stdout: "devicehealth", 2024-03-26T08:40:04.050 INFO:teuthology.orchestra.run.smithi043.stdout: "orchestrator", 2024-03-26T08:40:04.050 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscaler", 2024-03-26T08:40:04.050 INFO:teuthology.orchestra.run.smithi043.stdout: "progress", 2024-03-26T08:40:04.050 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_support", 2024-03-26T08:40:04.050 INFO:teuthology.orchestra.run.smithi043.stdout: "status", 2024-03-26T08:40:04.050 INFO:teuthology.orchestra.run.smithi043.stdout: "telemetry", 2024-03-26T08:40:04.051 INFO:teuthology.orchestra.run.smithi043.stdout: "volumes" 2024-03-26T08:40:04.051 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-03-26T08:40:04.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.051 INFO:teuthology.orchestra.run.smithi043.stdout: "last_failure_osd_epoch": 3, 2024-03-26T08:40:04.051 INFO:teuthology.orchestra.run.smithi043.stdout: "active_clients": [ 2024-03-26T08:40:04.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:04.051 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "devicehealth", 2024-03-26T08:40:04.051 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-03-26T08:40:04.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:04.051 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-03-26T08:40:04.051 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-03-26T08:40:04.051 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1151707060 2024-03-26T08:40:04.051 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:04.051 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-03-26T08:40:04.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:04.052 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "libcephsqlite", 2024-03-26T08:40:04.052 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-03-26T08:40:04.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:04.052 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-03-26T08:40:04.052 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-03-26T08:40:04.052 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1743507119 2024-03-26T08:40:04.052 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:04.052 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-03-26T08:40:04.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:04.052 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "rbd_support", 2024-03-26T08:40:04.052 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-03-26T08:40:04.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:04.052 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-03-26T08:40:04.053 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-03-26T08:40:04.053 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3641175829 2024-03-26T08:40:04.053 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:04.053 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-03-26T08:40:04.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-26T08:40:04.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:04.053 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "volumes", 2024-03-26T08:40:04.053 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-03-26T08:40:04.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-26T08:40:04.053 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-03-26T08:40:04.053 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:0", 2024-03-26T08:40:04.053 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2978127728 2024-03-26T08:40:04.053 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:04.053 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-03-26T08:40:04.054 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-26T08:40:04.054 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-03-26T08:40:04.054 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-26T08:40:04.055 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:02 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3961070671' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T08:40:04.055 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:02 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2596703184' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-26T08:40:04.055 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:02 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3961070671' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T08:40:04.055 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:02 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2596703184' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-26T08:40:04.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.789+0000 7fa389e00640 1 -- 172.21.15.43:0/2596703184 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa36c0a2800 msgr2=0x7fa36c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:04.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.789+0000 7fa389e00640 1 --2- 172.21.15.43:0/2596703184 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa36c0a2800 0x7fa36c0a4cc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa38419eb70 tx=0x7fa378005e90 comp rx=0 tx=0).stop 2024-03-26T08:40:04.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.789+0000 7fa389e00640 1 -- 172.21.15.43:0/2596703184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3841096b0 msgr2=0x7fa3841a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:04.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.789+0000 7fa389e00640 1 --2- 172.21.15.43:0/2596703184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3841096b0 0x7fa3841a5240 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa37c032520 tx=0x7fa37c034880 comp rx=0 tx=0).stop 2024-03-26T08:40:04.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.789+0000 7fa389e00640 1 -- 172.21.15.43:0/2596703184 shutdown_connections 2024-03-26T08:40:04.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.789+0000 7fa389e00640 1 --2- 172.21.15.43:0/2596703184 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa36c0a2800 0x7fa36c0a4cc0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:04.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.789+0000 7fa389e00640 1 --2- 172.21.15.43:0/2596703184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3841096b0 0x7fa3841a5240 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:04.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.789+0000 7fa389e00640 1 --2- 172.21.15.43:0/2596703184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa384108cf0 0x7fa38419e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:04.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.789+0000 7fa389e00640 1 --2- 172.21.15.43:0/2596703184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa384107b00 0x7fa38419dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:04.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.789+0000 7fa389e00640 1 -- 172.21.15.43:0/2596703184 >> 172.21.15.43:0/2596703184 conn(0x7fa384076ab0 msgr2=0x7fa38410c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:04.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.789+0000 7fa389e00640 1 -- 172.21.15.43:0/2596703184 shutdown_connections 2024-03-26T08:40:04.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:02.789+0000 7fa389e00640 1 -- 172.21.15.43:0/2596703184 wait complete. 2024-03-26T08:40:04.063 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-26T08:40:04.073 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-26T08:40:04.073 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-26T08:40:04.073 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-26T08:40:04.073 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:02 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3961070671' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T08:40:04.074 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:02 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2596703184' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-26T08:40:04.135 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-26T08:40:04.136 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-26T08:40:04.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:03 smithi043 ceph-mon[31929]: pgmap v637: 129 pgs: 129 active+clean; 681 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.2 KiB/s wr, 202 op/s 2024-03-26T08:40:04.191 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:40:04.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:03 smithi183 ceph-mon[35818]: pgmap v637: 129 pgs: 129 active+clean; 681 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.2 KiB/s wr, 202 op/s 2024-03-26T08:40:04.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:03 smithi052 ceph-mon[35071]: pgmap v637: 129 pgs: 129 active+clean; 681 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.2 KiB/s wr, 202 op/s 2024-03-26T08:40:04.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.471+0000 7f5459800640 1 -- 172.21.15.43:0/1200311366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5454105490 msgr2=0x7f545410bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:04.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.471+0000 7f5459800640 1 --2- 172.21.15.43:0/1200311366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5454105490 0x7f545410bd20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f544c009ca0 tx=0x7f544c02c100 comp rx=0 tx=0).stop 2024-03-26T08:40:04.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.471+0000 7f5459800640 1 -- 172.21.15.43:0/1200311366 shutdown_connections 2024-03-26T08:40:04.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.471+0000 7f5459800640 1 --2- 172.21.15.43:0/1200311366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5454105490 0x7f545410bd20 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:04.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.471+0000 7f5459800640 1 --2- 172.21.15.43:0/1200311366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5454104ad0 0x7f5454104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:04.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.471+0000 7f5459800640 1 --2- 172.21.15.43:0/1200311366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54541038e0 0x7f5454103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:04.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.471+0000 7f5459800640 1 -- 172.21.15.43:0/1200311366 >> 172.21.15.43:0/1200311366 conn(0x7f54540fee50 msgr2=0x7f5454101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:04.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.471+0000 7f5459800640 1 -- 172.21.15.43:0/1200311366 shutdown_connections 2024-03-26T08:40:04.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.471+0000 7f5459800640 1 -- 172.21.15.43:0/1200311366 wait complete. 2024-03-26T08:40:04.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.472+0000 7f5459800640 1 Processor -- start 2024-03-26T08:40:04.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.472+0000 7f5459800640 1 -- start start 2024-03-26T08:40:04.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.472+0000 7f5459800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54541038e0 0x7f54541999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:04.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.472+0000 7f5459800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5454104ad0 0x7f5454199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:04.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.472+0000 7f5459800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5454105490 0x7f54541a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:04.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.472+0000 7f5459800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545410f3f0 con 0x7f5454105490 2024-03-26T08:40:04.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.472+0000 7f5459800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545410f270 con 0x7f54541038e0 2024-03-26T08:40:04.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.472+0000 7f5459800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545410f570 con 0x7f5454104ad0 2024-03-26T08:40:04.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.473+0000 7f5453400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5454105490 0x7f54541a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:04.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.473+0000 7f5453400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5454105490 0x7f54541a0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39558/0 (socket says 172.21.15.43:39558) 2024-03-26T08:40:04.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.473+0000 7f5453400640 1 -- 172.21.15.43:0/1471423874 learned_addr learned my addr 172.21.15.43:0/1471423874 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:40:04.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.473+0000 7f5452a00640 1 --2- 172.21.15.43:0/1471423874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54541038e0 0x7f54541999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:04.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.473+0000 7f544be00640 1 --2- 172.21.15.43:0/1471423874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5454104ad0 0x7f5454199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:04.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.473+0000 7f5453400640 1 -- 172.21.15.43:0/1471423874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5454104ad0 msgr2=0x7f5454199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:04.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.473+0000 7f5453400640 1 --2- 172.21.15.43:0/1471423874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5454104ad0 0x7f5454199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:04.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.473+0000 7f5453400640 1 -- 172.21.15.43:0/1471423874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54541038e0 msgr2=0x7f54541999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:04.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.473+0000 7f5453400640 1 --2- 172.21.15.43:0/1471423874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54541038e0 0x7f54541999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:04.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.473+0000 7f5453400640 1 -- 172.21.15.43:0/1471423874 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54541a15c0 con 0x7f5454105490 2024-03-26T08:40:04.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.474+0000 7f5453400640 1 --2- 172.21.15.43:0/1471423874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5454105490 0x7f54541a0f50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f544c032380 tx=0x7f544c035e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:04.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.474+0000 7f544b400640 1 -- 172.21.15.43:0/1471423874 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c039040 con 0x7f5454105490 2024-03-26T08:40:04.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.474+0000 7f544b400640 1 -- 172.21.15.43:0/1471423874 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f544c044070 con 0x7f5454105490 2024-03-26T08:40:04.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.474+0000 7f544b400640 1 -- 172.21.15.43:0/1471423874 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c004670 con 0x7f5454105490 2024-03-26T08:40:04.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.474+0000 7f5459800640 1 -- 172.21.15.43:0/1471423874 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54541a1850 con 0x7f5454105490 2024-03-26T08:40:04.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.474+0000 7f5459800640 1 -- 172.21.15.43:0/1471423874 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54541a1cf0 con 0x7f5454105490 2024-03-26T08:40:04.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.475+0000 7f5459800640 1 -- 172.21.15.43:0/1471423874 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f545410aba0 con 0x7f5454105490 2024-03-26T08:40:04.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.477+0000 7f544b400640 1 -- 172.21.15.43:0/1471423874 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f544c003970 con 0x7f5454105490 2024-03-26T08:40:04.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.478+0000 7f544b400640 1 --2- 172.21.15.43:0/1471423874 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f54380a2800 0x7f54380a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:04.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.479+0000 7f544b400640 1 -- 172.21.15.43:0/1471423874 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f544c0f1850 con 0x7f5454105490 2024-03-26T08:40:04.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.479+0000 7f5452a00640 1 --2- 172.21.15.43:0/1471423874 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f54380a2800 0x7f54380a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:04.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.480+0000 7f5452a00640 1 --2- 172.21.15.43:0/1471423874 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f54380a2800 0x7f54380a4cc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f54440058c0 tx=0x7f5444005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:04.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.483+0000 7f544b400640 1 -- 172.21.15.43:0/1471423874 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f544c0be510 con 0x7f5454105490 2024-03-26T08:40:04.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.687+0000 7f5459800640 1 -- 172.21.15.43:0/1471423874 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f545402d290 con 0x7f5454105490 2024-03-26T08:40:04.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.691+0000 7f544b400640 1 -- 172.21.15.43:0/1471423874 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24670 (secure 0 0 0) 0x7f544c0098b0 con 0x7f5454105490 2024-03-26T08:40:04.683 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:40:04.683 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":94,"fsid":"88759420-eb49-11ee-95d0-87774f69a715","created":"2024-03-26T08:20:27.817107+0000","modified":"2024-03-26T08:32:38.256877+0000","last_up_change":"2024-03-26T08:30:10.520976+0000","last_in_change":"2024-03-26T08:29:46.394360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:24:54.852427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:31:19.924902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:31:21.323881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T08:31:24.951015+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"28670673-9193-4fa3-9cc0-c1a5a6b27284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6803","nonce":936601146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6805","nonce":936601146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6809","nonce":936601146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6807","nonce":936601146}]},"public_addr":"172.21.15.43:6803/936601146","cluster_addr":"172.21.15.43:6805/936601146","heartbeat_back_addr":"172.21.15.43:6809/936601146","heartbeat_front_addr":"172.21.15.43:6807/936601146","state":["exists","up"]},{"osd":1,"uuid":"fa8ad6b6-277a-42b8-b0f8-b36fc219de2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6801","nonce":3268412328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6803","nonce":3268412328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6807","nonce":3268412328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6805","nonce":3268412328}]},"public_addr":"172.21.15.52:6801/3268412328","cluster_addr":"172.21.15.52:6803/3268412328","heartbeat_back_addr":"172.21.15.52:6807/3268412328","heartbeat_front_addr":"172.21.15.52:6805/3268412328","state":["exists","up"]},{"osd":2,"uuid":"ec0039c3-f9bd-419a-81de-fa4418359808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6801","nonce":2103180908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6803","nonce":2103180908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6807","nonce":2103180908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6805","nonce":2103180908}]},"public_addr":"172.21.15.183:6801/2103180908","cluster_addr":"172.21.15.183:6803/2103180908","heartbeat_back_addr":"172.21.15.183:6807/2103180908","heartbeat_front_addr":"172.21.15.183:6805/2103180908","state":["exists","up"]},{"osd":3,"uuid":"e7dcbf00-6586-4664-a69e-3cb0df8dfd30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6811","nonce":3722846033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6813","nonce":3722846033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6817","nonce":3722846033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6815","nonce":3722846033}]},"public_addr":"172.21.15.43:6811/3722846033","cluster_addr":"172.21.15.43:6813/3722846033","heartbeat_back_addr":"172.21.15.43:6817/3722846033","heartbeat_front_addr":"172.21.15.43:6815/3722846033","state":["exists","up"]},{"osd":4,"uuid":"db089a49-5e98-432d-9687-cdc044114c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6809","nonce":4028436523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6811","nonce":4028436523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6815","nonce":4028436523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6813","nonce":4028436523}]},"public_addr":"172.21.15.52:6809/4028436523","cluster_addr":"172.21.15.52:6811/4028436523","heartbeat_back_addr":"172.21.15.52:6815/4028436523","heartbeat_front_addr":"172.21.15.52:6813/4028436523","state":["exists","up"]},{"osd":5,"uuid":"556ac804-02ea-423b-a599-55baf5be4ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6809","nonce":787132252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6811","nonce":787132252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6815","nonce":787132252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6813","nonce":787132252}]},"public_addr":"172.21.15.183:6809/787132252","cluster_addr":"172.21.15.183:6811/787132252","heartbeat_back_addr":"172.21.15.183:6815/787132252","heartbeat_front_addr":"172.21.15.183:6813/787132252","state":["exists","up"]},{"osd":6,"uuid":"3de2a49d-6cef-4580-9d8a-f90cdfcecd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6819","nonce":4206861158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6821","nonce":4206861158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6825","nonce":4206861158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6823","nonce":4206861158}]},"public_addr":"172.21.15.43:6819/4206861158","cluster_addr":"172.21.15.43:6821/4206861158","heartbeat_back_addr":"172.21.15.43:6825/4206861158","heartbeat_front_addr":"172.21.15.43:6823/4206861158","state":["exists","up"]},{"osd":7,"uuid":"a3b832f7-7d7d-4a62-b679-db6da98275d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6817","nonce":3452064146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6819","nonce":3452064146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6823","nonce":3452064146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6821","nonce":3452064146}]},"public_addr":"172.21.15.52:6817/3452064146","cluster_addr":"172.21.15.52:6819/3452064146","heartbeat_back_addr":"172.21.15.52:6823/3452064146","heartbeat_front_addr":"172.21.15.52:6821/3452064146","state":["exists","up"]},{"osd":8,"uuid":"ddc6b07f-d703-414d-a59a-02316376b13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6817","nonce":1023131725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6819","nonce":1023131725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6823","nonce":1023131725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6821","nonce":1023131725}]},"public_addr":"172.21.15.183:6817/1023131725","cluster_addr":"172.21.15.183:6819/1023131725","heartbeat_back_addr":"172.21.15.183:6823/1023131725","heartbeat_front_addr":"172.21.15.183:6821/1023131725","state":["exists","up"]},{"osd":9,"uuid":"ac16bb0e-6232-45c8-b16a-4d878874c588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6827","nonce":3935195657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6829","nonce":3935195657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6833","nonce":3935195657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6831","nonce":3935195657}]},"public_addr":"172.21.15.43:6827/3935195657","cluster_addr":"172.21.15.43:6829/3935195657","heartbeat_back_addr":"172.21.15.43:6833/3935195657","heartbeat_front_addr":"172.21.15.43:6831/3935195657","state":["exists","up"]},{"osd":10,"uuid":"fd839188-f398-4cff-aa7b-36522ff1a8a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6825","nonce":377628320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6827","nonce":377628320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6831","nonce":377628320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6829","nonce":377628320}]},"public_addr":"172.21.15.52:6825/377628320","cluster_addr":"172.21.15.52:6827/377628320","heartbeat_back_addr":"172.21.15.52:6831/377628320","heartbeat_front_addr":"172.21.15.52:6829/377628320","state":["exists","up"]},{"osd":11,"uuid":"a40282a0-0c64-49ee-b64e-6f3f8119e827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6825","nonce":1373086433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6827","nonce":1373086433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6831","nonce":1373086433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6829","nonce":1373086433}]},"public_addr":"172.21.15.183:6825/1373086433","cluster_addr":"172.21.15.183:6827/1373086433","heartbeat_back_addr":"172.21.15.183:6831/1373086433","heartbeat_front_addr":"172.21.15.183:6829/1373086433","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:23:43.478905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:17.573532+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:50.286610+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:25:27.407663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:03.081294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:36.057416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:12.062013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:47.032271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:21.023201+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:57.364739+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:29:34.039995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:30:08.827577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6801/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/4240775295":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/510203140":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/4251957687":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:6801/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1885244665":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/298320867":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1301092625":"2024-03-27T08:21:36.149092+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:40:04.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.694+0000 7f5459800640 1 -- 172.21.15.43:0/1471423874 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f54380a2800 msgr2=0x7f54380a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:04.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.694+0000 7f5459800640 1 --2- 172.21.15.43:0/1471423874 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f54380a2800 0x7f54380a4cc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f54440058c0 tx=0x7f5444005850 comp rx=0 tx=0).stop 2024-03-26T08:40:04.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.694+0000 7f5459800640 1 -- 172.21.15.43:0/1471423874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5454105490 msgr2=0x7f54541a0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:04.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.694+0000 7f5459800640 1 --2- 172.21.15.43:0/1471423874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5454105490 0x7f54541a0f50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f544c032380 tx=0x7f544c035e20 comp rx=0 tx=0).stop 2024-03-26T08:40:04.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.694+0000 7f5459800640 1 -- 172.21.15.43:0/1471423874 shutdown_connections 2024-03-26T08:40:04.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.694+0000 7f5459800640 1 --2- 172.21.15.43:0/1471423874 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f54380a2800 0x7f54380a4cc0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:04.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.694+0000 7f5459800640 1 --2- 172.21.15.43:0/1471423874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5454105490 0x7f54541a0f50 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:04.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.694+0000 7f5459800640 1 --2- 172.21.15.43:0/1471423874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5454104ad0 0x7f5454199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:04.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.694+0000 7f5459800640 1 --2- 172.21.15.43:0/1471423874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54541038e0 0x7f54541999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:04.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.694+0000 7f5459800640 1 -- 172.21.15.43:0/1471423874 >> 172.21.15.43:0/1471423874 conn(0x7f54540fee50 msgr2=0x7f5454107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:04.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.694+0000 7f5459800640 1 -- 172.21.15.43:0/1471423874 shutdown_connections 2024-03-26T08:40:04.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.694+0000 7f5459800640 1 -- 172.21.15.43:0/1471423874 wait complete. 2024-03-26T08:40:04.708 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T08:24:54.852427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T08:31:19.924902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T08:31:21.323881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T08:31:24.951015+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T08:40:04.708 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T08:40:04.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.954+0000 7fa6ab800640 1 -- 172.21.15.43:0/2683848401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6a4067c00 msgr2=0x7fa6a4100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:04.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.954+0000 7fa6ab800640 1 --2- 172.21.15.43:0/2683848401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6a4067c00 0x7fa6a4100ae0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa698009900 tx=0x7fa69802bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:04.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.955+0000 7fa6ab800640 1 -- 172.21.15.43:0/2683848401 shutdown_connections 2024-03-26T08:40:04.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.955+0000 7fa6ab800640 1 --2- 172.21.15.43:0/2683848401 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6a4105d20 0x7fa6a4108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:04.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.955+0000 7fa6ab800640 1 --2- 172.21.15.43:0/2683848401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6a4101020 0x7fa6a41014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:04.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.955+0000 7fa6ab800640 1 --2- 172.21.15.43:0/2683848401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6a4067c00 0x7fa6a4100ae0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:04.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.955+0000 7fa6ab800640 1 -- 172.21.15.43:0/2683848401 >> 172.21.15.43:0/2683848401 conn(0x7fa6a40fae60 msgr2=0x7fa6a40fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:04.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.955+0000 7fa6ab800640 1 -- 172.21.15.43:0/2683848401 shutdown_connections 2024-03-26T08:40:04.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.955+0000 7fa6ab800640 1 -- 172.21.15.43:0/2683848401 wait complete. 2024-03-26T08:40:04.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.956+0000 7fa6ab800640 1 Processor -- start 2024-03-26T08:40:04.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.956+0000 7fa6ab800640 1 -- start start 2024-03-26T08:40:04.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.956+0000 7fa6ab800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6a4067c00 0x7fa6a4199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:04.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.956+0000 7fa6ab800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6a4101020 0x7fa6a419a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:04.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.956+0000 7fa6ab800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6a4105d20 0x7fa6a41a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:04.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.956+0000 7fa6ab800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a410d1f0 con 0x7fa6a4101020 2024-03-26T08:40:04.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.956+0000 7fa6ab800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a410d070 con 0x7fa6a4067c00 2024-03-26T08:40:04.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.956+0000 7fa6ab800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a410d370 con 0x7fa6a4105d20 2024-03-26T08:40:04.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.957+0000 7fa6a8e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6a4067c00 0x7fa6a4199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:04.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.957+0000 7fa6a3e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6a4101020 0x7fa6a419a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:04.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.957+0000 7fa6a8e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6a4067c00 0x7fa6a4199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.43:47742/0 (socket says 172.21.15.43:47742) 2024-03-26T08:40:04.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.957+0000 7fa6a8e00640 1 -- 172.21.15.43:0/3731915231 learned_addr learned my addr 172.21.15.43:0/3731915231 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:40:04.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.957+0000 7fa6a9800640 1 --2- 172.21.15.43:0/3731915231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6a4105d20 0x7fa6a41a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:04.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.957+0000 7fa6a3e00640 1 -- 172.21.15.43:0/3731915231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6a4105d20 msgr2=0x7fa6a41a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:04.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.957+0000 7fa6a3e00640 1 --2- 172.21.15.43:0/3731915231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6a4105d20 0x7fa6a41a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:04.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.957+0000 7fa6a3e00640 1 -- 172.21.15.43:0/3731915231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6a4067c00 msgr2=0x7fa6a4199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:04.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.957+0000 7fa6a3e00640 1 --2- 172.21.15.43:0/3731915231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6a4067c00 0x7fa6a4199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:04.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.957+0000 7fa6a3e00640 1 -- 172.21.15.43:0/3731915231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a41a17e0 con 0x7fa6a4101020 2024-03-26T08:40:04.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.958+0000 7fa6a3e00640 1 --2- 172.21.15.43:0/3731915231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6a4101020 0x7fa6a419a100 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4068020 tx=0x7fa69000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:04.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.958+0000 7fa6a1600640 1 -- 172.21.15.43:0/3731915231 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa690017070 con 0x7fa6a4101020 2024-03-26T08:40:04.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.958+0000 7fa6a1600640 1 -- 172.21.15.43:0/3731915231 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa690009c30 con 0x7fa6a4101020 2024-03-26T08:40:04.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.958+0000 7fa6a1600640 1 -- 172.21.15.43:0/3731915231 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69000f620 con 0x7fa6a4101020 2024-03-26T08:40:04.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.958+0000 7fa6ab800640 1 -- 172.21.15.43:0/3731915231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a41a1ad0 con 0x7fa6a4101020 2024-03-26T08:40:04.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.958+0000 7fa6ab800640 1 -- 172.21.15.43:0/3731915231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a41a1fd0 con 0x7fa6a4101020 2024-03-26T08:40:04.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.959+0000 7fa6ab800640 1 -- 172.21.15.43:0/3731915231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6a41a2200 con 0x7fa6a4101020 2024-03-26T08:40:04.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.962+0000 7fa6a1600640 1 -- 172.21.15.43:0/3731915231 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa69001a070 con 0x7fa6a4101020 2024-03-26T08:40:04.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.963+0000 7fa6a1600640 1 --2- 172.21.15.43:0/3731915231 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa68c0a2800 0x7fa68c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:04.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.964+0000 7fa6a1600640 1 -- 172.21.15.43:0/3731915231 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7fa6900ce7a0 con 0x7fa6a4101020 2024-03-26T08:40:04.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.964+0000 7fa6a8e00640 1 --2- 172.21.15.43:0/3731915231 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa68c0a2800 0x7fa68c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:04.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.964+0000 7fa6a8e00640 1 --2- 172.21.15.43:0/3731915231 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa68c0a2800 0x7fa68c0a4cc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa69802c840 tx=0x7fa6980095a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:04.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:04.968+0000 7fa6a1600640 1 -- 172.21.15.43:0/3731915231 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa690021030 con 0x7fa6a4101020 2024-03-26T08:40:05.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:04 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1471423874' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:05.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.172+0000 7fa6ab800640 1 -- 172.21.15.43:0/3731915231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa6a41a2590 con 0x7fa6a4101020 2024-03-26T08:40:05.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.173+0000 7fa6a1600640 1 -- 172.21.15.43:0/3731915231 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa690009de0 con 0x7fa6a4101020 2024-03-26T08:40:05.165 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-26T08:40:05.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.176+0000 7fa6ab800640 1 -- 172.21.15.43:0/3731915231 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa68c0a2800 msgr2=0x7fa68c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:05.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.176+0000 7fa6ab800640 1 --2- 172.21.15.43:0/3731915231 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa68c0a2800 0x7fa68c0a4cc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa69802c840 tx=0x7fa6980095a0 comp rx=0 tx=0).stop 2024-03-26T08:40:05.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.176+0000 7fa6ab800640 1 -- 172.21.15.43:0/3731915231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6a4101020 msgr2=0x7fa6a419a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:05.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.176+0000 7fa6ab800640 1 --2- 172.21.15.43:0/3731915231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6a4101020 0x7fa6a419a100 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4068020 tx=0x7fa69000dde0 comp rx=0 tx=0).stop 2024-03-26T08:40:05.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.176+0000 7fa6ab800640 1 -- 172.21.15.43:0/3731915231 shutdown_connections 2024-03-26T08:40:05.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.176+0000 7fa6ab800640 1 --2- 172.21.15.43:0/3731915231 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa68c0a2800 0x7fa68c0a4cc0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:05.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.176+0000 7fa6ab800640 1 --2- 172.21.15.43:0/3731915231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6a4105d20 0x7fa6a41a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:05.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.176+0000 7fa6ab800640 1 --2- 172.21.15.43:0/3731915231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6a4101020 0x7fa6a419a100 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:05.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.176+0000 7fa6ab800640 1 --2- 172.21.15.43:0/3731915231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6a4067c00 0x7fa6a4199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:05.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.176+0000 7fa6ab800640 1 -- 172.21.15.43:0/3731915231 >> 172.21.15.43:0/3731915231 conn(0x7fa6a40fae60 msgr2=0x7fa6a40fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:05.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.176+0000 7fa6ab800640 1 -- 172.21.15.43:0/3731915231 shutdown_connections 2024-03-26T08:40:05.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.176+0000 7fa6ab800640 1 -- 172.21.15.43:0/3731915231 wait complete. 2024-03-26T08:40:05.189 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T08:40:05.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:04 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1471423874' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:05.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:04 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1471423874' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:05.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.429+0000 7f10afa00640 1 -- 172.21.15.43:0/209131172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10a8104ab0 msgr2=0x7f10a8104f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:05.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.429+0000 7f10afa00640 1 --2- 172.21.15.43:0/209131172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10a8104ab0 0x7f10a8104f30 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1094009900 tx=0x7f109402bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:05.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.429+0000 7f10afa00640 1 -- 172.21.15.43:0/209131172 shutdown_connections 2024-03-26T08:40:05.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.429+0000 7f10afa00640 1 --2- 172.21.15.43:0/209131172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a8105470 0x7f10a810bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:05.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.429+0000 7f10afa00640 1 --2- 172.21.15.43:0/209131172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10a8104ab0 0x7f10a8104f30 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:05.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.429+0000 7f10afa00640 1 --2- 172.21.15.43:0/209131172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10a81038c0 0x7f10a8103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:05.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.429+0000 7f10afa00640 1 -- 172.21.15.43:0/209131172 >> 172.21.15.43:0/209131172 conn(0x7f10a80fee50 msgr2=0x7f10a8101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:05.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.429+0000 7f10afa00640 1 -- 172.21.15.43:0/209131172 shutdown_connections 2024-03-26T08:40:05.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.430+0000 7f10afa00640 1 -- 172.21.15.43:0/209131172 wait complete. 2024-03-26T08:40:05.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.430+0000 7f10afa00640 1 Processor -- start 2024-03-26T08:40:05.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.431+0000 7f10afa00640 1 -- start start 2024-03-26T08:40:05.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.431+0000 7f10afa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a81038c0 0x7f10a8199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:05.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.431+0000 7f10afa00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10a8104ab0 0x7f10a819a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:05.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.431+0000 7f10afa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10a8105470 0x7f10a81a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:05.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.431+0000 7f10afa00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a810f440 con 0x7f10a8105470 2024-03-26T08:40:05.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.431+0000 7f10afa00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a810f2c0 con 0x7f10a8104ab0 2024-03-26T08:40:05.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.431+0000 7f10afa00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a810f5c0 con 0x7f10a81038c0 2024-03-26T08:40:05.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.431+0000 7f10ada00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10a8105470 0x7f10a81a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:05.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.431+0000 7f10ada00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10a8105470 0x7f10a81a10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39602/0 (socket says 172.21.15.43:39602) 2024-03-26T08:40:05.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.431+0000 7f10ada00640 1 -- 172.21.15.43:0/46407995 learned_addr learned my addr 172.21.15.43:0/46407995 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:40:05.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.431+0000 7f10ad000640 1 --2- 172.21.15.43:0/46407995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a81038c0 0x7f10a8199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:05.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.432+0000 7f10a7e00640 1 --2- 172.21.15.43:0/46407995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10a8104ab0 0x7f10a819a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:05.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.432+0000 7f10ada00640 1 -- 172.21.15.43:0/46407995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a81038c0 msgr2=0x7f10a8199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:05.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.432+0000 7f10ada00640 1 --2- 172.21.15.43:0/46407995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a81038c0 0x7f10a8199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:05.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.432+0000 7f10ada00640 1 -- 172.21.15.43:0/46407995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10a8104ab0 msgr2=0x7f10a819a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:05.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.432+0000 7f10ada00640 1 --2- 172.21.15.43:0/46407995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10a8104ab0 0x7f10a819a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:05.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.432+0000 7f10ada00640 1 -- 172.21.15.43:0/46407995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a81a1730 con 0x7f10a8105470 2024-03-26T08:40:05.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.432+0000 7f10ada00640 1 --2- 172.21.15.43:0/46407995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10a8105470 0x7f10a81a10c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f10a000cad0 tx=0x7f10a000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:05.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.432+0000 7f10a5600640 1 -- 172.21.15.43:0/46407995 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a000ae20 con 0x7f10a8105470 2024-03-26T08:40:05.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.432+0000 7f10a5600640 1 -- 172.21.15.43:0/46407995 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f10a0005020 con 0x7f10a8105470 2024-03-26T08:40:05.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.432+0000 7f10a5600640 1 -- 172.21.15.43:0/46407995 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a000f6b0 con 0x7f10a8105470 2024-03-26T08:40:05.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.432+0000 7f10afa00640 1 -- 172.21.15.43:0/46407995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a81a1a20 con 0x7f10a8105470 2024-03-26T08:40:05.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.432+0000 7f10afa00640 1 -- 172.21.15.43:0/46407995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a81a1fc0 con 0x7f10a8105470 2024-03-26T08:40:05.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.434+0000 7f10afa00640 1 -- 172.21.15.43:0/46407995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a810ad00 con 0x7f10a8105470 2024-03-26T08:40:05.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.436+0000 7f10a5600640 1 -- 172.21.15.43:0/46407995 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f10a00051d0 con 0x7f10a8105470 2024-03-26T08:40:05.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.438+0000 7f10a5600640 1 --2- 172.21.15.43:0/46407995 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f10900a2880 0x7f10900a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:05.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.438+0000 7f10a5600640 1 -- 172.21.15.43:0/46407995 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f10a00cee90 con 0x7f10a8105470 2024-03-26T08:40:05.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.438+0000 7f10ad000640 1 --2- 172.21.15.43:0/46407995 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f10900a2880 0x7f10900a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:05.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.439+0000 7f10ad000640 1 --2- 172.21.15.43:0/46407995 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f10900a2880 0x7f10900a4d40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f109c005db0 tx=0x7f109c005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:05.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.442+0000 7f10a5600640 1 -- 172.21.15.43:0/46407995 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f10a0014030 con 0x7f10a8105470 2024-03-26T08:40:05.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.642+0000 7f10afa00640 1 -- 172.21.15.43:0/46407995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f10a80622a0 con 0x7f10a8105470 2024-03-26T08:40:05.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.643+0000 7f10a5600640 1 -- 172.21.15.43:0/46407995 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f10a009bb50 con 0x7f10a8105470 2024-03-26T08:40:05.635 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-26T08:40:05.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.646+0000 7f10afa00640 1 -- 172.21.15.43:0/46407995 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f10900a2880 msgr2=0x7f10900a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:05.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.646+0000 7f10afa00640 1 --2- 172.21.15.43:0/46407995 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f10900a2880 0x7f10900a4d40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f109c005db0 tx=0x7f109c005d40 comp rx=0 tx=0).stop 2024-03-26T08:40:05.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.646+0000 7f10afa00640 1 -- 172.21.15.43:0/46407995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10a8105470 msgr2=0x7f10a81a10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:05.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.646+0000 7f10afa00640 1 --2- 172.21.15.43:0/46407995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10a8105470 0x7f10a81a10c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f10a000cad0 tx=0x7f10a000cf90 comp rx=0 tx=0).stop 2024-03-26T08:40:05.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.646+0000 7f10afa00640 1 -- 172.21.15.43:0/46407995 shutdown_connections 2024-03-26T08:40:05.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.646+0000 7f10afa00640 1 --2- 172.21.15.43:0/46407995 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f10900a2880 0x7f10900a4d40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:05.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.646+0000 7f10afa00640 1 --2- 172.21.15.43:0/46407995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10a8105470 0x7f10a81a10c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:05.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.646+0000 7f10afa00640 1 --2- 172.21.15.43:0/46407995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10a8104ab0 0x7f10a819a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:05.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.646+0000 7f10afa00640 1 --2- 172.21.15.43:0/46407995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a81038c0 0x7f10a8199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:05.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.646+0000 7f10afa00640 1 -- 172.21.15.43:0/46407995 >> 172.21.15.43:0/46407995 conn(0x7f10a80fee50 msgr2=0x7f10a8107ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:05.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.646+0000 7f10afa00640 1 -- 172.21.15.43:0/46407995 shutdown_connections 2024-03-26T08:40:05.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.646+0000 7f10afa00640 1 -- 172.21.15.43:0/46407995 wait complete. 2024-03-26T08:40:05.661 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T08:40:05.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.905+0000 7f3b11200640 1 -- 172.21.15.43:0/1512885664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b0c105490 msgr2=0x7f3b0c10bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:05.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.905+0000 7f3b11200640 1 --2- 172.21.15.43:0/1512885664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b0c105490 0x7f3b0c10bd20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009900 tx=0x7f3b0402bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:05.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.906+0000 7f3b11200640 1 -- 172.21.15.43:0/1512885664 shutdown_connections 2024-03-26T08:40:05.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.906+0000 7f3b11200640 1 --2- 172.21.15.43:0/1512885664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b0c105490 0x7f3b0c10bd20 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:05.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.906+0000 7f3b11200640 1 --2- 172.21.15.43:0/1512885664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b0c104ad0 0x7f3b0c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:05.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.906+0000 7f3b11200640 1 --2- 172.21.15.43:0/1512885664 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b0c1038e0 0x7f3b0c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:05.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.906+0000 7f3b11200640 1 -- 172.21.15.43:0/1512885664 >> 172.21.15.43:0/1512885664 conn(0x7f3b0c0fee50 msgr2=0x7f3b0c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:05.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.906+0000 7f3b11200640 1 -- 172.21.15.43:0/1512885664 shutdown_connections 2024-03-26T08:40:05.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.906+0000 7f3b11200640 1 -- 172.21.15.43:0/1512885664 wait complete. 2024-03-26T08:40:05.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.907+0000 7f3b11200640 1 Processor -- start 2024-03-26T08:40:05.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.907+0000 7f3b11200640 1 -- start start 2024-03-26T08:40:05.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.907+0000 7f3b11200640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b0c1038e0 0x7f3b0c199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:05.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.907+0000 7f3b11200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b0c104ad0 0x7f3b0c199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:05.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.907+0000 7f3b11200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b0c105490 0x7f3b0c1a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:05.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.907+0000 7f3b11200640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c10f440 con 0x7f3b0c104ad0 2024-03-26T08:40:05.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.907+0000 7f3b11200640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c10f2c0 con 0x7f3b0c1038e0 2024-03-26T08:40:05.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.907+0000 7f3b11200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c10f5c0 con 0x7f3b0c105490 2024-03-26T08:40:05.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.907+0000 7f3b01400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b0c104ad0 0x7f3b0c199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:05.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.908+0000 7f3b01400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b0c104ad0 0x7f3b0c199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39622/0 (socket says 172.21.15.43:39622) 2024-03-26T08:40:05.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.908+0000 7f3b01400640 1 -- 172.21.15.43:0/2270609310 learned_addr learned my addr 172.21.15.43:0/2270609310 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:40:05.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.908+0000 7f3b0a800640 1 --2- 172.21.15.43:0/2270609310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b0c105490 0x7f3b0c1a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:05.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.908+0000 7f3b09e00640 1 --2- 172.21.15.43:0/2270609310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b0c1038e0 0x7f3b0c199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:05.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.908+0000 7f3b01400640 1 -- 172.21.15.43:0/2270609310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b0c105490 msgr2=0x7f3b0c1a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:05.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.908+0000 7f3b01400640 1 --2- 172.21.15.43:0/2270609310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b0c105490 0x7f3b0c1a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:05.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.908+0000 7f3b01400640 1 -- 172.21.15.43:0/2270609310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b0c1038e0 msgr2=0x7f3b0c199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:05.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.908+0000 7f3b01400640 1 --2- 172.21.15.43:0/2270609310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b0c1038e0 0x7f3b0c199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:05.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.908+0000 7f3b01400640 1 -- 172.21.15.43:0/2270609310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0c1a16a0 con 0x7f3b0c104ad0 2024-03-26T08:40:05.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.908+0000 7f3b01400640 1 --2- 172.21.15.43:0/2270609310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b0c104ad0 0x7f3b0c199fc0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3af400d950 tx=0x7f3af400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:05.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.909+0000 7f3b03400640 1 -- 172.21.15.43:0/2270609310 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af4017070 con 0x7f3b0c104ad0 2024-03-26T08:40:05.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.909+0000 7f3b03400640 1 -- 172.21.15.43:0/2270609310 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3af4009c30 con 0x7f3b0c104ad0 2024-03-26T08:40:05.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.909+0000 7f3b03400640 1 -- 172.21.15.43:0/2270609310 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af400f620 con 0x7f3b0c104ad0 2024-03-26T08:40:05.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.909+0000 7f3b11200640 1 -- 172.21.15.43:0/2270609310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b0c1a1990 con 0x7f3b0c104ad0 2024-03-26T08:40:05.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.909+0000 7f3b11200640 1 -- 172.21.15.43:0/2270609310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b0c1a1f30 con 0x7f3b0c104ad0 2024-03-26T08:40:05.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.910+0000 7f3b11200640 1 -- 172.21.15.43:0/2270609310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b0c10ac00 con 0x7f3b0c104ad0 2024-03-26T08:40:05.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.912+0000 7f3b03400640 1 -- 172.21.15.43:0/2270609310 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f3af40109a0 con 0x7f3b0c104ad0 2024-03-26T08:40:05.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.913+0000 7f3b03400640 1 --2- 172.21.15.43:0/2270609310 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3af00a2880 0x7f3af00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:05.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.913+0000 7f3b03400640 1 -- 172.21.15.43:0/2270609310 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f3af40ceac0 con 0x7f3b0c104ad0 2024-03-26T08:40:05.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.914+0000 7f3b09e00640 1 --2- 172.21.15.43:0/2270609310 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3af00a2880 0x7f3af00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:05.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.915+0000 7f3b09e00640 1 --2- 172.21.15.43:0/2270609310 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3af00a2880 0x7f3af00a4d40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c104930 tx=0x7f3afc009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:05.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:05.918+0000 7f3b03400640 1 -- 172.21.15.43:0/2270609310 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3af4009de0 con 0x7f3b0c104ad0 2024-03-26T08:40:06.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.122+0000 7f3b11200640 1 -- 172.21.15.43:0/2270609310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3b0c0622a0 con 0x7f3b0c104ad0 2024-03-26T08:40:06.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.123+0000 7f3b03400640 1 -- 172.21.15.43:0/2270609310 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3af409b780 con 0x7f3b0c104ad0 2024-03-26T08:40:06.115 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-26T08:40:06.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.125+0000 7f3b11200640 1 -- 172.21.15.43:0/2270609310 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3af00a2880 msgr2=0x7f3af00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:06.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.125+0000 7f3b11200640 1 --2- 172.21.15.43:0/2270609310 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3af00a2880 0x7f3af00a4d40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c104930 tx=0x7f3afc009870 comp rx=0 tx=0).stop 2024-03-26T08:40:06.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.125+0000 7f3b11200640 1 -- 172.21.15.43:0/2270609310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b0c104ad0 msgr2=0x7f3b0c199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:06.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.125+0000 7f3b11200640 1 --2- 172.21.15.43:0/2270609310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b0c104ad0 0x7f3b0c199fc0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3af400d950 tx=0x7f3af400de10 comp rx=0 tx=0).stop 2024-03-26T08:40:06.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.125+0000 7f3b11200640 1 -- 172.21.15.43:0/2270609310 shutdown_connections 2024-03-26T08:40:06.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.126+0000 7f3b11200640 1 --2- 172.21.15.43:0/2270609310 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f3af00a2880 0x7f3af00a4d40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.126+0000 7f3b11200640 1 --2- 172.21.15.43:0/2270609310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b0c105490 0x7f3b0c1a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.126+0000 7f3b11200640 1 --2- 172.21.15.43:0/2270609310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b0c104ad0 0x7f3b0c199fc0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.126+0000 7f3b11200640 1 --2- 172.21.15.43:0/2270609310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b0c1038e0 0x7f3b0c199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.126+0000 7f3b11200640 1 -- 172.21.15.43:0/2270609310 >> 172.21.15.43:0/2270609310 conn(0x7f3b0c0fee50 msgr2=0x7f3b0c107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:06.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.126+0000 7f3b11200640 1 -- 172.21.15.43:0/2270609310 shutdown_connections 2024-03-26T08:40:06.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.126+0000 7f3b11200640 1 -- 172.21.15.43:0/2270609310 wait complete. 2024-03-26T08:40:06.133 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:05 smithi043 ceph-mon[31929]: pgmap v638: 129 pgs: 129 active+clean; 985 KiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.3 KiB/s wr, 216 op/s 2024-03-26T08:40:06.134 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:05 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3731915231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:40:06.134 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:05 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/46407995' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:40:06.136 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T08:40:06.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:05 smithi183 ceph-mon[35818]: pgmap v638: 129 pgs: 129 active+clean; 985 KiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.3 KiB/s wr, 216 op/s 2024-03-26T08:40:06.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:05 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3731915231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:40:06.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:05 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/46407995' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:40:06.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:05 smithi052 ceph-mon[35071]: pgmap v638: 129 pgs: 129 active+clean; 985 KiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.3 KiB/s wr, 216 op/s 2024-03-26T08:40:06.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:05 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3731915231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:40:06.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:05 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/46407995' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:40:06.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.369+0000 7f39b2000640 1 -- 172.21.15.43:0/2961186751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39ac104930 msgr2=0x7f39ac106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:06.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.369+0000 7f39b2000640 1 --2- 172.21.15.43:0/2961186751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39ac104930 0x7f39ac106d20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3998009900 tx=0x7f399802bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:06.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.369+0000 7f39b2000640 1 -- 172.21.15.43:0/2961186751 shutdown_connections 2024-03-26T08:40:06.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.369+0000 7f39b2000640 1 --2- 172.21.15.43:0/2961186751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39ac107260 0x7f39ac1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.369+0000 7f39b2000640 1 --2- 172.21.15.43:0/2961186751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39ac104930 0x7f39ac106d20 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.369+0000 7f39b2000640 1 --2- 172.21.15.43:0/2961186751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39ac102000 0x7f39ac1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.369+0000 7f39b2000640 1 -- 172.21.15.43:0/2961186751 >> 172.21.15.43:0/2961186751 conn(0x7f39ac0fba10 msgr2=0x7f39ac0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:06.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.369+0000 7f39b2000640 1 -- 172.21.15.43:0/2961186751 shutdown_connections 2024-03-26T08:40:06.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.369+0000 7f39b2000640 1 -- 172.21.15.43:0/2961186751 wait complete. 2024-03-26T08:40:06.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.370+0000 7f39b2000640 1 Processor -- start 2024-03-26T08:40:06.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.371+0000 7f39b2000640 1 -- start start 2024-03-26T08:40:06.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.371+0000 7f39b2000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39ac102000 0x7f39ac199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:06.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.371+0000 7f39b2000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39ac104930 0x7f39ac199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:06.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.371+0000 7f39b2000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39ac107260 0x7f39ac1a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:06.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.371+0000 7f39b2000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac10d110 con 0x7f39ac102000 2024-03-26T08:40:06.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.371+0000 7f39b2000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac10cf90 con 0x7f39ac107260 2024-03-26T08:40:06.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.371+0000 7f39b2000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac10d290 con 0x7f39ac104930 2024-03-26T08:40:06.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.371+0000 7f39b0c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39ac102000 0x7f39ac199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:06.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.371+0000 7f39b0c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39ac102000 0x7f39ac199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39650/0 (socket says 172.21.15.43:39650) 2024-03-26T08:40:06.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.371+0000 7f39b0c00640 1 -- 172.21.15.43:0/100067035 learned_addr learned my addr 172.21.15.43:0/100067035 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:40:06.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.372+0000 7f39abe00640 1 --2- 172.21.15.43:0/100067035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39ac104930 0x7f39ac199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:06.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.372+0000 7f39b1600640 1 --2- 172.21.15.43:0/100067035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39ac107260 0x7f39ac1a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:06.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.372+0000 7f39b0c00640 1 -- 172.21.15.43:0/100067035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39ac104930 msgr2=0x7f39ac199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:06.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.372+0000 7f39b0c00640 1 --2- 172.21.15.43:0/100067035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39ac104930 0x7f39ac199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.372+0000 7f39b0c00640 1 -- 172.21.15.43:0/100067035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39ac107260 msgr2=0x7f39ac1a0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:06.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.372+0000 7f39b0c00640 1 --2- 172.21.15.43:0/100067035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39ac107260 0x7f39ac1a0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.372+0000 7f39b0c00640 1 -- 172.21.15.43:0/100067035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ac1a1580 con 0x7f39ac102000 2024-03-26T08:40:06.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.372+0000 7f39b1600640 1 --2- 172.21.15.43:0/100067035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39ac107260 0x7f39ac1a0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:40:06.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.372+0000 7f39abe00640 1 --2- 172.21.15.43:0/100067035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39ac104930 0x7f39ac199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:40:06.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.372+0000 7f39b0c00640 1 --2- 172.21.15.43:0/100067035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39ac102000 0x7f39ac199960 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f39a000c920 tx=0x7f39a000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:06.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.372+0000 7f39a9600640 1 -- 172.21.15.43:0/100067035 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a0007c70 con 0x7f39ac102000 2024-03-26T08:40:06.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.372+0000 7f39a9600640 1 -- 172.21.15.43:0/100067035 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f39a0007e10 con 0x7f39ac102000 2024-03-26T08:40:06.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.372+0000 7f39b2000640 1 -- 172.21.15.43:0/100067035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39ac1a1870 con 0x7f39ac102000 2024-03-26T08:40:06.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.372+0000 7f39a9600640 1 -- 172.21.15.43:0/100067035 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a0018820 con 0x7f39ac102000 2024-03-26T08:40:06.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.373+0000 7f39b2000640 1 -- 172.21.15.43:0/100067035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39ac1a1de0 con 0x7f39ac102000 2024-03-26T08:40:06.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.374+0000 7f39b2000640 1 -- 172.21.15.43:0/100067035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39ac108db0 con 0x7f39ac102000 2024-03-26T08:40:06.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.376+0000 7f39a9600640 1 -- 172.21.15.43:0/100067035 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f39a00189c0 con 0x7f39ac102000 2024-03-26T08:40:06.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.377+0000 7f39a9600640 1 --2- 172.21.15.43:0/100067035 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f39940a2880 0x7f39940a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:06.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.377+0000 7f39a9600640 1 -- 172.21.15.43:0/100067035 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f39a00cefc0 con 0x7f39ac102000 2024-03-26T08:40:06.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.377+0000 7f39abe00640 1 --2- 172.21.15.43:0/100067035 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f39940a2880 0x7f39940a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:06.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.378+0000 7f39abe00640 1 --2- 172.21.15.43:0/100067035 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f39940a2880 0x7f39940a4d40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f399802c240 tx=0x7f3998005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:06.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.381+0000 7f39a9600640 1 -- 172.21.15.43:0/100067035 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f39a0014030 con 0x7f39ac102000 2024-03-26T08:40:06.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.591+0000 7f39b2000640 1 -- 172.21.15.43:0/100067035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f39ac0622a0 con 0x7f39ac102000 2024-03-26T08:40:06.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.592+0000 7f39a9600640 1 -- 172.21.15.43:0/100067035 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7f39a009bc80 con 0x7f39ac102000 2024-03-26T08:40:06.584 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-26T08:40:06.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.595+0000 7f39b2000640 1 -- 172.21.15.43:0/100067035 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f39940a2880 msgr2=0x7f39940a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:06.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.595+0000 7f39b2000640 1 --2- 172.21.15.43:0/100067035 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f39940a2880 0x7f39940a4d40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f399802c240 tx=0x7f3998005fd0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.595+0000 7f39b2000640 1 -- 172.21.15.43:0/100067035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39ac102000 msgr2=0x7f39ac199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:06.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.595+0000 7f39b2000640 1 --2- 172.21.15.43:0/100067035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39ac102000 0x7f39ac199960 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f39a000c920 tx=0x7f39a000cde0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.595+0000 7f39b2000640 1 -- 172.21.15.43:0/100067035 shutdown_connections 2024-03-26T08:40:06.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.595+0000 7f39b2000640 1 --2- 172.21.15.43:0/100067035 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f39940a2880 0x7f39940a4d40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.595+0000 7f39b2000640 1 --2- 172.21.15.43:0/100067035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39ac107260 0x7f39ac1a0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.595+0000 7f39b2000640 1 --2- 172.21.15.43:0/100067035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39ac104930 0x7f39ac199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.595+0000 7f39b2000640 1 --2- 172.21.15.43:0/100067035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39ac102000 0x7f39ac199960 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.595+0000 7f39b2000640 1 -- 172.21.15.43:0/100067035 >> 172.21.15.43:0/100067035 conn(0x7f39ac0fba10 msgr2=0x7f39ac101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:06.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.595+0000 7f39b2000640 1 -- 172.21.15.43:0/100067035 shutdown_connections 2024-03-26T08:40:06.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.595+0000 7f39b2000640 1 -- 172.21.15.43:0/100067035 wait complete. 2024-03-26T08:40:06.609 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-26T08:40:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.842+0000 7f26b0a00640 1 -- 172.21.15.43:0/2803268475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26ac105370 msgr2=0x7f26ac10bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.842+0000 7f26b0a00640 1 --2- 172.21.15.43:0/2803268475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26ac105370 0x7f26ac10bc00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f26a4009900 tx=0x7f26a402bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.842+0000 7f26b0a00640 1 -- 172.21.15.43:0/2803268475 shutdown_connections 2024-03-26T08:40:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.842+0000 7f26b0a00640 1 --2- 172.21.15.43:0/2803268475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26ac105370 0x7f26ac10bc00 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.842+0000 7f26b0a00640 1 --2- 172.21.15.43:0/2803268475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26ac1049b0 0x7f26ac104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.842+0000 7f26b0a00640 1 --2- 172.21.15.43:0/2803268475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26ac1037c0 0x7f26ac103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.842+0000 7f26b0a00640 1 -- 172.21.15.43:0/2803268475 >> 172.21.15.43:0/2803268475 conn(0x7f26ac0fed50 msgr2=0x7f26ac101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.842+0000 7f26b0a00640 1 -- 172.21.15.43:0/2803268475 shutdown_connections 2024-03-26T08:40:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.842+0000 7f26b0a00640 1 -- 172.21.15.43:0/2803268475 wait complete. 2024-03-26T08:40:06.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.843+0000 7f26b0a00640 1 Processor -- start 2024-03-26T08:40:06.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.843+0000 7f26b0a00640 1 -- start start 2024-03-26T08:40:06.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.843+0000 7f26b0a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26ac1037c0 0x7f26ac199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:06.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.843+0000 7f26b0a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26ac1049b0 0x7f26ac199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:06.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.843+0000 7f26b0a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26ac105370 0x7f26ac1a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:06.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.843+0000 7f26b0a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac10f320 con 0x7f26ac1037c0 2024-03-26T08:40:06.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.843+0000 7f26b0a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac10f1a0 con 0x7f26ac1049b0 2024-03-26T08:40:06.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.843+0000 7f26b0a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac10f4a0 con 0x7f26ac105370 2024-03-26T08:40:06.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.844+0000 7f26ab400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26ac1037c0 0x7f26ac199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:06.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.844+0000 7f26ab400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26ac1037c0 0x7f26ac199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39674/0 (socket says 172.21.15.43:39674) 2024-03-26T08:40:06.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.844+0000 7f26ab400640 1 -- 172.21.15.43:0/4262193435 learned_addr learned my addr 172.21.15.43:0/4262193435 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:40:06.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.844+0000 7f26aaa00640 1 --2- 172.21.15.43:0/4262193435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26ac1049b0 0x7f26ac199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:06.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.844+0000 7f26abe00640 1 --2- 172.21.15.43:0/4262193435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26ac105370 0x7f26ac1a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:06.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.844+0000 7f26ab400640 1 -- 172.21.15.43:0/4262193435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26ac105370 msgr2=0x7f26ac1a0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:06.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.844+0000 7f26ab400640 1 --2- 172.21.15.43:0/4262193435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26ac105370 0x7f26ac1a0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.844+0000 7f26ab400640 1 -- 172.21.15.43:0/4262193435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26ac1049b0 msgr2=0x7f26ac199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:06.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.844+0000 7f26ab400640 1 --2- 172.21.15.43:0/4262193435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26ac1049b0 0x7f26ac199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.844+0000 7f26ab400640 1 -- 172.21.15.43:0/4262193435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ac1a1580 con 0x7f26ac1037c0 2024-03-26T08:40:06.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.844+0000 7f26abe00640 1 --2- 172.21.15.43:0/4262193435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26ac105370 0x7f26ac1a0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:40:06.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.845+0000 7f26ab400640 1 --2- 172.21.15.43:0/4262193435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26ac1037c0 0x7f26ac199960 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f269c00ea80 tx=0x7f269c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:06.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.845+0000 7f26a3e00640 1 -- 172.21.15.43:0/4262193435 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c00ce40 con 0x7f26ac1037c0 2024-03-26T08:40:06.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.845+0000 7f26a3e00640 1 -- 172.21.15.43:0/4262193435 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f269c011920 con 0x7f26ac1037c0 2024-03-26T08:40:06.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.845+0000 7f26b0a00640 1 -- 172.21.15.43:0/4262193435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26ac1a1870 con 0x7f26ac1037c0 2024-03-26T08:40:06.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.845+0000 7f26a3e00640 1 -- 172.21.15.43:0/4262193435 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c01aaa0 con 0x7f26ac1037c0 2024-03-26T08:40:06.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.845+0000 7f26b0a00640 1 -- 172.21.15.43:0/4262193435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26ac1a1e10 con 0x7f26ac1037c0 2024-03-26T08:40:06.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.846+0000 7f26b0a00640 1 -- 172.21.15.43:0/4262193435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26ac10aae0 con 0x7f26ac1037c0 2024-03-26T08:40:06.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.848+0000 7f26a3e00640 1 -- 172.21.15.43:0/4262193435 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f269c011ad0 con 0x7f26ac1037c0 2024-03-26T08:40:06.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.850+0000 7f26a3e00640 1 --2- 172.21.15.43:0/4262193435 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f26900a2880 0x7f26900a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:06.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.850+0000 7f26a3e00640 1 -- 172.21.15.43:0/4262193435 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f269c0cf440 con 0x7f26ac1037c0 2024-03-26T08:40:06.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.850+0000 7f26aaa00640 1 --2- 172.21.15.43:0/4262193435 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f26900a2880 0x7f26900a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:06.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.851+0000 7f26aaa00640 1 --2- 172.21.15.43:0/4262193435 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f26900a2880 0x7f26900a4d40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2694005fd0 tx=0x7f2694005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:06.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:06.854+0000 7f26a3e00640 1 -- 172.21.15.43:0/4262193435 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f269c016030 con 0x7f26ac1037c0 2024-03-26T08:40:07.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.076+0000 7f26b0a00640 1 -- 172.21.15.43:0/4262193435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f26ac02d070 con 0x7f26ac1037c0 2024-03-26T08:40:07.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.077+0000 7f26a3e00640 1 -- 172.21.15.43:0/4262193435 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v94) v1 ==== 81+0+590 (secure 0 0 0) 0x7f26ac02d070 con 0x7f26ac1037c0 2024-03-26T08:40:07.068 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6801/2446912488 2024-03-27T08:21:36.149092+0000 2024-03-26T08:40:07.069 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6800/2446912488 2024-03-27T08:21:36.149092+0000 2024-03-26T08:40:07.069 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/4240775295 2024-03-27T08:21:36.149092+0000 2024-03-26T08:40:07.069 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/510203140 2024-03-27T08:21:01.256355+0000 2024-03-26T08:40:07.069 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/4251957687 2024-03-27T08:21:36.149092+0000 2024-03-26T08:40:07.069 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6800/4108777098 2024-03-27T08:21:01.256355+0000 2024-03-26T08:40:07.069 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6801/4108777098 2024-03-27T08:21:01.256355+0000 2024-03-26T08:40:07.069 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/1885244665 2024-03-27T08:21:01.256355+0000 2024-03-26T08:40:07.069 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/298320867 2024-03-27T08:21:01.256355+0000 2024-03-26T08:40:07.069 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/1301092625 2024-03-27T08:21:36.149092+0000 2024-03-26T08:40:07.069 INFO:teuthology.orchestra.run.smithi043.stderr:listed 10 entries 2024-03-26T08:40:07.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.079+0000 7f26b0a00640 1 -- 172.21.15.43:0/4262193435 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f26900a2880 msgr2=0x7f26900a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:07.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.079+0000 7f26b0a00640 1 --2- 172.21.15.43:0/4262193435 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f26900a2880 0x7f26900a4d40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2694005fd0 tx=0x7f2694005850 comp rx=0 tx=0).stop 2024-03-26T08:40:07.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.079+0000 7f26b0a00640 1 -- 172.21.15.43:0/4262193435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26ac1037c0 msgr2=0x7f26ac199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:07.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.079+0000 7f26b0a00640 1 --2- 172.21.15.43:0/4262193435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26ac1037c0 0x7f26ac199960 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f269c00ea80 tx=0x7f269c00ef40 comp rx=0 tx=0).stop 2024-03-26T08:40:07.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.079+0000 7f26b0a00640 1 -- 172.21.15.43:0/4262193435 shutdown_connections 2024-03-26T08:40:07.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.079+0000 7f26b0a00640 1 --2- 172.21.15.43:0/4262193435 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f26900a2880 0x7f26900a4d40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.079+0000 7f26b0a00640 1 --2- 172.21.15.43:0/4262193435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26ac105370 0x7f26ac1a0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.079+0000 7f26b0a00640 1 --2- 172.21.15.43:0/4262193435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26ac1049b0 0x7f26ac199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.079+0000 7f26b0a00640 1 --2- 172.21.15.43:0/4262193435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26ac1037c0 0x7f26ac199960 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.079+0000 7f26b0a00640 1 -- 172.21.15.43:0/4262193435 >> 172.21.15.43:0/4262193435 conn(0x7f26ac0fed50 msgr2=0x7f26ac107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:07.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.080+0000 7f26b0a00640 1 -- 172.21.15.43:0/4262193435 shutdown_connections 2024-03-26T08:40:07.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.080+0000 7f26b0a00640 1 -- 172.21.15.43:0/4262193435 wait complete. 2024-03-26T08:40:07.089 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-26T08:40:07.089 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-26T08:40:07.144 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:40:07.223 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:06 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2270609310' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:40:07.223 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:06 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/100067035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T08:40:07.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:06 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2270609310' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:40:07.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:06 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/100067035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T08:40:07.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:06 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2270609310' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:40:07.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:06 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/100067035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T08:40:07.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.365+0000 7f1ebe600640 1 -- 172.21.15.43:0/2572081186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1eb8076060 msgr2=0x7f1eb8076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:07.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.365+0000 7f1ebe600640 1 --2- 172.21.15.43:0/2572081186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1eb8076060 0x7f1eb8076480 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8009900 tx=0x7f1ea802bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:07.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.366+0000 7f1ebe600640 1 -- 172.21.15.43:0/2572081186 shutdown_connections 2024-03-26T08:40:07.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.366+0000 7f1ebe600640 1 --2- 172.21.15.43:0/2572081186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1eb8077c40 0x7f1eb80784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.366+0000 7f1ebe600640 1 --2- 172.21.15.43:0/2572081186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eb80772a0 0x7f1eb8077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.366+0000 7f1ebe600640 1 --2- 172.21.15.43:0/2572081186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1eb8076060 0x7f1eb8076480 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.366+0000 7f1ebe600640 1 -- 172.21.15.43:0/2572081186 >> 172.21.15.43:0/2572081186 conn(0x7f1eb80fee50 msgr2=0x7f1eb8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:07.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.366+0000 7f1ebe600640 1 -- 172.21.15.43:0/2572081186 shutdown_connections 2024-03-26T08:40:07.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.366+0000 7f1ebe600640 1 -- 172.21.15.43:0/2572081186 wait complete. 2024-03-26T08:40:07.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.367+0000 7f1ebe600640 1 Processor -- start 2024-03-26T08:40:07.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.367+0000 7f1ebe600640 1 -- start start 2024-03-26T08:40:07.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.367+0000 7f1ebe600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1eb8076060 0x7f1eb819df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:07.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.367+0000 7f1ebe600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eb80772a0 0x7f1eb819e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:07.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.367+0000 7f1ebe600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1eb8077c40 0x7f1eb81a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:07.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.367+0000 7f1ebe600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb8113860 con 0x7f1eb8077c40 2024-03-26T08:40:07.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.367+0000 7f1ebe600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb81136e0 con 0x7f1eb8076060 2024-03-26T08:40:07.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.367+0000 7f1ebe600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb81139e0 con 0x7f1eb80772a0 2024-03-26T08:40:07.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.368+0000 7f1eb7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1eb8077c40 0x7f1eb81a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:07.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.368+0000 7f1eb7e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1eb8077c40 0x7f1eb81a5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39696/0 (socket says 172.21.15.43:39696) 2024-03-26T08:40:07.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.368+0000 7f1eb7e00640 1 -- 172.21.15.43:0/4230705849 learned_addr learned my addr 172.21.15.43:0/4230705849 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:40:07.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.368+0000 7f1eb7400640 1 --2- 172.21.15.43:0/4230705849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1eb8076060 0x7f1eb819df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:07.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.368+0000 7f1eb6a00640 1 --2- 172.21.15.43:0/4230705849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eb80772a0 0x7f1eb819e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:07.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.368+0000 7f1eb7e00640 1 -- 172.21.15.43:0/4230705849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eb80772a0 msgr2=0x7f1eb819e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:07.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.368+0000 7f1eb7e00640 1 --2- 172.21.15.43:0/4230705849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eb80772a0 0x7f1eb819e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.368+0000 7f1eb7e00640 1 -- 172.21.15.43:0/4230705849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1eb8076060 msgr2=0x7f1eb819df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:07.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.368+0000 7f1eb7e00640 1 --2- 172.21.15.43:0/4230705849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1eb8076060 0x7f1eb819df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.368+0000 7f1eb7e00640 1 -- 172.21.15.43:0/4230705849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb81a5b70 con 0x7f1eb8077c40 2024-03-26T08:40:07.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.368+0000 7f1eb7400640 1 --2- 172.21.15.43:0/4230705849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1eb8076060 0x7f1eb819df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:40:07.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.368+0000 7f1eb7e00640 1 --2- 172.21.15.43:0/4230705849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1eb8077c40 0x7f1eb81a5500 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0009fd0 tx=0x7f1eb000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:07.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.369+0000 7f1eafe00640 1 -- 172.21.15.43:0/4230705849 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0017070 con 0x7f1eb8077c40 2024-03-26T08:40:07.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.369+0000 7f1eafe00640 1 -- 172.21.15.43:0/4230705849 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1eb0003e20 con 0x7f1eb8077c40 2024-03-26T08:40:07.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.369+0000 7f1eafe00640 1 -- 172.21.15.43:0/4230705849 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0010460 con 0x7f1eb8077c40 2024-03-26T08:40:07.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.369+0000 7f1ebe600640 1 -- 172.21.15.43:0/4230705849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb81a5e60 con 0x7f1eb8077c40 2024-03-26T08:40:07.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.369+0000 7f1ebe600640 1 -- 172.21.15.43:0/4230705849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb81a6320 con 0x7f1eb8077c40 2024-03-26T08:40:07.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.370+0000 7f1ebe600640 1 -- 172.21.15.43:0/4230705849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eb8110490 con 0x7f1eb8077c40 2024-03-26T08:40:07.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.372+0000 7f1eafe00640 1 -- 172.21.15.43:0/4230705849 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f1eb0003970 con 0x7f1eb8077c40 2024-03-26T08:40:07.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.373+0000 7f1eafe00640 1 --2- 172.21.15.43:0/4230705849 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1e9c0a2880 0x7f1e9c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:07.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.373+0000 7f1eafe00640 1 -- 172.21.15.43:0/4230705849 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f1eb00cecb0 con 0x7f1eb8077c40 2024-03-26T08:40:07.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.374+0000 7f1eb7400640 1 --2- 172.21.15.43:0/4230705849 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1e9c0a2880 0x7f1e9c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:07.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.375+0000 7f1eb7400640 1 --2- 172.21.15.43:0/4230705849 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1e9c0a2880 0x7f1e9c0a4d40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8006010 tx=0x7f1ea802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:07.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.378+0000 7f1eafe00640 1 -- 172.21.15.43:0/4230705849 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1eb009b970 con 0x7f1eb8077c40 2024-03-26T08:40:07.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.581+0000 7f1ebe600640 1 -- 172.21.15.43:0/4230705849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1eb80622a0 con 0x7f1eb8077c40 2024-03-26T08:40:07.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.584+0000 7f1eafe00640 1 -- 172.21.15.43:0/4230705849 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24670 (secure 0 0 0) 0x7f1eb009b270 con 0x7f1eb8077c40 2024-03-26T08:40:07.576 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:40:07.577 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":94,"fsid":"88759420-eb49-11ee-95d0-87774f69a715","created":"2024-03-26T08:20:27.817107+0000","modified":"2024-03-26T08:32:38.256877+0000","last_up_change":"2024-03-26T08:30:10.520976+0000","last_in_change":"2024-03-26T08:29:46.394360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:24:54.852427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:31:19.924902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:31:21.323881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T08:31:24.951015+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"28670673-9193-4fa3-9cc0-c1a5a6b27284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6803","nonce":936601146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6805","nonce":936601146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6809","nonce":936601146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6807","nonce":936601146}]},"public_addr":"172.21.15.43:6803/936601146","cluster_addr":"172.21.15.43:6805/936601146","heartbeat_back_addr":"172.21.15.43:6809/936601146","heartbeat_front_addr":"172.21.15.43:6807/936601146","state":["exists","up"]},{"osd":1,"uuid":"fa8ad6b6-277a-42b8-b0f8-b36fc219de2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6801","nonce":3268412328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6803","nonce":3268412328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6807","nonce":3268412328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6805","nonce":3268412328}]},"public_addr":"172.21.15.52:6801/3268412328","cluster_addr":"172.21.15.52:6803/3268412328","heartbeat_back_addr":"172.21.15.52:6807/3268412328","heartbeat_front_addr":"172.21.15.52:6805/3268412328","state":["exists","up"]},{"osd":2,"uuid":"ec0039c3-f9bd-419a-81de-fa4418359808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6801","nonce":2103180908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6803","nonce":2103180908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6807","nonce":2103180908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6805","nonce":2103180908}]},"public_addr":"172.21.15.183:6801/2103180908","cluster_addr":"172.21.15.183:6803/2103180908","heartbeat_back_addr":"172.21.15.183:6807/2103180908","heartbeat_front_addr":"172.21.15.183:6805/2103180908","state":["exists","up"]},{"osd":3,"uuid":"e7dcbf00-6586-4664-a69e-3cb0df8dfd30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6811","nonce":3722846033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6813","nonce":3722846033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6817","nonce":3722846033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6815","nonce":3722846033}]},"public_addr":"172.21.15.43:6811/3722846033","cluster_addr":"172.21.15.43:6813/3722846033","heartbeat_back_addr":"172.21.15.43:6817/3722846033","heartbeat_front_addr":"172.21.15.43:6815/3722846033","state":["exists","up"]},{"osd":4,"uuid":"db089a49-5e98-432d-9687-cdc044114c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6809","nonce":4028436523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6811","nonce":4028436523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6815","nonce":4028436523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6813","nonce":4028436523}]},"public_addr":"172.21.15.52:6809/4028436523","cluster_addr":"172.21.15.52:6811/4028436523","heartbeat_back_addr":"172.21.15.52:6815/4028436523","heartbeat_front_addr":"172.21.15.52:6813/4028436523","state":["exists","up"]},{"osd":5,"uuid":"556ac804-02ea-423b-a599-55baf5be4ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6809","nonce":787132252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6811","nonce":787132252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6815","nonce":787132252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6813","nonce":787132252}]},"public_addr":"172.21.15.183:6809/787132252","cluster_addr":"172.21.15.183:6811/787132252","heartbeat_back_addr":"172.21.15.183:6815/787132252","heartbeat_front_addr":"172.21.15.183:6813/787132252","state":["exists","up"]},{"osd":6,"uuid":"3de2a49d-6cef-4580-9d8a-f90cdfcecd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6819","nonce":4206861158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6821","nonce":4206861158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6825","nonce":4206861158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6823","nonce":4206861158}]},"public_addr":"172.21.15.43:6819/4206861158","cluster_addr":"172.21.15.43:6821/4206861158","heartbeat_back_addr":"172.21.15.43:6825/4206861158","heartbeat_front_addr":"172.21.15.43:6823/4206861158","state":["exists","up"]},{"osd":7,"uuid":"a3b832f7-7d7d-4a62-b679-db6da98275d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6817","nonce":3452064146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6819","nonce":3452064146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6823","nonce":3452064146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6821","nonce":3452064146}]},"public_addr":"172.21.15.52:6817/3452064146","cluster_addr":"172.21.15.52:6819/3452064146","heartbeat_back_addr":"172.21.15.52:6823/3452064146","heartbeat_front_addr":"172.21.15.52:6821/3452064146","state":["exists","up"]},{"osd":8,"uuid":"ddc6b07f-d703-414d-a59a-02316376b13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6817","nonce":1023131725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6819","nonce":1023131725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6823","nonce":1023131725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6821","nonce":1023131725}]},"public_addr":"172.21.15.183:6817/1023131725","cluster_addr":"172.21.15.183:6819/1023131725","heartbeat_back_addr":"172.21.15.183:6823/1023131725","heartbeat_front_addr":"172.21.15.183:6821/1023131725","state":["exists","up"]},{"osd":9,"uuid":"ac16bb0e-6232-45c8-b16a-4d878874c588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6827","nonce":3935195657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6829","nonce":3935195657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6833","nonce":3935195657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6831","nonce":3935195657}]},"public_addr":"172.21.15.43:6827/3935195657","cluster_addr":"172.21.15.43:6829/3935195657","heartbeat_back_addr":"172.21.15.43:6833/3935195657","heartbeat_front_addr":"172.21.15.43:6831/3935195657","state":["exists","up"]},{"osd":10,"uuid":"fd839188-f398-4cff-aa7b-36522ff1a8a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6825","nonce":377628320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6827","nonce":377628320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6831","nonce":377628320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6829","nonce":377628320}]},"public_addr":"172.21.15.52:6825/377628320","cluster_addr":"172.21.15.52:6827/377628320","heartbeat_back_addr":"172.21.15.52:6831/377628320","heartbeat_front_addr":"172.21.15.52:6829/377628320","state":["exists","up"]},{"osd":11,"uuid":"a40282a0-0c64-49ee-b64e-6f3f8119e827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6825","nonce":1373086433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6827","nonce":1373086433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6831","nonce":1373086433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6829","nonce":1373086433}]},"public_addr":"172.21.15.183:6825/1373086433","cluster_addr":"172.21.15.183:6827/1373086433","heartbeat_back_addr":"172.21.15.183:6831/1373086433","heartbeat_front_addr":"172.21.15.183:6829/1373086433","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:23:43.478905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:17.573532+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:50.286610+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:25:27.407663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:03.081294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:36.057416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:12.062013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:47.032271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:21.023201+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:57.364739+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:29:34.039995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:30:08.827577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6801/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/4240775295":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/510203140":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/4251957687":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:6801/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1885244665":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/298320867":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1301092625":"2024-03-27T08:21:36.149092+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:40:07.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.587+0000 7f1ebe600640 1 -- 172.21.15.43:0/4230705849 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1e9c0a2880 msgr2=0x7f1e9c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:07.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.587+0000 7f1ebe600640 1 --2- 172.21.15.43:0/4230705849 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1e9c0a2880 0x7f1e9c0a4d40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8006010 tx=0x7f1ea802c7e0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.587+0000 7f1ebe600640 1 -- 172.21.15.43:0/4230705849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1eb8077c40 msgr2=0x7f1eb81a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:07.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.587+0000 7f1ebe600640 1 --2- 172.21.15.43:0/4230705849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1eb8077c40 0x7f1eb81a5500 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0009fd0 tx=0x7f1eb000b260 comp rx=0 tx=0).stop 2024-03-26T08:40:07.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.587+0000 7f1ebe600640 1 -- 172.21.15.43:0/4230705849 shutdown_connections 2024-03-26T08:40:07.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.587+0000 7f1ebe600640 1 --2- 172.21.15.43:0/4230705849 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f1e9c0a2880 0x7f1e9c0a4d40 secure :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8006010 tx=0x7f1ea802c7e0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.587+0000 7f1ebe600640 1 --2- 172.21.15.43:0/4230705849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1eb8077c40 0x7f1eb81a5500 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.587+0000 7f1ebe600640 1 --2- 172.21.15.43:0/4230705849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eb80772a0 0x7f1eb819e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.587+0000 7f1ebe600640 1 --2- 172.21.15.43:0/4230705849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1eb8076060 0x7f1eb819df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.587+0000 7f1ebe600640 1 -- 172.21.15.43:0/4230705849 >> 172.21.15.43:0/4230705849 conn(0x7f1eb80fee50 msgr2=0x7f1eb8100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:07.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.587+0000 7f1ebe600640 1 -- 172.21.15.43:0/4230705849 shutdown_connections 2024-03-26T08:40:07.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.587+0000 7f1ebe600640 1 -- 172.21.15.43:0/4230705849 wait complete. 2024-03-26T08:40:07.600 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T08:24:54.852427+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T08:31:19.924902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T08:31:21.323881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T08:31:24.951015+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T08:40:07.601 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T08:40:07.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.834+0000 7f538c800640 1 -- 172.21.15.43:0/4209630666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5384102120 msgr2=0x7f5384104510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:07.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.834+0000 7f538c800640 1 --2- 172.21.15.43:0/4209630666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5384102120 0x7f5384104510 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5378009900 tx=0x7f537802bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:07.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.834+0000 7f538c800640 1 -- 172.21.15.43:0/4209630666 shutdown_connections 2024-03-26T08:40:07.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.834+0000 7f538c800640 1 --2- 172.21.15.43:0/4209630666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5384107380 0x7f53841097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.834+0000 7f538c800640 1 --2- 172.21.15.43:0/4209630666 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5384104a50 0x7f5384106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.834+0000 7f538c800640 1 --2- 172.21.15.43:0/4209630666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5384102120 0x7f5384104510 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.834+0000 7f538c800640 1 -- 172.21.15.43:0/4209630666 >> 172.21.15.43:0/4209630666 conn(0x7f53840fbb30 msgr2=0x7f53840fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:07.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.834+0000 7f538c800640 1 -- 172.21.15.43:0/4209630666 shutdown_connections 2024-03-26T08:40:07.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.834+0000 7f538c800640 1 -- 172.21.15.43:0/4209630666 wait complete. 2024-03-26T08:40:07.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.835+0000 7f538c800640 1 Processor -- start 2024-03-26T08:40:07.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.835+0000 7f538c800640 1 -- start start 2024-03-26T08:40:07.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.835+0000 7f538c800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5384102120 0x7f5384197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:07.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.835+0000 7f538c800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5384104a50 0x7f5384197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:07.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.835+0000 7f538c800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5384107380 0x7f538419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:07.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.835+0000 7f538c800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538410d250 con 0x7f5384102120 2024-03-26T08:40:07.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.835+0000 7f538c800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538410d0d0 con 0x7f5384104a50 2024-03-26T08:40:07.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.835+0000 7f538c800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538410d3d0 con 0x7f5384107380 2024-03-26T08:40:07.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.836+0000 7f5389e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5384102120 0x7f5384197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:07.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.836+0000 7f5389e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5384102120 0x7f5384197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39722/0 (socket says 172.21.15.43:39722) 2024-03-26T08:40:07.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.836+0000 7f5389e00640 1 -- 172.21.15.43:0/3468910130 learned_addr learned my addr 172.21.15.43:0/3468910130 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:40:07.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.836+0000 7f5389400640 1 --2- 172.21.15.43:0/3468910130 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5384104a50 0x7f5384197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:07.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.836+0000 7f538a800640 1 --2- 172.21.15.43:0/3468910130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5384107380 0x7f538419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:07.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.836+0000 7f5389e00640 1 -- 172.21.15.43:0/3468910130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5384107380 msgr2=0x7f538419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:07.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.836+0000 7f5389e00640 1 --2- 172.21.15.43:0/3468910130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5384107380 0x7f538419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.836+0000 7f5389e00640 1 -- 172.21.15.43:0/3468910130 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5384104a50 msgr2=0x7f5384197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:07.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.836+0000 7f5389e00640 1 --2- 172.21.15.43:0/3468910130 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5384104a50 0x7f5384197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.836+0000 7f5389e00640 1 -- 172.21.15.43:0/3468910130 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538419f5a0 con 0x7f5384102120 2024-03-26T08:40:07.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.836+0000 7f538a800640 1 --2- 172.21.15.43:0/3468910130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5384107380 0x7f538419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:40:07.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.837+0000 7f5389e00640 1 --2- 172.21.15.43:0/3468910130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5384102120 0x7f5384197980 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5378000c00 tx=0x7f53780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:07.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.837+0000 7f537ea00640 1 -- 172.21.15.43:0/3468910130 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5378043020 con 0x7f5384102120 2024-03-26T08:40:07.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.837+0000 7f537ea00640 1 -- 172.21.15.43:0/3468910130 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f53780054c0 con 0x7f5384102120 2024-03-26T08:40:07.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.837+0000 7f537ea00640 1 -- 172.21.15.43:0/3468910130 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5378003d40 con 0x7f5384102120 2024-03-26T08:40:07.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.837+0000 7f538c800640 1 -- 172.21.15.43:0/3468910130 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538419f830 con 0x7f5384102120 2024-03-26T08:40:07.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.837+0000 7f538c800640 1 -- 172.21.15.43:0/3468910130 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538419fdd0 con 0x7f5384102120 2024-03-26T08:40:07.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.838+0000 7f538c800640 1 -- 172.21.15.43:0/3468910130 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53840fd620 con 0x7f5384102120 2024-03-26T08:40:07.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.841+0000 7f537ea00640 1 -- 172.21.15.43:0/3468910130 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f5378046020 con 0x7f5384102120 2024-03-26T08:40:07.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.843+0000 7f537ea00640 1 --2- 172.21.15.43:0/3468910130 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f536c0a2880 0x7f536c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:07.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.843+0000 7f537ea00640 1 -- 172.21.15.43:0/3468910130 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f53780f1760 con 0x7f5384102120 2024-03-26T08:40:07.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.844+0000 7f5389400640 1 --2- 172.21.15.43:0/3468910130 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f536c0a2880 0x7f536c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:07.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.844+0000 7f5389400640 1 --2- 172.21.15.43:0/3468910130 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f536c0a2880 0x7f536c0a4d40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5374005d00 tx=0x7f5374005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:07.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:07.846+0000 7f537ea00640 1 -- 172.21.15.43:0/3468910130 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f537804d020 con 0x7f5384102120 2024-03-26T08:40:08.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.052+0000 7f538c800640 1 -- 172.21.15.43:0/3468910130 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f538402d2e0 con 0x7f5384102120 2024-03-26T08:40:08.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.053+0000 7f537ea00640 1 -- 172.21.15.43:0/3468910130 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f53780be540 con 0x7f5384102120 2024-03-26T08:40:08.045 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-26T08:40:08.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.056+0000 7f538c800640 1 -- 172.21.15.43:0/3468910130 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f536c0a2880 msgr2=0x7f536c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:08.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.056+0000 7f538c800640 1 --2- 172.21.15.43:0/3468910130 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f536c0a2880 0x7f536c0a4d40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5374005d00 tx=0x7f5374005c50 comp rx=0 tx=0).stop 2024-03-26T08:40:08.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.056+0000 7f538c800640 1 -- 172.21.15.43:0/3468910130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5384102120 msgr2=0x7f5384197980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:08.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.056+0000 7f538c800640 1 --2- 172.21.15.43:0/3468910130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5384102120 0x7f5384197980 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5378000c00 tx=0x7f53780042f0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.056+0000 7f538c800640 1 -- 172.21.15.43:0/3468910130 shutdown_connections 2024-03-26T08:40:08.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.056+0000 7f538c800640 1 --2- 172.21.15.43:0/3468910130 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f536c0a2880 0x7f536c0a4d40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.056+0000 7f538c800640 1 --2- 172.21.15.43:0/3468910130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5384107380 0x7f538419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.056+0000 7f538c800640 1 --2- 172.21.15.43:0/3468910130 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5384104a50 0x7f5384197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.056+0000 7f538c800640 1 --2- 172.21.15.43:0/3468910130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5384102120 0x7f5384197980 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.056+0000 7f538c800640 1 -- 172.21.15.43:0/3468910130 >> 172.21.15.43:0/3468910130 conn(0x7f53840fbb30 msgr2=0x7f5384102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:08.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.056+0000 7f538c800640 1 -- 172.21.15.43:0/3468910130 shutdown_connections 2024-03-26T08:40:08.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.056+0000 7f538c800640 1 -- 172.21.15.43:0/3468910130 wait complete. 2024-03-26T08:40:08.066 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T08:40:08.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.303+0000 7f724f000640 1 -- 172.21.15.43:0/274136987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7248077c40 msgr2=0x7f72480784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:08.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.303+0000 7f724f000640 1 --2- 172.21.15.43:0/274136987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7248077c40 0x7f72480784e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7240009ca0 tx=0x7f724002c100 comp rx=0 tx=0).stop 2024-03-26T08:40:08.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.303+0000 7f724f000640 1 -- 172.21.15.43:0/274136987 shutdown_connections 2024-03-26T08:40:08.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.303+0000 7f724f000640 1 --2- 172.21.15.43:0/274136987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7248077c40 0x7f72480784e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.303+0000 7f724f000640 1 --2- 172.21.15.43:0/274136987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72480772a0 0x7f7248077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.303+0000 7f724f000640 1 --2- 172.21.15.43:0/274136987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7248076060 0x7f7248076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.303+0000 7f724f000640 1 -- 172.21.15.43:0/274136987 >> 172.21.15.43:0/274136987 conn(0x7f72480fee50 msgr2=0x7f7248101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:08.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.303+0000 7f724f000640 1 -- 172.21.15.43:0/274136987 shutdown_connections 2024-03-26T08:40:08.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.303+0000 7f724f000640 1 -- 172.21.15.43:0/274136987 wait complete. 2024-03-26T08:40:08.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.304+0000 7f724f000640 1 Processor -- start 2024-03-26T08:40:08.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.304+0000 7f724f000640 1 -- start start 2024-03-26T08:40:08.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.304+0000 7f724f000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7248076060 0x7f724819def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:08.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.304+0000 7f724f000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72480772a0 0x7f724819e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:08.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.304+0000 7f724f000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7248077c40 0x7f72481a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:08.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.304+0000 7f724f000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7248113820 con 0x7f7248076060 2024-03-26T08:40:08.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.304+0000 7f724f000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72481136a0 con 0x7f72480772a0 2024-03-26T08:40:08.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.304+0000 7f724f000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72481139a0 con 0x7f7248077c40 2024-03-26T08:40:08.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.305+0000 7f7247e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7248076060 0x7f724819def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:08.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.305+0000 7f7247e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7248076060 0x7f724819def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39742/0 (socket says 172.21.15.43:39742) 2024-03-26T08:40:08.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.305+0000 7f7247e00640 1 -- 172.21.15.43:0/3933454346 learned_addr learned my addr 172.21.15.43:0/3933454346 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:40:08.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.305+0000 7f7247400640 1 --2- 172.21.15.43:0/3933454346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72480772a0 0x7f724819e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:08.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.305+0000 7f7247e00640 1 -- 172.21.15.43:0/3933454346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7248077c40 msgr2=0x7f72481a54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:08.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.305+0000 7f7247e00640 1 --2- 172.21.15.43:0/3933454346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7248077c40 0x7f72481a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.305+0000 7f7247e00640 1 -- 172.21.15.43:0/3933454346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72480772a0 msgr2=0x7f724819e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:08.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.305+0000 7f7247e00640 1 --2- 172.21.15.43:0/3933454346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72480772a0 0x7f724819e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.305+0000 7f7247e00640 1 -- 172.21.15.43:0/3933454346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72481a5b10 con 0x7f7248076060 2024-03-26T08:40:08.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.305+0000 7f7247400640 1 --2- 172.21.15.43:0/3933454346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72480772a0 0x7f724819e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:40:08.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.305+0000 7f7247e00640 1 --2- 172.21.15.43:0/3933454346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7248076060 0x7f724819def0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f723800d770 tx=0x7f723800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:08.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:07 smithi183 ceph-mon[35818]: pgmap v639: 129 pgs: 129 active+clean; 992 KiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 3.2 KiB/s wr, 217 op/s 2024-03-26T08:40:08.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:07 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/4262193435' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T08:40:08.299 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:07 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/4230705849' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:08.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.306+0000 7f7244c00640 1 -- 172.21.15.43:0/3933454346 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7238017070 con 0x7f7248076060 2024-03-26T08:40:08.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.306+0000 7f7244c00640 1 -- 172.21.15.43:0/3933454346 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f72380054d0 con 0x7f7248076060 2024-03-26T08:40:08.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.306+0000 7f724f000640 1 -- 172.21.15.43:0/3933454346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72481a5e00 con 0x7f7248076060 2024-03-26T08:40:08.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.306+0000 7f7244c00640 1 -- 172.21.15.43:0/3933454346 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7238004cf0 con 0x7f7248076060 2024-03-26T08:40:08.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.306+0000 7f724f000640 1 -- 172.21.15.43:0/3933454346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72481a6260 con 0x7f7248076060 2024-03-26T08:40:08.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.307+0000 7f724f000640 1 -- 172.21.15.43:0/3933454346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7248110250 con 0x7f7248076060 2024-03-26T08:40:08.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.309+0000 7f7244c00640 1 -- 172.21.15.43:0/3933454346 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f7238004e90 con 0x7f7248076060 2024-03-26T08:40:08.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.310+0000 7f7244c00640 1 --2- 172.21.15.43:0/3933454346 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f722c0a2800 0x7f722c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:08.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.310+0000 7f7244c00640 1 -- 172.21.15.43:0/3933454346 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f72380ce950 con 0x7f7248076060 2024-03-26T08:40:08.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.311+0000 7f7247400640 1 --2- 172.21.15.43:0/3933454346 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f722c0a2800 0x7f722c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:08.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.311+0000 7f7247400640 1 --2- 172.21.15.43:0/3933454346 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f722c0a2800 0x7f722c0a4cc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f724819f400 tx=0x7f7234005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:08.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.315+0000 7f7244c00640 1 -- 172.21.15.43:0/3933454346 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f723809b610 con 0x7f7248076060 2024-03-26T08:40:08.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:07 smithi052 ceph-mon[35071]: pgmap v639: 129 pgs: 129 active+clean; 992 KiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 3.2 KiB/s wr, 217 op/s 2024-03-26T08:40:08.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:07 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/4262193435' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T08:40:08.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:07 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/4230705849' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:08.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:07 smithi043 ceph-mon[31929]: pgmap v639: 129 pgs: 129 active+clean; 992 KiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 3.2 KiB/s wr, 217 op/s 2024-03-26T08:40:08.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:07 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/4262193435' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T08:40:08.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:07 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/4230705849' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:08.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.519+0000 7f724f000640 1 -- 172.21.15.43:0/3933454346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f72480623a0 con 0x7f7248076060 2024-03-26T08:40:08.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.520+0000 7f7244c00640 1 -- 172.21.15.43:0/3933454346 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f723809af10 con 0x7f7248076060 2024-03-26T08:40:08.512 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-26T08:40:08.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.522+0000 7f724f000640 1 -- 172.21.15.43:0/3933454346 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f722c0a2800 msgr2=0x7f722c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:08.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.522+0000 7f724f000640 1 --2- 172.21.15.43:0/3933454346 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f722c0a2800 0x7f722c0a4cc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f724819f400 tx=0x7f7234005e50 comp rx=0 tx=0).stop 2024-03-26T08:40:08.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.522+0000 7f724f000640 1 -- 172.21.15.43:0/3933454346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7248076060 msgr2=0x7f724819def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:08.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.522+0000 7f724f000640 1 --2- 172.21.15.43:0/3933454346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7248076060 0x7f724819def0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f723800d770 tx=0x7f723800dc30 comp rx=0 tx=0).stop 2024-03-26T08:40:08.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.523+0000 7f724f000640 1 -- 172.21.15.43:0/3933454346 shutdown_connections 2024-03-26T08:40:08.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.523+0000 7f724f000640 1 --2- 172.21.15.43:0/3933454346 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f722c0a2800 0x7f722c0a4cc0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.523+0000 7f724f000640 1 --2- 172.21.15.43:0/3933454346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7248077c40 0x7f72481a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.523+0000 7f724f000640 1 --2- 172.21.15.43:0/3933454346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72480772a0 0x7f724819e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.523+0000 7f724f000640 1 --2- 172.21.15.43:0/3933454346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7248076060 0x7f724819def0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.523+0000 7f724f000640 1 -- 172.21.15.43:0/3933454346 >> 172.21.15.43:0/3933454346 conn(0x7f72480fee50 msgr2=0x7f7248100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:08.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.523+0000 7f724f000640 1 -- 172.21.15.43:0/3933454346 shutdown_connections 2024-03-26T08:40:08.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.523+0000 7f724f000640 1 -- 172.21.15.43:0/3933454346 wait complete. 2024-03-26T08:40:08.535 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T08:40:08.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.767+0000 7fb6b2600640 1 -- 172.21.15.43:0/235759250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6ac1037a0 msgr2=0x7fb6ac103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:08.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.767+0000 7fb6b2600640 1 --2- 172.21.15.43:0/235759250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6ac1037a0 0x7fb6ac103bc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009900 tx=0x7fb6a002bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:08.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.767+0000 7fb6b2600640 1 -- 172.21.15.43:0/235759250 shutdown_connections 2024-03-26T08:40:08.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.767+0000 7fb6b2600640 1 --2- 172.21.15.43:0/235759250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6ac105350 0x7fb6ac10bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.767+0000 7fb6b2600640 1 --2- 172.21.15.43:0/235759250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6ac104990 0x7fb6ac104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.767+0000 7fb6b2600640 1 --2- 172.21.15.43:0/235759250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6ac1037a0 0x7fb6ac103bc0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.767+0000 7fb6b2600640 1 -- 172.21.15.43:0/235759250 >> 172.21.15.43:0/235759250 conn(0x7fb6ac0fed10 msgr2=0x7fb6ac101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:08.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.767+0000 7fb6b2600640 1 -- 172.21.15.43:0/235759250 shutdown_connections 2024-03-26T08:40:08.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.767+0000 7fb6b2600640 1 -- 172.21.15.43:0/235759250 wait complete. 2024-03-26T08:40:08.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.768+0000 7fb6b2600640 1 Processor -- start 2024-03-26T08:40:08.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.768+0000 7fb6b2600640 1 -- start start 2024-03-26T08:40:08.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.769+0000 7fb6b2600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6ac1037a0 0x7fb6ac1999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:08.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.769+0000 7fb6b2600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6ac104990 0x7fb6ac199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:08.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.769+0000 7fb6b2600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6ac105350 0x7fb6ac1a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:08.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.769+0000 7fb6b2600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac10f320 con 0x7fb6ac1037a0 2024-03-26T08:40:08.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.769+0000 7fb6b2600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac10f1a0 con 0x7fb6ac105350 2024-03-26T08:40:08.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.769+0000 7fb6b2600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac10f4a0 con 0x7fb6ac104990 2024-03-26T08:40:08.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.769+0000 7fb6b1200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6ac1037a0 0x7fb6ac1999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:08.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.769+0000 7fb6b1200640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6ac1037a0 0x7fb6ac1999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39764/0 (socket says 172.21.15.43:39764) 2024-03-26T08:40:08.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.769+0000 7fb6b1200640 1 -- 172.21.15.43:0/3318933289 learned_addr learned my addr 172.21.15.43:0/3318933289 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:40:08.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.769+0000 7fb6abe00640 1 --2- 172.21.15.43:0/3318933289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6ac104990 0x7fb6ac199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:08.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.770+0000 7fb6b1c00640 1 --2- 172.21.15.43:0/3318933289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6ac105350 0x7fb6ac1a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:08.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.770+0000 7fb6b1200640 1 -- 172.21.15.43:0/3318933289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6ac104990 msgr2=0x7fb6ac199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:08.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.770+0000 7fb6b1200640 1 --2- 172.21.15.43:0/3318933289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6ac104990 0x7fb6ac199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.770+0000 7fb6b1200640 1 -- 172.21.15.43:0/3318933289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6ac105350 msgr2=0x7fb6ac1a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:08.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.770+0000 7fb6b1200640 1 --2- 172.21.15.43:0/3318933289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6ac105350 0x7fb6ac1a0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.770+0000 7fb6b1200640 1 -- 172.21.15.43:0/3318933289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ac1a15e0 con 0x7fb6ac1037a0 2024-03-26T08:40:08.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.770+0000 7fb6b1200640 1 --2- 172.21.15.43:0/3318933289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6ac1037a0 0x7fb6ac1999c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0000c00 tx=0x7fb6a00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:08.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.770+0000 7fb6a9600640 1 -- 172.21.15.43:0/3318933289 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0043020 con 0x7fb6ac1037a0 2024-03-26T08:40:08.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.771+0000 7fb6a9600640 1 -- 172.21.15.43:0/3318933289 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb6a00044c0 con 0x7fb6ac1037a0 2024-03-26T08:40:08.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.771+0000 7fb6a9600640 1 -- 172.21.15.43:0/3318933289 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0003c60 con 0x7fb6ac1037a0 2024-03-26T08:40:08.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.771+0000 7fb6b2600640 1 -- 172.21.15.43:0/3318933289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6ac1a1870 con 0x7fb6ac1037a0 2024-03-26T08:40:08.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.771+0000 7fb6b2600640 1 -- 172.21.15.43:0/3318933289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ac1a1d70 con 0x7fb6ac1037a0 2024-03-26T08:40:08.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.772+0000 7fb6b2600640 1 -- 172.21.15.43:0/3318933289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6ac10aac0 con 0x7fb6ac1037a0 2024-03-26T08:40:08.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.774+0000 7fb6a9600640 1 -- 172.21.15.43:0/3318933289 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fb6a0046020 con 0x7fb6ac1037a0 2024-03-26T08:40:08.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.775+0000 7fb6a9600640 1 --2- 172.21.15.43:0/3318933289 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb6940a2880 0x7fb6940a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:08.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.775+0000 7fb6a9600640 1 -- 172.21.15.43:0/3318933289 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7fb6a00f0fe0 con 0x7fb6ac1037a0 2024-03-26T08:40:08.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.776+0000 7fb6abe00640 1 --2- 172.21.15.43:0/3318933289 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb6940a2880 0x7fb6940a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:08.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.776+0000 7fb6abe00640 1 --2- 172.21.15.43:0/3318933289 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb6940a2880 0x7fb6940a4d40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac19aed0 tx=0x7fb69800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:08.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.779+0000 7fb6a9600640 1 -- 172.21.15.43:0/3318933289 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb6a004d020 con 0x7fb6ac1037a0 2024-03-26T08:40:08.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.986+0000 7fb6b2600640 1 -- 172.21.15.43:0/3318933289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb6ac0622a0 con 0x7fb6ac1037a0 2024-03-26T08:40:08.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.987+0000 7fb6a9600640 1 -- 172.21.15.43:0/3318933289 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb6a00bdce0 con 0x7fb6ac1037a0 2024-03-26T08:40:08.979 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-26T08:40:08.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.990+0000 7fb6b2600640 1 -- 172.21.15.43:0/3318933289 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb6940a2880 msgr2=0x7fb6940a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:08.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.990+0000 7fb6b2600640 1 --2- 172.21.15.43:0/3318933289 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb6940a2880 0x7fb6940a4d40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac19aed0 tx=0x7fb69800b390 comp rx=0 tx=0).stop 2024-03-26T08:40:08.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.990+0000 7fb6b2600640 1 -- 172.21.15.43:0/3318933289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6ac1037a0 msgr2=0x7fb6ac1999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:08.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.990+0000 7fb6b2600640 1 --2- 172.21.15.43:0/3318933289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6ac1037a0 0x7fb6ac1999c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0000c00 tx=0x7fb6a00040c0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.990+0000 7fb6b2600640 1 -- 172.21.15.43:0/3318933289 shutdown_connections 2024-03-26T08:40:08.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.990+0000 7fb6b2600640 1 --2- 172.21.15.43:0/3318933289 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fb6940a2880 0x7fb6940a4d40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.990+0000 7fb6b2600640 1 --2- 172.21.15.43:0/3318933289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb6ac105350 0x7fb6ac1a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.990+0000 7fb6b2600640 1 --2- 172.21.15.43:0/3318933289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6ac104990 0x7fb6ac199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.990+0000 7fb6b2600640 1 --2- 172.21.15.43:0/3318933289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6ac1037a0 0x7fb6ac1999c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.990+0000 7fb6b2600640 1 -- 172.21.15.43:0/3318933289 >> 172.21.15.43:0/3318933289 conn(0x7fb6ac0fed10 msgr2=0x7fb6ac107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:08.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.990+0000 7fb6b2600640 1 -- 172.21.15.43:0/3318933289 shutdown_connections 2024-03-26T08:40:08.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:08.990+0000 7fb6b2600640 1 -- 172.21.15.43:0/3318933289 wait complete. 2024-03-26T08:40:09.001 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T08:40:09.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.230+0000 7ffad2600640 1 -- 172.21.15.43:0/3910242257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffacc104ad0 msgr2=0x7ffacc104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:09.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.230+0000 7ffad2600640 1 --2- 172.21.15.43:0/3910242257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffacc104ad0 0x7ffacc104f50 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ffab8009900 tx=0x7ffab802bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:09.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.230+0000 7ffad2600640 1 -- 172.21.15.43:0/3910242257 shutdown_connections 2024-03-26T08:40:09.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.230+0000 7ffad2600640 1 --2- 172.21.15.43:0/3910242257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffacc105490 0x7ffacc10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:09.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.230+0000 7ffad2600640 1 --2- 172.21.15.43:0/3910242257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffacc104ad0 0x7ffacc104f50 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:09.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.230+0000 7ffad2600640 1 --2- 172.21.15.43:0/3910242257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffacc1038e0 0x7ffacc103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:09.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.230+0000 7ffad2600640 1 -- 172.21.15.43:0/3910242257 >> 172.21.15.43:0/3910242257 conn(0x7ffacc0fee50 msgr2=0x7ffacc101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:09.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.231+0000 7ffad2600640 1 -- 172.21.15.43:0/3910242257 shutdown_connections 2024-03-26T08:40:09.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.231+0000 7ffad2600640 1 -- 172.21.15.43:0/3910242257 wait complete. 2024-03-26T08:40:09.224 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:08 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3468910130' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:40:09.224 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:08 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3933454346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:40:09.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.231+0000 7ffad2600640 1 Processor -- start 2024-03-26T08:40:09.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.232+0000 7ffad2600640 1 -- start start 2024-03-26T08:40:09.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.232+0000 7ffad2600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffacc1038e0 0x7ffacc199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:09.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.232+0000 7ffad2600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffacc104ad0 0x7ffacc19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:09.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.232+0000 7ffad2600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffacc105490 0x7ffacc1a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:09.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.232+0000 7ffad2600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc10f410 con 0x7ffacc1038e0 2024-03-26T08:40:09.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.232+0000 7ffad2600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc10f290 con 0x7ffacc104ad0 2024-03-26T08:40:09.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.232+0000 7ffad2600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc10f590 con 0x7ffacc105490 2024-03-26T08:40:09.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.232+0000 7ffacb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffacc1038e0 0x7ffacc199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:09.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.232+0000 7ffacb400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffacc1038e0 0x7ffacc199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39788/0 (socket says 172.21.15.43:39788) 2024-03-26T08:40:09.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.232+0000 7ffacb400640 1 -- 172.21.15.43:0/2564746066 learned_addr learned my addr 172.21.15.43:0/2564746066 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:40:09.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.233+0000 7ffacaa00640 1 --2- 172.21.15.43:0/2564746066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffacc104ad0 0x7ffacc19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:09.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.233+0000 7ffacbe00640 1 --2- 172.21.15.43:0/2564746066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffacc105490 0x7ffacc1a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:09.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.233+0000 7ffacb400640 1 -- 172.21.15.43:0/2564746066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffacc105490 msgr2=0x7ffacc1a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:09.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.233+0000 7ffacb400640 1 --2- 172.21.15.43:0/2564746066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffacc105490 0x7ffacc1a1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:09.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.233+0000 7ffacb400640 1 -- 172.21.15.43:0/2564746066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffacc104ad0 msgr2=0x7ffacc19a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:09.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.233+0000 7ffacb400640 1 --2- 172.21.15.43:0/2564746066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffacc104ad0 0x7ffacc19a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:09.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.233+0000 7ffacb400640 1 -- 172.21.15.43:0/2564746066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffacc1a1770 con 0x7ffacc1038e0 2024-03-26T08:40:09.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.233+0000 7ffacb400640 1 --2- 172.21.15.43:0/2564746066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffacc1038e0 0x7ffacc199b50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00c970 tx=0x7ffabc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:09.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.233+0000 7ffac3e00640 1 -- 172.21.15.43:0/2564746066 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc004220 con 0x7ffacc1038e0 2024-03-26T08:40:09.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.233+0000 7ffac3e00640 1 -- 172.21.15.43:0/2564746066 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffabc0043c0 con 0x7ffacc1038e0 2024-03-26T08:40:09.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.234+0000 7ffac3e00640 1 -- 172.21.15.43:0/2564746066 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc004a00 con 0x7ffacc1038e0 2024-03-26T08:40:09.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.234+0000 7ffad2600640 1 -- 172.21.15.43:0/2564746066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffacc1a1a60 con 0x7ffacc1038e0 2024-03-26T08:40:09.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.234+0000 7ffad2600640 1 -- 172.21.15.43:0/2564746066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffacc1a1f60 con 0x7ffacc1038e0 2024-03-26T08:40:09.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.235+0000 7ffad2600640 1 -- 172.21.15.43:0/2564746066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffacc10ad20 con 0x7ffacc1038e0 2024-03-26T08:40:09.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.237+0000 7ffac3e00640 1 -- 172.21.15.43:0/2564746066 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7ffabc0079b0 con 0x7ffacc1038e0 2024-03-26T08:40:09.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.238+0000 7ffac3e00640 1 --2- 172.21.15.43:0/2564746066 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ffab00a2800 0x7ffab00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:09.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.238+0000 7ffac3e00640 1 -- 172.21.15.43:0/2564746066 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7ffabc0cec70 con 0x7ffacc1038e0 2024-03-26T08:40:09.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.238+0000 7ffacaa00640 1 --2- 172.21.15.43:0/2564746066 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ffab00a2800 0x7ffab00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:09.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.239+0000 7ffacaa00640 1 --2- 172.21.15.43:0/2564746066 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ffab00a2800 0x7ffab00a4cc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ffacc19b060 tx=0x7ffab8005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:09.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.243+0000 7ffac3e00640 1 -- 172.21.15.43:0/2564746066 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ffabc014030 con 0x7ffacc1038e0 2024-03-26T08:40:09.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:08 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3468910130' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:40:09.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:08 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3933454346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:40:09.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:08 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3468910130' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:40:09.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:08 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3933454346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:40:09.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.450+0000 7ffad2600640 1 -- 172.21.15.43:0/2564746066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ffacc0622a0 con 0x7ffacc1038e0 2024-03-26T08:40:09.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.451+0000 7ffac3e00640 1 -- 172.21.15.43:0/2564746066 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7ffabc09b930 con 0x7ffacc1038e0 2024-03-26T08:40:09.442 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-26T08:40:09.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.453+0000 7ffad2600640 1 -- 172.21.15.43:0/2564746066 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ffab00a2800 msgr2=0x7ffab00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:09.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.453+0000 7ffad2600640 1 --2- 172.21.15.43:0/2564746066 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ffab00a2800 0x7ffab00a4cc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ffacc19b060 tx=0x7ffab8005be0 comp rx=0 tx=0).stop 2024-03-26T08:40:09.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.454+0000 7ffad2600640 1 -- 172.21.15.43:0/2564746066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffacc1038e0 msgr2=0x7ffacc199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:09.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.454+0000 7ffad2600640 1 --2- 172.21.15.43:0/2564746066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffacc1038e0 0x7ffacc199b50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00c970 tx=0x7ffabc00ce30 comp rx=0 tx=0).stop 2024-03-26T08:40:09.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.454+0000 7ffad2600640 1 -- 172.21.15.43:0/2564746066 shutdown_connections 2024-03-26T08:40:09.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.454+0000 7ffad2600640 1 --2- 172.21.15.43:0/2564746066 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7ffab00a2800 0x7ffab00a4cc0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:09.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.454+0000 7ffad2600640 1 --2- 172.21.15.43:0/2564746066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffacc105490 0x7ffacc1a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:09.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.454+0000 7ffad2600640 1 --2- 172.21.15.43:0/2564746066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffacc104ad0 0x7ffacc19a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:09.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.454+0000 7ffad2600640 1 --2- 172.21.15.43:0/2564746066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffacc1038e0 0x7ffacc199b50 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:09.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.454+0000 7ffad2600640 1 -- 172.21.15.43:0/2564746066 >> 172.21.15.43:0/2564746066 conn(0x7ffacc0fee50 msgr2=0x7ffacc107ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:09.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.454+0000 7ffad2600640 1 -- 172.21.15.43:0/2564746066 shutdown_connections 2024-03-26T08:40:09.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.454+0000 7ffad2600640 1 -- 172.21.15.43:0/2564746066 wait complete. 2024-03-26T08:40:09.466 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-26T08:40:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.701+0000 7fa426000640 1 -- 172.21.15.43:0/3306364093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa420102000 msgr2=0x7fa4201043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.701+0000 7fa426000640 1 --2- 172.21.15.43:0/3306364093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa420102000 0x7fa4201043f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa410009900 tx=0x7fa41002bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.701+0000 7fa426000640 1 -- 172.21.15.43:0/3306364093 shutdown_connections 2024-03-26T08:40:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.701+0000 7fa426000640 1 --2- 172.21.15.43:0/3306364093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa420107260 0x7fa4201096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.701+0000 7fa426000640 1 --2- 172.21.15.43:0/3306364093 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa420104930 0x7fa420106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.701+0000 7fa426000640 1 --2- 172.21.15.43:0/3306364093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa420102000 0x7fa4201043f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.701+0000 7fa426000640 1 -- 172.21.15.43:0/3306364093 >> 172.21.15.43:0/3306364093 conn(0x7fa4200fba10 msgr2=0x7fa4200fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.701+0000 7fa426000640 1 -- 172.21.15.43:0/3306364093 shutdown_connections 2024-03-26T08:40:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.701+0000 7fa426000640 1 -- 172.21.15.43:0/3306364093 wait complete. 2024-03-26T08:40:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.702+0000 7fa426000640 1 Processor -- start 2024-03-26T08:40:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.702+0000 7fa426000640 1 -- start start 2024-03-26T08:40:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.703+0000 7fa426000640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa420102000 0x7fa4201999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.703+0000 7fa426000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa420104930 0x7fa420199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.703+0000 7fa426000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa420107260 0x7fa4201a0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.703+0000 7fa426000640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42010d0c0 con 0x7fa420102000 2024-03-26T08:40:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.703+0000 7fa426000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42010cf40 con 0x7fa420107260 2024-03-26T08:40:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.703+0000 7fa426000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42010d240 con 0x7fa420104930 2024-03-26T08:40:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.703+0000 7fa424c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa420102000 0x7fa4201999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.703+0000 7fa424c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa420102000 0x7fa4201999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39808/0 (socket says 172.21.15.43:39808) 2024-03-26T08:40:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.703+0000 7fa424c00640 1 -- 172.21.15.43:0/1376253731 learned_addr learned my addr 172.21.15.43:0/1376253731 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:40:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.703+0000 7fa425600640 1 --2- 172.21.15.43:0/1376253731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa420107260 0x7fa4201a0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.703+0000 7fa417e00640 1 --2- 172.21.15.43:0/1376253731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa420104930 0x7fa420199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.703+0000 7fa424c00640 1 -- 172.21.15.43:0/1376253731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa420104930 msgr2=0x7fa420199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.703+0000 7fa424c00640 1 --2- 172.21.15.43:0/1376253731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa420104930 0x7fa420199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.703+0000 7fa424c00640 1 -- 172.21.15.43:0/1376253731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa420107260 msgr2=0x7fa4201a0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.703+0000 7fa424c00640 1 --2- 172.21.15.43:0/1376253731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa420107260 0x7fa4201a0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:09.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.703+0000 7fa424c00640 1 -- 172.21.15.43:0/1376253731 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4201a15d0 con 0x7fa420102000 2024-03-26T08:40:09.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.704+0000 7fa424c00640 1 --2- 172.21.15.43:0/1376253731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa420102000 0x7fa4201999b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa410006010 tx=0x7fa410005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:09.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.704+0000 7fa41e000640 1 -- 172.21.15.43:0/1376253731 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa410043020 con 0x7fa420102000 2024-03-26T08:40:09.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.704+0000 7fa41e000640 1 -- 172.21.15.43:0/1376253731 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa4100043d0 con 0x7fa420102000 2024-03-26T08:40:09.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.704+0000 7fa41e000640 1 -- 172.21.15.43:0/1376253731 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa410003b80 con 0x7fa420102000 2024-03-26T08:40:09.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.704+0000 7fa426000640 1 -- 172.21.15.43:0/1376253731 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4201a1860 con 0x7fa420102000 2024-03-26T08:40:09.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.704+0000 7fa426000640 1 -- 172.21.15.43:0/1376253731 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4201a1e00 con 0x7fa420102000 2024-03-26T08:40:09.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.705+0000 7fa426000640 1 -- 172.21.15.43:0/1376253731 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa420108db0 con 0x7fa420102000 2024-03-26T08:40:09.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.707+0000 7fa41e000640 1 -- 172.21.15.43:0/1376253731 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fa410046020 con 0x7fa420102000 2024-03-26T08:40:09.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.709+0000 7fa41e000640 1 --2- 172.21.15.43:0/1376253731 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa4040a2880 0x7fa4040a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:09.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.709+0000 7fa41e000640 1 -- 172.21.15.43:0/1376253731 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7fa4100f1130 con 0x7fa420102000 2024-03-26T08:40:09.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.709+0000 7fa417e00640 1 --2- 172.21.15.43:0/1376253731 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa4040a2880 0x7fa4040a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:09.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.710+0000 7fa417e00640 1 --2- 172.21.15.43:0/1376253731 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa4040a2880 0x7fa4040a4d40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa42019aec0 tx=0x7fa408005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:09.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.713+0000 7fa41e000640 1 -- 172.21.15.43:0/1376253731 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa41004d020 con 0x7fa420102000 2024-03-26T08:40:09.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.936+0000 7fa426000640 1 -- 172.21.15.43:0/1376253731 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa42002d070 con 0x7fa420102000 2024-03-26T08:40:09.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.936+0000 7fa41e000640 1 -- 172.21.15.43:0/1376253731 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v94) v1 ==== 81+0+590 (secure 0 0 0) 0x7fa41003c350 con 0x7fa420102000 2024-03-26T08:40:09.928 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6801/2446912488 2024-03-27T08:21:36.149092+0000 2024-03-26T08:40:09.928 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6800/2446912488 2024-03-27T08:21:36.149092+0000 2024-03-26T08:40:09.928 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/4240775295 2024-03-27T08:21:36.149092+0000 2024-03-26T08:40:09.928 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/510203140 2024-03-27T08:21:01.256355+0000 2024-03-26T08:40:09.928 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/4251957687 2024-03-27T08:21:36.149092+0000 2024-03-26T08:40:09.929 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6800/4108777098 2024-03-27T08:21:01.256355+0000 2024-03-26T08:40:09.929 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6801/4108777098 2024-03-27T08:21:01.256355+0000 2024-03-26T08:40:09.929 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/1885244665 2024-03-27T08:21:01.256355+0000 2024-03-26T08:40:09.929 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/298320867 2024-03-27T08:21:01.256355+0000 2024-03-26T08:40:09.929 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:0/1301092625 2024-03-27T08:21:36.149092+0000 2024-03-26T08:40:09.929 INFO:teuthology.orchestra.run.smithi043.stderr:listed 10 entries 2024-03-26T08:40:09.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.939+0000 7fa426000640 1 -- 172.21.15.43:0/1376253731 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa4040a2880 msgr2=0x7fa4040a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:09.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.939+0000 7fa426000640 1 --2- 172.21.15.43:0/1376253731 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa4040a2880 0x7fa4040a4d40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa42019aec0 tx=0x7fa408005f90 comp rx=0 tx=0).stop 2024-03-26T08:40:09.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.939+0000 7fa426000640 1 -- 172.21.15.43:0/1376253731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa420102000 msgr2=0x7fa4201999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:09.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.939+0000 7fa426000640 1 --2- 172.21.15.43:0/1376253731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa420102000 0x7fa4201999b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa410006010 tx=0x7fa410005e40 comp rx=0 tx=0).stop 2024-03-26T08:40:09.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.940+0000 7fa426000640 1 -- 172.21.15.43:0/1376253731 shutdown_connections 2024-03-26T08:40:09.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.940+0000 7fa426000640 1 --2- 172.21.15.43:0/1376253731 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fa4040a2880 0x7fa4040a4d40 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:09.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.940+0000 7fa426000640 1 --2- 172.21.15.43:0/1376253731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa420107260 0x7fa4201a0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:09.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.940+0000 7fa426000640 1 --2- 172.21.15.43:0/1376253731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa420104930 0x7fa420199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:09.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.940+0000 7fa426000640 1 --2- 172.21.15.43:0/1376253731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa420102000 0x7fa4201999b0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:09.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.940+0000 7fa426000640 1 -- 172.21.15.43:0/1376253731 >> 172.21.15.43:0/1376253731 conn(0x7fa4200fba10 msgr2=0x7fa4201016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:09.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.940+0000 7fa426000640 1 -- 172.21.15.43:0/1376253731 shutdown_connections 2024-03-26T08:40:09.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:09.940+0000 7fa426000640 1 -- 172.21.15.43:0/1376253731 wait complete. 2024-03-26T08:40:09.950 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2024-03-26T08:40:09.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:40:09.950 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-26T08:40:10.048 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[83304]: fuse finished with error 0 and tester_r 0 2024-03-26T08:40:10.049 INFO:teuthology.orchestra.run:waiting for 300 2024-03-26T08:40:10.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:09 smithi183 ceph-mon[35818]: pgmap v640: 129 pgs: 129 active+clean; 992 KiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.1 KiB/s wr, 216 op/s 2024-03-26T08:40:10.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:09 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3318933289' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:40:10.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:09 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2564746066' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T08:40:10.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:09 smithi052 ceph-mon[35071]: pgmap v640: 129 pgs: 129 active+clean; 992 KiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.1 KiB/s wr, 216 op/s 2024-03-26T08:40:10.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:09 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3318933289' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:40:10.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:09 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2564746066' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T08:40:10.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:09 smithi043 ceph-mon[31929]: pgmap v640: 129 pgs: 129 active+clean; 992 KiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.1 KiB/s wr, 216 op/s 2024-03-26T08:40:10.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:09 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3318933289' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:40:10.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:09 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2564746066' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T08:40:11.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:10 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1376253731' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T08:40:11.304 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:10 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1376253731' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T08:40:11.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:10 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1376253731' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T08:40:12.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:11 smithi183 ceph-mon[35818]: pgmap v641: 129 pgs: 129 active+clean; 993 KiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.4 KiB/s wr, 202 op/s 2024-03-26T08:40:12.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:11 smithi052 ceph-mon[35071]: pgmap v641: 129 pgs: 129 active+clean; 993 KiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.4 KiB/s wr, 202 op/s 2024-03-26T08:40:12.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:11 smithi043 ceph-mon[31929]: pgmap v641: 129 pgs: 129 active+clean; 993 KiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.4 KiB/s wr, 202 op/s 2024-03-26T08:40:14.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:13 smithi183 ceph-mon[35818]: pgmap v642: 129 pgs: 129 active+clean; 993 KiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 82 op/s 2024-03-26T08:40:14.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:13 smithi052 ceph-mon[35071]: pgmap v642: 129 pgs: 129 active+clean; 993 KiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 82 op/s 2024-03-26T08:40:14.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:13 smithi043 ceph-mon[31929]: pgmap v642: 129 pgs: 129 active+clean; 993 KiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 82 op/s 2024-03-26T08:40:16.151 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-03-26T08:40:16.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:40:16.152 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-26T08:40:16.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:40:16.217 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-03-26T08:40:16.275 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-26T08:40:16.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:40:16.276 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-26T08:40:16.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:15 smithi183 ceph-mon[35818]: pgmap v643: 129 pgs: 129 active+clean; 993 KiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 17 op/s 2024-03-26T08:40:16.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:15 smithi052 ceph-mon[35071]: pgmap v643: 129 pgs: 129 active+clean; 993 KiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 17 op/s 2024-03-26T08:40:16.367 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:15 smithi043 ceph-mon[31929]: pgmap v643: 129 pgs: 129 active+clean; 993 KiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 17 op/s 2024-03-26T08:40:16.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:40:16.383 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link delete ceph-brx 2024-03-26T08:40:16.500 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-26T08:40:16.511 INFO:tasks.cephadm:Teardown begin 2024-03-26T08:40:16.511 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-26T08:40:16.512 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:40:16.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.716+0000 7fcfd6c00640 1 -- 172.21.15.43:0/1785624991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfd01038e0 msgr2=0x7fcfd0103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:16.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.716+0000 7fcfd6c00640 1 --2- 172.21.15.43:0/1785624991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfd01038e0 0x7fcfd0103d00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009900 tx=0x7fcfc002bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:16.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.716+0000 7fcfd6c00640 1 -- 172.21.15.43:0/1785624991 shutdown_connections 2024-03-26T08:40:16.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.716+0000 7fcfd6c00640 1 --2- 172.21.15.43:0/1785624991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfd0105490 0x7fcfd010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.716+0000 7fcfd6c00640 1 --2- 172.21.15.43:0/1785624991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcfd0104ad0 0x7fcfd0104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.716+0000 7fcfd6c00640 1 --2- 172.21.15.43:0/1785624991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfd01038e0 0x7fcfd0103d00 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.716+0000 7fcfd6c00640 1 -- 172.21.15.43:0/1785624991 >> 172.21.15.43:0/1785624991 conn(0x7fcfd00fee90 msgr2=0x7fcfd01012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:16.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.716+0000 7fcfd6c00640 1 -- 172.21.15.43:0/1785624991 shutdown_connections 2024-03-26T08:40:16.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.716+0000 7fcfd6c00640 1 -- 172.21.15.43:0/1785624991 wait complete. 2024-03-26T08:40:16.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.717+0000 7fcfd6c00640 1 Processor -- start 2024-03-26T08:40:16.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.717+0000 7fcfd6c00640 1 -- start start 2024-03-26T08:40:16.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.718+0000 7fcfd6c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcfd01038e0 0x7fcfd0199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:16.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.718+0000 7fcfd6c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfd0104ad0 0x7fcfd0199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:16.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.718+0000 7fcfd6c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfd0105490 0x7fcfd01a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:16.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.718+0000 7fcfd6c00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd010f460 con 0x7fcfd0105490 2024-03-26T08:40:16.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.718+0000 7fcfd6c00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd010f2e0 con 0x7fcfd01038e0 2024-03-26T08:40:16.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.718+0000 7fcfd6c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd010f5e0 con 0x7fcfd0104ad0 2024-03-26T08:40:16.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.718+0000 7fcfd4c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfd0105490 0x7fcfd01a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:16.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.718+0000 7fcfd4c00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfd0105490 0x7fcfd01a1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59236/0 (socket says 172.21.15.43:59236) 2024-03-26T08:40:16.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.718+0000 7fcfd4c00640 1 -- 172.21.15.43:0/285288191 learned_addr learned my addr 172.21.15.43:0/285288191 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:40:16.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.718+0000 7fcfcf400640 1 --2- 172.21.15.43:0/285288191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfd0104ad0 0x7fcfd0199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:16.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.718+0000 7fcfcfe00640 1 --2- 172.21.15.43:0/285288191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcfd01038e0 0x7fcfd0199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:16.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.718+0000 7fcfd4c00640 1 -- 172.21.15.43:0/285288191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfd0104ad0 msgr2=0x7fcfd0199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:16.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.718+0000 7fcfd4c00640 1 --2- 172.21.15.43:0/285288191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfd0104ad0 0x7fcfd0199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.718+0000 7fcfd4c00640 1 -- 172.21.15.43:0/285288191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcfd01038e0 msgr2=0x7fcfd0199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:16.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.718+0000 7fcfd4c00640 1 --2- 172.21.15.43:0/285288191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcfd01038e0 0x7fcfd0199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.718+0000 7fcfd4c00640 1 -- 172.21.15.43:0/285288191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd01a16a0 con 0x7fcfd0105490 2024-03-26T08:40:16.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.718+0000 7fcfcfe00640 1 --2- 172.21.15.43:0/285288191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcfd01038e0 0x7fcfd0199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:40:16.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.719+0000 7fcfd4c00640 1 --2- 172.21.15.43:0/285288191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfd0105490 0x7fcfd01a1030 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800eaa0 tx=0x7fcfc800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:16.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.719+0000 7fcfccc00640 1 -- 172.21.15.43:0/285288191 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc800cdb0 con 0x7fcfd0105490 2024-03-26T08:40:16.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.719+0000 7fcfd6c00640 1 -- 172.21.15.43:0/285288191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd01a1990 con 0x7fcfd0105490 2024-03-26T08:40:16.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.719+0000 7fcfccc00640 1 -- 172.21.15.43:0/285288191 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcfc8005020 con 0x7fcfd0105490 2024-03-26T08:40:16.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.719+0000 7fcfccc00640 1 -- 172.21.15.43:0/285288191 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc80116d0 con 0x7fcfd0105490 2024-03-26T08:40:16.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.719+0000 7fcfd6c00640 1 -- 172.21.15.43:0/285288191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd01a1f30 con 0x7fcfd0105490 2024-03-26T08:40:16.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.721+0000 7fcfd6c00640 1 -- 172.21.15.43:0/285288191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd010aba0 con 0x7fcfd0105490 2024-03-26T08:40:16.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.723+0000 7fcfccc00640 1 -- 172.21.15.43:0/285288191 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7fcfc80051d0 con 0x7fcfd0105490 2024-03-26T08:40:16.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.724+0000 7fcfccc00640 1 --2- 172.21.15.43:0/285288191 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fcfb40a2800 0x7fcfb40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:16.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.725+0000 7fcfccc00640 1 -- 172.21.15.43:0/285288191 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7fcfc80cf070 con 0x7fcfd0105490 2024-03-26T08:40:16.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.725+0000 7fcfcfe00640 1 --2- 172.21.15.43:0/285288191 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fcfb40a2800 0x7fcfb40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:16.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.726+0000 7fcfcfe00640 1 --2- 172.21.15.43:0/285288191 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fcfb40a2800 0x7fcfb40a4cc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0036040 tx=0x7fcfc0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:16.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.729+0000 7fcfccc00640 1 -- 172.21.15.43:0/285288191 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcfc8016030 con 0x7fcfd0105490 2024-03-26T08:40:16.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.969+0000 7fcfd6c00640 1 -- 172.21.15.43:0/285288191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfd002d070 con 0x7fcfd0105490 2024-03-26T08:40:16.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.972+0000 7fcfccc00640 1 -- 172.21.15.43:0/285288191 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11331 (secure 0 0 0) 0x7fcfc809bd30 con 0x7fcfd0105490 2024-03-26T08:40:16.964 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:40:16.964 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1452206807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1452206807},{"type":"v1","addr":"172.21.15.43:6835","nonce":1452206807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3151775467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3151775467},{"type":"v1","addr":"172.21.15.43:6837","nonce":3151775467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/1284655682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":1284655682},{"type":"v1","addr":"172.21.15.43:6839","nonce":1284655682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3081085787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3081085787},{"type":"v1","addr":"172.21.15.43:6841","nonce":3081085787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1842295306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1842295306},{"type":"v1","addr":"172.21.15.183:6835","nonce":1842295306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2267781666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2267781666},{"type":"v1","addr":"172.21.15.183:6837","nonce":2267781666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3572429948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3572429948},{"type":"v1","addr":"172.21.15.183:6839","nonce":3572429948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34448,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/1108327179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108327179},{"type":"v1","addr":"172.21.15.52:6833","nonce":1108327179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34487,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/742057410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":742057410},{"type":"v1","addr":"172.21.15.52:6835","nonce":742057410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:31:21.819689+0000","modified":"2024-03-26T08:31:48.080233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34517":{"gid":34517,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/748239918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":748239918},{"type":"v1","addr":"172.21.15.52:6837","nonce":748239918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34559":{"gid":34559,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.52:6839/2551965101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2551965101},{"type":"v1","addr":"172.21.15.52:6839","nonce":2551965101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34559,"qdb_cluster":[34559]},"id":1}]} 2024-03-26T08:40:16.964 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 21 2024-03-26T08:40:16.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.974+0000 7fcfd6c00640 1 -- 172.21.15.43:0/285288191 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fcfb40a2800 msgr2=0x7fcfb40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:16.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.974+0000 7fcfd6c00640 1 --2- 172.21.15.43:0/285288191 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fcfb40a2800 0x7fcfb40a4cc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0036040 tx=0x7fcfc0005fd0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.974+0000 7fcfd6c00640 1 -- 172.21.15.43:0/285288191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfd0105490 msgr2=0x7fcfd01a1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:16.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.974+0000 7fcfd6c00640 1 --2- 172.21.15.43:0/285288191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfd0105490 0x7fcfd01a1030 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800eaa0 tx=0x7fcfc800ef60 comp rx=0 tx=0).stop 2024-03-26T08:40:16.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.975+0000 7fcfd6c00640 1 -- 172.21.15.43:0/285288191 shutdown_connections 2024-03-26T08:40:16.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.975+0000 7fcfd6c00640 1 --2- 172.21.15.43:0/285288191 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7fcfb40a2800 0x7fcfb40a4cc0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.975+0000 7fcfd6c00640 1 --2- 172.21.15.43:0/285288191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfd0105490 0x7fcfd01a1030 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.975+0000 7fcfd6c00640 1 --2- 172.21.15.43:0/285288191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfd0104ad0 0x7fcfd0199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.975+0000 7fcfd6c00640 1 --2- 172.21.15.43:0/285288191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcfd01038e0 0x7fcfd0199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.975+0000 7fcfd6c00640 1 -- 172.21.15.43:0/285288191 >> 172.21.15.43:0/285288191 conn(0x7fcfd00fee90 msgr2=0x7fcfd0107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:16.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.975+0000 7fcfd6c00640 1 -- 172.21.15.43:0/285288191 shutdown_connections 2024-03-26T08:40:16.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:16.975+0000 7fcfd6c00640 1 -- 172.21.15.43:0/285288191 wait complete. 2024-03-26T08:40:16.986 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:40:17.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.210+0000 7f08dd800640 1 -- 172.21.15.43:0/1585711570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d81052a0 msgr2=0x7f08d810bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:17.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.210+0000 7f08dd800640 1 --2- 172.21.15.43:0/1585711570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d81052a0 0x7f08d810bb30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f08d0009ca0 tx=0x7f08d002c100 comp rx=0 tx=0).stop 2024-03-26T08:40:17.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.211+0000 7f08dd800640 1 -- 172.21.15.43:0/1585711570 shutdown_connections 2024-03-26T08:40:17.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.211+0000 7f08dd800640 1 --2- 172.21.15.43:0/1585711570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d81052a0 0x7f08d810bb30 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.211+0000 7f08dd800640 1 --2- 172.21.15.43:0/1585711570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08d81048e0 0x7f08d8104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.211+0000 7f08dd800640 1 --2- 172.21.15.43:0/1585711570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08d81036f0 0x7f08d8103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.211+0000 7f08dd800640 1 -- 172.21.15.43:0/1585711570 >> 172.21.15.43:0/1585711570 conn(0x7f08d80fec80 msgr2=0x7f08d81010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:17.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.211+0000 7f08dd800640 1 -- 172.21.15.43:0/1585711570 shutdown_connections 2024-03-26T08:40:17.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.211+0000 7f08dd800640 1 -- 172.21.15.43:0/1585711570 wait complete. 2024-03-26T08:40:17.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.212+0000 7f08dd800640 1 Processor -- start 2024-03-26T08:40:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.212+0000 7f08dd800640 1 -- start start 2024-03-26T08:40:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.212+0000 7f08dd800640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08d81036f0 0x7f08d8199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.212+0000 7f08dd800640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d81048e0 0x7f08d8199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.212+0000 7f08dd800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08d81052a0 0x7f08d81a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.212+0000 7f08dd800640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d810f200 con 0x7f08d81048e0 2024-03-26T08:40:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.212+0000 7f08dd800640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d810f080 con 0x7f08d81036f0 2024-03-26T08:40:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.212+0000 7f08dd800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d810f380 con 0x7f08d81052a0 2024-03-26T08:40:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.212+0000 7f08cf400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d81048e0 0x7f08d8199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.212+0000 7f08cf400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d81048e0 0x7f08d8199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59250/0 (socket says 172.21.15.43:59250) 2024-03-26T08:40:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.212+0000 7f08cf400640 1 -- 172.21.15.43:0/530495773 learned_addr learned my addr 172.21.15.43:0/530495773 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:40:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.213+0000 7f08d7e00640 1 --2- 172.21.15.43:0/530495773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08d81036f0 0x7f08d8199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:17.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.213+0000 7f08dce00640 1 --2- 172.21.15.43:0/530495773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08d81052a0 0x7f08d81a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:17.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.213+0000 7f08cf400640 1 -- 172.21.15.43:0/530495773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08d81052a0 msgr2=0x7f08d81a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:17.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.213+0000 7f08cf400640 1 --2- 172.21.15.43:0/530495773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08d81052a0 0x7f08d81a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.213+0000 7f08cf400640 1 -- 172.21.15.43:0/530495773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08d81036f0 msgr2=0x7f08d8199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:17.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.213+0000 7f08cf400640 1 --2- 172.21.15.43:0/530495773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08d81036f0 0x7f08d8199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.213+0000 7f08cf400640 1 -- 172.21.15.43:0/530495773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d81a13a0 con 0x7f08d81048e0 2024-03-26T08:40:17.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.213+0000 7f08dce00640 1 --2- 172.21.15.43:0/530495773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08d81052a0 0x7f08d81a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:40:17.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.213+0000 7f08cf400640 1 --2- 172.21.15.43:0/530495773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d81048e0 0x7f08d8199cc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f08c000c920 tx=0x7f08c000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:17.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.213+0000 7f08d5600640 1 -- 172.21.15.43:0/530495773 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c0007c70 con 0x7f08d81048e0 2024-03-26T08:40:17.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.213+0000 7f08d5600640 1 -- 172.21.15.43:0/530495773 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f08c0007e10 con 0x7f08d81048e0 2024-03-26T08:40:17.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.213+0000 7f08dd800640 1 -- 172.21.15.43:0/530495773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d81a1690 con 0x7f08d81048e0 2024-03-26T08:40:17.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.213+0000 7f08d5600640 1 -- 172.21.15.43:0/530495773 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c0018820 con 0x7f08d81048e0 2024-03-26T08:40:17.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.214+0000 7f08dd800640 1 -- 172.21.15.43:0/530495773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d81a1be0 con 0x7f08d81048e0 2024-03-26T08:40:17.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.215+0000 7f08dd800640 1 -- 172.21.15.43:0/530495773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d810a940 con 0x7f08d81048e0 2024-03-26T08:40:17.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.216+0000 7f08d5600640 1 -- 172.21.15.43:0/530495773 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f08c00189c0 con 0x7f08d81048e0 2024-03-26T08:40:17.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.218+0000 7f08d5600640 1 --2- 172.21.15.43:0/530495773 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f08bc0a2800 0x7f08bc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:17.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.218+0000 7f08d5600640 1 -- 172.21.15.43:0/530495773 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f08c00cefe0 con 0x7f08d81048e0 2024-03-26T08:40:17.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.218+0000 7f08d7e00640 1 --2- 172.21.15.43:0/530495773 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f08bc0a2800 0x7f08bc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:17.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.219+0000 7f08d7e00640 1 --2- 172.21.15.43:0/530495773 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f08bc0a2800 0x7f08bc0a4cc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f08c80058c0 tx=0x7f08c8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:17.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.226+0000 7f08d5600640 1 -- 172.21.15.43:0/530495773 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f08c0014030 con 0x7f08d81048e0 2024-03-26T08:40:17.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.469+0000 7f08dd800640 1 -- 172.21.15.43:0/530495773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08d802d070 con 0x7f08d81048e0 2024-03-26T08:40:17.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.470+0000 7f08d5600640 1 -- 172.21.15.43:0/530495773 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11331 (secure 0 0 0) 0x7f08c009bd00 con 0x7f08d81048e0 2024-03-26T08:40:17.462 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:40:17.462 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1452206807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":1452206807},{"type":"v1","addr":"172.21.15.43:6835","nonce":1452206807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/3151775467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":3151775467},{"type":"v1","addr":"172.21.15.43:6837","nonce":3151775467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6839/1284655682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6838","nonce":1284655682},{"type":"v1","addr":"172.21.15.43:6839","nonce":1284655682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14664,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6841/3081085787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6840","nonce":3081085787},{"type":"v1","addr":"172.21.15.43:6841","nonce":3081085787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24409,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3522741573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3522741573},{"type":"v1","addr":"172.21.15.183:6833","nonce":3522741573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1842295306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1842295306},{"type":"v1","addr":"172.21.15.183:6835","nonce":1842295306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2267781666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2267781666},{"type":"v1","addr":"172.21.15.183:6837","nonce":2267781666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24511,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3572429948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3572429948},{"type":"v1","addr":"172.21.15.183:6839","nonce":3572429948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34448,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/1108327179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":1108327179},{"type":"v1","addr":"172.21.15.52:6833","nonce":1108327179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34487,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/742057410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":742057410},{"type":"v1","addr":"172.21.15.52:6835","nonce":742057410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:31:21.819689+0000","modified":"2024-03-26T08:31:48.080233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34517":{"gid":34517,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/748239918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":748239918},{"type":"v1","addr":"172.21.15.52:6837","nonce":748239918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34559":{"gid":34559,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.52:6839/2551965101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2551965101},{"type":"v1","addr":"172.21.15.52:6839","nonce":2551965101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34559,"qdb_cluster":[34559]},"id":1}]} 2024-03-26T08:40:17.463 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 21 2024-03-26T08:40:17.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.473+0000 7f08dd800640 1 -- 172.21.15.43:0/530495773 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f08bc0a2800 msgr2=0x7f08bc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:17.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.473+0000 7f08dd800640 1 --2- 172.21.15.43:0/530495773 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f08bc0a2800 0x7f08bc0a4cc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f08c80058c0 tx=0x7f08c8005850 comp rx=0 tx=0).stop 2024-03-26T08:40:17.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.473+0000 7f08dd800640 1 -- 172.21.15.43:0/530495773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d81048e0 msgr2=0x7f08d8199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:17.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.473+0000 7f08dd800640 1 --2- 172.21.15.43:0/530495773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d81048e0 0x7f08d8199cc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f08c000c920 tx=0x7f08c000cde0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.473+0000 7f08dd800640 1 -- 172.21.15.43:0/530495773 shutdown_connections 2024-03-26T08:40:17.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.473+0000 7f08dd800640 1 --2- 172.21.15.43:0/530495773 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f08bc0a2800 0x7f08bc0a4cc0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.473+0000 7f08dd800640 1 --2- 172.21.15.43:0/530495773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08d81052a0 0x7f08d81a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.473+0000 7f08dd800640 1 --2- 172.21.15.43:0/530495773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d81048e0 0x7f08d8199cc0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.473+0000 7f08dd800640 1 --2- 172.21.15.43:0/530495773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08d81036f0 0x7f08d8199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.473+0000 7f08dd800640 1 -- 172.21.15.43:0/530495773 >> 172.21.15.43:0/530495773 conn(0x7f08d80fec80 msgr2=0x7f08d8107b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:17.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.473+0000 7f08dd800640 1 -- 172.21.15.43:0/530495773 shutdown_connections 2024-03-26T08:40:17.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.473+0000 7f08dd800640 1 -- 172.21.15.43:0/530495773 wait complete. 2024-03-26T08:40:17.483 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:40:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.697+0000 7f87a8a00640 1 -- 172.21.15.43:0/3648556097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87a41048a0 msgr2=0x7f87a4106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.697+0000 7f87a8a00640 1 --2- 172.21.15.43:0/3648556097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87a41048a0 0x7f87a4106c90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f878c009900 tx=0x7f878c02bc40 comp rx=0 tx=0).stop 2024-03-26T08:40:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.697+0000 7f87a8a00640 1 -- 172.21.15.43:0/3648556097 shutdown_connections 2024-03-26T08:40:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.697+0000 7f87a8a00640 1 --2- 172.21.15.43:0/3648556097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87a41071d0 0x7f87a4109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.697+0000 7f87a8a00640 1 --2- 172.21.15.43:0/3648556097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87a41048a0 0x7f87a4106c90 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.697+0000 7f87a8a00640 1 --2- 172.21.15.43:0/3648556097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87a4101f70 0x7f87a4104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.697+0000 7f87a8a00640 1 -- 172.21.15.43:0/3648556097 >> 172.21.15.43:0/3648556097 conn(0x7f87a40fb980 msgr2=0x7f87a40fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.697+0000 7f87a8a00640 1 -- 172.21.15.43:0/3648556097 shutdown_connections 2024-03-26T08:40:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.698+0000 7f87a8a00640 1 -- 172.21.15.43:0/3648556097 wait complete. 2024-03-26T08:40:17.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.698+0000 7f87a8a00640 1 Processor -- start 2024-03-26T08:40:17.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.699+0000 7f87a8a00640 1 -- start start 2024-03-26T08:40:17.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.699+0000 7f87a8a00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87a4101f70 0x7f87a4197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:17.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.699+0000 7f87a8a00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87a41048a0 0x7f87a4197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:17.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.699+0000 7f87a8a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87a41071d0 0x7f87a419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:17.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.699+0000 7f87a8a00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a410d080 con 0x7f87a41048a0 2024-03-26T08:40:17.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.699+0000 7f87a8a00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a410cf00 con 0x7f87a4101f70 2024-03-26T08:40:17.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.699+0000 7f87a8a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a410d200 con 0x7f87a41071d0 2024-03-26T08:40:17.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.700+0000 7f8799e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87a41048a0 0x7f87a4197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:17.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.700+0000 7f8799e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87a41048a0 0x7f87a4197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59268/0 (socket says 172.21.15.43:59268) 2024-03-26T08:40:17.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.700+0000 7f87a3200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87a41071d0 0x7f87a419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:17.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.700+0000 7f8799e00640 1 -- 172.21.15.43:0/752710101 learned_addr learned my addr 172.21.15.43:0/752710101 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:40:17.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.700+0000 7f87a2800640 1 --2- 172.21.15.43:0/752710101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87a4101f70 0x7f87a4197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:17.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.700+0000 7f8799e00640 1 -- 172.21.15.43:0/752710101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87a41071d0 msgr2=0x7f87a419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:17.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.700+0000 7f8799e00640 1 --2- 172.21.15.43:0/752710101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87a41071d0 0x7f87a419ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.700+0000 7f8799e00640 1 -- 172.21.15.43:0/752710101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87a4101f70 msgr2=0x7f87a4197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:17.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.700+0000 7f8799e00640 1 --2- 172.21.15.43:0/752710101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87a4101f70 0x7f87a4197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.700+0000 7f8799e00640 1 -- 172.21.15.43:0/752710101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a419f380 con 0x7f87a41048a0 2024-03-26T08:40:17.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.700+0000 7f8799e00640 1 --2- 172.21.15.43:0/752710101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87a41048a0 0x7f87a4197ca0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f878c005ab0 tx=0x7f878c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:17.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.701+0000 7f879be00640 1 -- 172.21.15.43:0/752710101 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c043020 con 0x7f87a41048a0 2024-03-26T08:40:17.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.701+0000 7f879be00640 1 -- 172.21.15.43:0/752710101 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f878c0054c0 con 0x7f87a41048a0 2024-03-26T08:40:17.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.701+0000 7f87a8a00640 1 -- 172.21.15.43:0/752710101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a419f610 con 0x7f87a41048a0 2024-03-26T08:40:17.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.701+0000 7f879be00640 1 -- 172.21.15.43:0/752710101 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c003d10 con 0x7f87a41048a0 2024-03-26T08:40:17.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.701+0000 7f87a8a00640 1 -- 172.21.15.43:0/752710101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a419fbb0 con 0x7f87a41048a0 2024-03-26T08:40:17.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.702+0000 7f87a8a00640 1 -- 172.21.15.43:0/752710101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a40fd470 con 0x7f87a41048a0 2024-03-26T08:40:17.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.704+0000 7f879be00640 1 -- 172.21.15.43:0/752710101 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f878c046020 con 0x7f87a41048a0 2024-03-26T08:40:17.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.706+0000 7f879be00640 1 --2- 172.21.15.43:0/752710101 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f87880a2880 0x7f87880a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:17.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.706+0000 7f879be00640 1 -- 172.21.15.43:0/752710101 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f878c0f1130 con 0x7f87a41048a0 2024-03-26T08:40:17.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.706+0000 7f87a2800640 1 --2- 172.21.15.43:0/752710101 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f87880a2880 0x7f87880a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:17.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.707+0000 7f87a2800640 1 --2- 172.21.15.43:0/752710101 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f87880a2880 0x7f87880a4d40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8794005fd0 tx=0x7f8794005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:17.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.710+0000 7f879be00640 1 -- 172.21.15.43:0/752710101 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f878c04d020 con 0x7f87a41048a0 2024-03-26T08:40:17.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.915+0000 7f87a8a00640 1 -- 172.21.15.43:0/752710101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87a40622a0 con 0x7f87a41048a0 2024-03-26T08:40:17.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.920+0000 7f879be00640 1 -- 172.21.15.43:0/752710101 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24670 (secure 0 0 0) 0x7f878c0bddf0 con 0x7f87a41048a0 2024-03-26T08:40:17.913 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-26T08:40:17.913 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":94,"fsid":"88759420-eb49-11ee-95d0-87774f69a715","created":"2024-03-26T08:20:27.817107+0000","modified":"2024-03-26T08:32:38.256877+0000","last_up_change":"2024-03-26T08:30:10.520976+0000","last_in_change":"2024-03-26T08:29:46.394360+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:24:54.852427+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:31:19.924902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:31:21.323881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T08:31:24.951015+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"28670673-9193-4fa3-9cc0-c1a5a6b27284","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6803","nonce":936601146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6805","nonce":936601146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6809","nonce":936601146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":936601146},{"type":"v1","addr":"172.21.15.43:6807","nonce":936601146}]},"public_addr":"172.21.15.43:6803/936601146","cluster_addr":"172.21.15.43:6805/936601146","heartbeat_back_addr":"172.21.15.43:6809/936601146","heartbeat_front_addr":"172.21.15.43:6807/936601146","state":["exists","up"]},{"osd":1,"uuid":"fa8ad6b6-277a-42b8-b0f8-b36fc219de2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6801","nonce":3268412328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6803","nonce":3268412328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6807","nonce":3268412328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":3268412328},{"type":"v1","addr":"172.21.15.52:6805","nonce":3268412328}]},"public_addr":"172.21.15.52:6801/3268412328","cluster_addr":"172.21.15.52:6803/3268412328","heartbeat_back_addr":"172.21.15.52:6807/3268412328","heartbeat_front_addr":"172.21.15.52:6805/3268412328","state":["exists","up"]},{"osd":2,"uuid":"ec0039c3-f9bd-419a-81de-fa4418359808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6801","nonce":2103180908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6803","nonce":2103180908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6807","nonce":2103180908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2103180908},{"type":"v1","addr":"172.21.15.183:6805","nonce":2103180908}]},"public_addr":"172.21.15.183:6801/2103180908","cluster_addr":"172.21.15.183:6803/2103180908","heartbeat_back_addr":"172.21.15.183:6807/2103180908","heartbeat_front_addr":"172.21.15.183:6805/2103180908","state":["exists","up"]},{"osd":3,"uuid":"e7dcbf00-6586-4664-a69e-3cb0df8dfd30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6811","nonce":3722846033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6813","nonce":3722846033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6817","nonce":3722846033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3722846033},{"type":"v1","addr":"172.21.15.43:6815","nonce":3722846033}]},"public_addr":"172.21.15.43:6811/3722846033","cluster_addr":"172.21.15.43:6813/3722846033","heartbeat_back_addr":"172.21.15.43:6817/3722846033","heartbeat_front_addr":"172.21.15.43:6815/3722846033","state":["exists","up"]},{"osd":4,"uuid":"db089a49-5e98-432d-9687-cdc044114c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6809","nonce":4028436523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6811","nonce":4028436523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6815","nonce":4028436523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":4028436523},{"type":"v1","addr":"172.21.15.52:6813","nonce":4028436523}]},"public_addr":"172.21.15.52:6809/4028436523","cluster_addr":"172.21.15.52:6811/4028436523","heartbeat_back_addr":"172.21.15.52:6815/4028436523","heartbeat_front_addr":"172.21.15.52:6813/4028436523","state":["exists","up"]},{"osd":5,"uuid":"556ac804-02ea-423b-a599-55baf5be4ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6809","nonce":787132252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6811","nonce":787132252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6815","nonce":787132252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":787132252},{"type":"v1","addr":"172.21.15.183:6813","nonce":787132252}]},"public_addr":"172.21.15.183:6809/787132252","cluster_addr":"172.21.15.183:6811/787132252","heartbeat_back_addr":"172.21.15.183:6815/787132252","heartbeat_front_addr":"172.21.15.183:6813/787132252","state":["exists","up"]},{"osd":6,"uuid":"3de2a49d-6cef-4580-9d8a-f90cdfcecd9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6819","nonce":4206861158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6821","nonce":4206861158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6825","nonce":4206861158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":4206861158},{"type":"v1","addr":"172.21.15.43:6823","nonce":4206861158}]},"public_addr":"172.21.15.43:6819/4206861158","cluster_addr":"172.21.15.43:6821/4206861158","heartbeat_back_addr":"172.21.15.43:6825/4206861158","heartbeat_front_addr":"172.21.15.43:6823/4206861158","state":["exists","up"]},{"osd":7,"uuid":"a3b832f7-7d7d-4a62-b679-db6da98275d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6817","nonce":3452064146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6819","nonce":3452064146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6823","nonce":3452064146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":3452064146},{"type":"v1","addr":"172.21.15.52:6821","nonce":3452064146}]},"public_addr":"172.21.15.52:6817/3452064146","cluster_addr":"172.21.15.52:6819/3452064146","heartbeat_back_addr":"172.21.15.52:6823/3452064146","heartbeat_front_addr":"172.21.15.52:6821/3452064146","state":["exists","up"]},{"osd":8,"uuid":"ddc6b07f-d703-414d-a59a-02316376b13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6817","nonce":1023131725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6819","nonce":1023131725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6823","nonce":1023131725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1023131725},{"type":"v1","addr":"172.21.15.183:6821","nonce":1023131725}]},"public_addr":"172.21.15.183:6817/1023131725","cluster_addr":"172.21.15.183:6819/1023131725","heartbeat_back_addr":"172.21.15.183:6823/1023131725","heartbeat_front_addr":"172.21.15.183:6821/1023131725","state":["exists","up"]},{"osd":9,"uuid":"ac16bb0e-6232-45c8-b16a-4d878874c588","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6827","nonce":3935195657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6829","nonce":3935195657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6833","nonce":3935195657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3935195657},{"type":"v1","addr":"172.21.15.43:6831","nonce":3935195657}]},"public_addr":"172.21.15.43:6827/3935195657","cluster_addr":"172.21.15.43:6829/3935195657","heartbeat_back_addr":"172.21.15.43:6833/3935195657","heartbeat_front_addr":"172.21.15.43:6831/3935195657","state":["exists","up"]},{"osd":10,"uuid":"fd839188-f398-4cff-aa7b-36522ff1a8a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6825","nonce":377628320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6827","nonce":377628320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6831","nonce":377628320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":377628320},{"type":"v1","addr":"172.21.15.52:6829","nonce":377628320}]},"public_addr":"172.21.15.52:6825/377628320","cluster_addr":"172.21.15.52:6827/377628320","heartbeat_back_addr":"172.21.15.52:6831/377628320","heartbeat_front_addr":"172.21.15.52:6829/377628320","state":["exists","up"]},{"osd":11,"uuid":"a40282a0-0c64-49ee-b64e-6f3f8119e827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6825","nonce":1373086433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6827","nonce":1373086433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6831","nonce":1373086433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1373086433},{"type":"v1","addr":"172.21.15.183:6829","nonce":1373086433}]},"public_addr":"172.21.15.183:6825/1373086433","cluster_addr":"172.21.15.183:6827/1373086433","heartbeat_back_addr":"172.21.15.183:6831/1373086433","heartbeat_front_addr":"172.21.15.183:6829/1373086433","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:23:43.478905+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:17.573532+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:24:50.286610+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:25:27.407663+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:03.081294+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:26:36.057416+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:12.062013+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:27:47.032271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:21.023201+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:28:57.364739+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:29:34.039995+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:30:08.827577+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6801/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/2446912488":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/4240775295":"2024-03-27T08:21:36.149092+0000","172.21.15.43:0/510203140":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/4251957687":"2024-03-27T08:21:36.149092+0000","172.21.15.43:6800/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:6801/4108777098":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1885244665":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/298320867":"2024-03-27T08:21:01.256355+0000","172.21.15.43:0/1301092625":"2024-03-27T08:21:36.149092+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:40:17.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.923+0000 7f87a8a00640 1 -- 172.21.15.43:0/752710101 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f87880a2880 msgr2=0x7f87880a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:17.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.923+0000 7f87a8a00640 1 --2- 172.21.15.43:0/752710101 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f87880a2880 0x7f87880a4d40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8794005fd0 tx=0x7f8794005850 comp rx=0 tx=0).stop 2024-03-26T08:40:17.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.923+0000 7f87a8a00640 1 -- 172.21.15.43:0/752710101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87a41048a0 msgr2=0x7f87a4197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:17.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.923+0000 7f87a8a00640 1 --2- 172.21.15.43:0/752710101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87a41048a0 0x7f87a4197ca0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f878c005ab0 tx=0x7f878c0042f0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.923+0000 7f87a8a00640 1 -- 172.21.15.43:0/752710101 shutdown_connections 2024-03-26T08:40:17.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.923+0000 7f87a8a00640 1 --2- 172.21.15.43:0/752710101 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f87880a2880 0x7f87880a4d40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.923+0000 7f87a8a00640 1 --2- 172.21.15.43:0/752710101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87a41071d0 0x7f87a419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.923+0000 7f87a8a00640 1 --2- 172.21.15.43:0/752710101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87a41048a0 0x7f87a4197ca0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.923+0000 7f87a8a00640 1 --2- 172.21.15.43:0/752710101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87a4101f70 0x7f87a4197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.923+0000 7f87a8a00640 1 -- 172.21.15.43:0/752710101 >> 172.21.15.43:0/752710101 conn(0x7f87a40fb980 msgr2=0x7f87a4102990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:17.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.923+0000 7f87a8a00640 1 -- 172.21.15.43:0/752710101 shutdown_connections 2024-03-26T08:40:17.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:17.923+0000 7f87a8a00640 1 -- 172.21.15.43:0/752710101 wait complete. 2024-03-26T08:40:17.937 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-26T08:40:18.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.174+0000 7f8702e00640 1 -- 172.21.15.43:0/1887655754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86fc105490 msgr2=0x7f86fc10bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:18.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.174+0000 7f8702e00640 1 --2- 172.21.15.43:0/1887655754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86fc105490 0x7f86fc10bd20 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f86ec009900 tx=0x7f86ec02bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:18.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.174+0000 7f8702e00640 1 -- 172.21.15.43:0/1887655754 shutdown_connections 2024-03-26T08:40:18.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.174+0000 7f8702e00640 1 --2- 172.21.15.43:0/1887655754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86fc105490 0x7f86fc10bd20 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.174+0000 7f8702e00640 1 --2- 172.21.15.43:0/1887655754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86fc104ad0 0x7f86fc104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.174+0000 7f8702e00640 1 --2- 172.21.15.43:0/1887655754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86fc1038e0 0x7f86fc103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.174+0000 7f8702e00640 1 -- 172.21.15.43:0/1887655754 >> 172.21.15.43:0/1887655754 conn(0x7f86fc0fee50 msgr2=0x7f86fc101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:18.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.174+0000 7f8702e00640 1 -- 172.21.15.43:0/1887655754 shutdown_connections 2024-03-26T08:40:18.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.174+0000 7f8702e00640 1 -- 172.21.15.43:0/1887655754 wait complete. 2024-03-26T08:40:18.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.175+0000 7f8702e00640 1 Processor -- start 2024-03-26T08:40:18.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.175+0000 7f8702e00640 1 -- start start 2024-03-26T08:40:18.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.175+0000 7f8702e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86fc1038e0 0x7f86fc199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:18.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.175+0000 7f8702e00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86fc104ad0 0x7f86fc19a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:18.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.175+0000 7f8702e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86fc105490 0x7f86fc1a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:18.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.175+0000 7f8702e00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86fc10f460 con 0x7f86fc105490 2024-03-26T08:40:18.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.175+0000 7f8702e00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86fc10f2e0 con 0x7f86fc104ad0 2024-03-26T08:40:18.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.175+0000 7f8702e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86fc10f5e0 con 0x7f86fc1038e0 2024-03-26T08:40:18.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.176+0000 7f8700e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86fc105490 0x7f86fc1a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:18.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.176+0000 7f8700e00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86fc105490 0x7f86fc1a1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59284/0 (socket says 172.21.15.43:59284) 2024-03-26T08:40:18.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.176+0000 7f8700e00640 1 -- 172.21.15.43:0/1895841166 learned_addr learned my addr 172.21.15.43:0/1895841166 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:40:18.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.176+0000 7f86fb400640 1 --2- 172.21.15.43:0/1895841166 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86fc104ad0 0x7f86fc19a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:18.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.176+0000 7f86fbe00640 1 --2- 172.21.15.43:0/1895841166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86fc1038e0 0x7f86fc199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:18.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.176+0000 7f8700e00640 1 -- 172.21.15.43:0/1895841166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86fc1038e0 msgr2=0x7f86fc199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:18.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.176+0000 7f8700e00640 1 --2- 172.21.15.43:0/1895841166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86fc1038e0 0x7f86fc199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.176+0000 7f8700e00640 1 -- 172.21.15.43:0/1895841166 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86fc104ad0 msgr2=0x7f86fc19a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:18.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.176+0000 7f8700e00640 1 --2- 172.21.15.43:0/1895841166 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86fc104ad0 0x7f86fc19a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.176+0000 7f8700e00640 1 -- 172.21.15.43:0/1895841166 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86fc1a16f0 con 0x7f86fc105490 2024-03-26T08:40:18.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.177+0000 7f8700e00640 1 --2- 172.21.15.43:0/1895841166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86fc105490 0x7f86fc1a1080 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f86ec036040 tx=0x7f86ec005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:18.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.177+0000 7f86f8c00640 1 -- 172.21.15.43:0/1895841166 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec043020 con 0x7f86fc105490 2024-03-26T08:40:18.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.177+0000 7f86f8c00640 1 -- 172.21.15.43:0/1895841166 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f86ec004410 con 0x7f86fc105490 2024-03-26T08:40:18.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.177+0000 7f8702e00640 1 -- 172.21.15.43:0/1895841166 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86fc1a1980 con 0x7f86fc105490 2024-03-26T08:40:18.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.177+0000 7f86f8c00640 1 -- 172.21.15.43:0/1895841166 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec003b80 con 0x7f86fc105490 2024-03-26T08:40:18.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.177+0000 7f8702e00640 1 -- 172.21.15.43:0/1895841166 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86fc1a1f20 con 0x7f86fc105490 2024-03-26T08:40:18.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.178+0000 7f8702e00640 1 -- 172.21.15.43:0/1895841166 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86fc10ac00 con 0x7f86fc105490 2024-03-26T08:40:18.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.180+0000 7f86f8c00640 1 -- 172.21.15.43:0/1895841166 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f86ec046020 con 0x7f86fc105490 2024-03-26T08:40:18.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.181+0000 7f86f8c00640 1 --2- 172.21.15.43:0/1895841166 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f86e00a2880 0x7f86e00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:18.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.182+0000 7f86f8c00640 1 -- 172.21.15.43:0/1895841166 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9129+0+0 (secure 0 0 0) 0x7f86ec0f1470 con 0x7f86fc105490 2024-03-26T08:40:18.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.182+0000 7f86fbe00640 1 --2- 172.21.15.43:0/1895841166 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f86e00a2880 0x7f86e00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:18.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.183+0000 7f86fbe00640 1 --2- 172.21.15.43:0/1895841166 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f86e00a2880 0x7f86e00a4d40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f86fc104930 tx=0x7f86f4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:18.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.184+0000 7f86f8c00640 1 -- 172.21.15.43:0/1895841166 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f86ec04d020 con 0x7f86fc105490 2024-03-26T08:40:18.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:17 smithi183 ceph-mon[35818]: pgmap v644: 129 pgs: 129 active+clean; 993 KiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-03-26T08:40:18.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:17 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/285288191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:40:18.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:17 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/530495773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:40:18.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:17 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/752710101' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:18.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:17 smithi052 ceph-mon[35071]: pgmap v644: 129 pgs: 129 active+clean; 993 KiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-03-26T08:40:18.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:17 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/285288191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:40:18.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:17 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/530495773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:40:18.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:17 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/752710101' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:18.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:17 smithi043 ceph-mon[31929]: pgmap v644: 129 pgs: 129 active+clean; 993 KiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-03-26T08:40:18.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:17 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/285288191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:40:18.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:17 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/530495773' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:40:18.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:17 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/752710101' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:18.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.441+0000 7f8702e00640 1 -- 172.21.15.43:0/1895841166 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f86fc02d1f0 con 0x7f86fc105490 2024-03-26T08:40:18.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.447+0000 7f86f8c00640 1 -- 172.21.15.43:0/1895841166 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v22) v1 ==== 161+0+0 (secure 0 0 0) 0x7f86ec03e450 con 0x7f86fc105490 2024-03-26T08:40:18.439 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-26T08:40:18.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.450+0000 7f8702e00640 1 -- 172.21.15.43:0/1895841166 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f86e00a2880 msgr2=0x7f86e00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:18.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.450+0000 7f8702e00640 1 --2- 172.21.15.43:0/1895841166 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f86e00a2880 0x7f86e00a4d40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f86fc104930 tx=0x7f86f4009500 comp rx=0 tx=0).stop 2024-03-26T08:40:18.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.450+0000 7f8702e00640 1 -- 172.21.15.43:0/1895841166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86fc105490 msgr2=0x7f86fc1a1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:18.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.450+0000 7f8702e00640 1 --2- 172.21.15.43:0/1895841166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86fc105490 0x7f86fc1a1080 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f86ec036040 tx=0x7f86ec005dd0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.450+0000 7f8702e00640 1 -- 172.21.15.43:0/1895841166 shutdown_connections 2024-03-26T08:40:18.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.450+0000 7f8702e00640 1 --2- 172.21.15.43:0/1895841166 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f86e00a2880 0x7f86e00a4d40 secure :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f86fc104930 tx=0x7f86f4009500 comp rx=0 tx=0).stop 2024-03-26T08:40:18.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.450+0000 7f8702e00640 1 --2- 172.21.15.43:0/1895841166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86fc105490 0x7f86fc1a1080 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.450+0000 7f8702e00640 1 --2- 172.21.15.43:0/1895841166 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86fc104ad0 0x7f86fc19a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.450+0000 7f8702e00640 1 --2- 172.21.15.43:0/1895841166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86fc1038e0 0x7f86fc199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.450+0000 7f8702e00640 1 -- 172.21.15.43:0/1895841166 >> 172.21.15.43:0/1895841166 conn(0x7f86fc0fee50 msgr2=0x7f86fc107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:18.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.451+0000 7f8702e00640 1 -- 172.21.15.43:0/1895841166 shutdown_connections 2024-03-26T08:40:18.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.451+0000 7f8702e00640 1 -- 172.21.15.43:0/1895841166 wait complete. 2024-03-26T08:40:18.463 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-26T08:40:18.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.670+0000 7f04ef600640 1 -- 172.21.15.43:0/1085768765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04e8103260 msgr2=0x7f04e8109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:18.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.670+0000 7f04ef600640 1 --2- 172.21.15.43:0/1085768765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04e8103260 0x7f04e8109af0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f04e0009ca0 tx=0x7f04e002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.670+0000 7f04ef600640 1 -- 172.21.15.43:0/1085768765 shutdown_connections 2024-03-26T08:40:18.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.670+0000 7f04ef600640 1 --2- 172.21.15.43:0/1085768765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04e8103260 0x7f04e8109af0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.670+0000 7f04ef600640 1 --2- 172.21.15.43:0/1085768765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04e81028a0 0x7f04e8102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.670+0000 7f04ef600640 1 --2- 172.21.15.43:0/1085768765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04e81016b0 0x7f04e8101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.670+0000 7f04ef600640 1 -- 172.21.15.43:0/1085768765 >> 172.21.15.43:0/1085768765 conn(0x7f04e80fcc40 msgr2=0x7f04e80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:18.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.670+0000 7f04ef600640 1 -- 172.21.15.43:0/1085768765 shutdown_connections 2024-03-26T08:40:18.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.670+0000 7f04ef600640 1 -- 172.21.15.43:0/1085768765 wait complete. 2024-03-26T08:40:18.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.671+0000 7f04ef600640 1 Processor -- start 2024-03-26T08:40:18.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.671+0000 7f04ef600640 1 -- start start 2024-03-26T08:40:18.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.671+0000 7f04ef600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04e81016b0 0x7f04e8199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:18.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.671+0000 7f04ef600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04e81028a0 0x7f04e819a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:18.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.671+0000 7f04ef600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04e8103260 0x7f04e81a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:18.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.671+0000 7f04ef600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e810d1e0 con 0x7f04e81016b0 2024-03-26T08:40:18.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.671+0000 7f04ef600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e810d060 con 0x7f04e81028a0 2024-03-26T08:40:18.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.671+0000 7f04ef600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e810d360 con 0x7f04e8103260 2024-03-26T08:40:18.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.672+0000 7f04ecc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04e81016b0 0x7f04e8199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:18.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.672+0000 7f04ecc00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04e81016b0 0x7f04e8199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59300/0 (socket says 172.21.15.43:59300) 2024-03-26T08:40:18.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.672+0000 7f04ecc00640 1 -- 172.21.15.43:0/371618801 learned_addr learned my addr 172.21.15.43:0/371618801 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:40:18.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.672+0000 7f04dfe00640 1 --2- 172.21.15.43:0/371618801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04e81028a0 0x7f04e819a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:18.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.672+0000 7f04ed600640 1 --2- 172.21.15.43:0/371618801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04e8103260 0x7f04e81a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:18.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.672+0000 7f04ecc00640 1 -- 172.21.15.43:0/371618801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04e8103260 msgr2=0x7f04e81a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:18.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.672+0000 7f04ecc00640 1 --2- 172.21.15.43:0/371618801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04e8103260 0x7f04e81a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.672+0000 7f04ecc00640 1 -- 172.21.15.43:0/371618801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04e81028a0 msgr2=0x7f04e819a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:18.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.672+0000 7f04ecc00640 1 --2- 172.21.15.43:0/371618801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04e81028a0 0x7f04e819a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.672+0000 7f04ecc00640 1 -- 172.21.15.43:0/371618801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e81a16f0 con 0x7f04e81016b0 2024-03-26T08:40:18.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.673+0000 7f04ecc00640 1 --2- 172.21.15.43:0/371618801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04e81016b0 0x7f04e8199ad0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f04d000e9c0 tx=0x7f04d000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:18.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.673+0000 7f04e6000640 1 -- 172.21.15.43:0/371618801 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d000cd80 con 0x7f04e81016b0 2024-03-26T08:40:18.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.673+0000 7f04e6000640 1 -- 172.21.15.43:0/371618801 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f04d00035f0 con 0x7f04e81016b0 2024-03-26T08:40:18.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.673+0000 7f04e6000640 1 -- 172.21.15.43:0/371618801 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d0010690 con 0x7f04e81016b0 2024-03-26T08:40:18.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.673+0000 7f04ef600640 1 -- 172.21.15.43:0/371618801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04e81a19e0 con 0x7f04e81016b0 2024-03-26T08:40:18.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.673+0000 7f04ef600640 1 -- 172.21.15.43:0/371618801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e81a1f30 con 0x7f04e81016b0 2024-03-26T08:40:18.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.675+0000 7f04ef600640 1 -- 172.21.15.43:0/371618801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04e81a21c0 con 0x7f04e81016b0 2024-03-26T08:40:18.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.678+0000 7f04e6000640 1 -- 172.21.15.43:0/371618801 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f04d00037a0 con 0x7f04e81016b0 2024-03-26T08:40:18.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.680+0000 7f04e6000640 1 --2- 172.21.15.43:0/371618801 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f04c80a2800 0x7f04c80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:18.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.681+0000 7f04e6000640 1 -- 172.21.15.43:0/371618801 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9215+0+0 (secure 0 0 0) 0x7f04d00cf370 con 0x7f04e81016b0 2024-03-26T08:40:18.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.681+0000 7f04dfe00640 1 --2- 172.21.15.43:0/371618801 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f04c80a2800 0x7f04c80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:18.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.682+0000 7f04dfe00640 1 --2- 172.21.15.43:0/371618801 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f04c80a2800 0x7f04c80a4cc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f04d8005fd0 tx=0x7f04d8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:18.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.685+0000 7f04e6000640 1 -- 172.21.15.43:0/371618801 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f04d009bfe0 con 0x7f04e81016b0 2024-03-26T08:40:18.722 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:18 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-a[31924]: 2024-03-26T08:40:18.443+0000 7f85a6e00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-26T08:40:18.822 INFO:journalctl@ceph.mds.b.smithi052.stdout:Mar 26 08:40:18 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-b[65603]: ignoring --setuser ceph since I am not root 2024-03-26T08:40:18.822 INFO:journalctl@ceph.mds.b.smithi052.stdout:Mar 26 08:40:18 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-b[65603]: ignoring --setgroup ceph since I am not root 2024-03-26T08:40:18.822 INFO:journalctl@ceph.mds.b.smithi052.stdout:Mar 26 08:40:18 smithi052 ceph-mds[65607]: -- 172.21.15.52:0/1722938131 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5609d21464e0 con 0x5609d211cc00 2024-03-26T08:40:18.822 INFO:journalctl@ceph.mds.b.smithi052.stdout:Mar 26 08:40:18 smithi052 ceph-mds[65607]: -- 172.21.15.52:0/1722938131 <== mon.2 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5609d2149a40 con 0x5609d211cc00 2024-03-26T08:40:18.822 INFO:journalctl@ceph.mds.b.smithi052.stdout:Mar 26 08:40:18 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-b[65603]: starting mds.b at 2024-03-26T08:40:18.822 INFO:journalctl@ceph.mds.h.smithi052.stdout:Mar 26 08:40:18 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-h[65286]: ignoring --setuser ceph since I am not root 2024-03-26T08:40:18.823 INFO:journalctl@ceph.mds.h.smithi052.stdout:Mar 26 08:40:18 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-h[65286]: ignoring --setgroup ceph since I am not root 2024-03-26T08:40:18.823 INFO:journalctl@ceph.mds.h.smithi052.stdout:Mar 26 08:40:18 smithi052 ceph-mds[65290]: -- 172.21.15.52:0/3089636945 <== mon.2 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f8c86b5a40 con 0x55f8c8688c00 2024-03-26T08:40:18.823 INFO:journalctl@ceph.mds.h.smithi052.stdout:Mar 26 08:40:18 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-h[65286]: starting mds.h at 2024-03-26T08:40:18.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:18.923+0000 7f04ef600640 1 -- 172.21.15.43:0/371618801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f04e81a2390 con 0x7f04e81016b0 2024-03-26T08:40:19.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:18 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1895841166' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-26T08:40:19.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:18 smithi043 ceph-mon[31929]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-26T08:40:19.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:18 smithi043 ceph-mon[31929]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-26T08:40:19.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:18 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1895841166' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-26T08:40:19.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:18 smithi043 ceph-mon[31929]: osdmap e95: 12 total, 12 up, 12 in 2024-03-26T08:40:19.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:18 smithi043 ceph-mon[31929]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-03-26T08:40:19.151 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:18 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/371618801' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T08:40:19.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:18 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1895841166' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-26T08:40:19.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:18 smithi183 ceph-mon[35818]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-26T08:40:19.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:18 smithi183 ceph-mon[35818]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-26T08:40:19.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:18 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1895841166' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-26T08:40:19.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:18 smithi183 ceph-mon[35818]: osdmap e95: 12 total, 12 up, 12 in 2024-03-26T08:40:19.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:18 smithi183 ceph-mon[35818]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-03-26T08:40:19.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:18 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/371618801' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T08:40:19.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:18 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1895841166' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-26T08:40:19.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:18 smithi052 ceph-mon[35071]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-26T08:40:19.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:18 smithi052 ceph-mon[35071]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-26T08:40:19.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:18 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1895841166' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-26T08:40:19.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:18 smithi052 ceph-mon[35071]: osdmap e95: 12 total, 12 up, 12 in 2024-03-26T08:40:19.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:18 smithi052 ceph-mon[35071]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-03-26T08:40:19.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:18 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/371618801' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T08:40:19.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.457+0000 7f04e6000640 1 -- 172.21.15.43:0/371618801 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f04d009b8e0 con 0x7f04e81016b0 2024-03-26T08:40:19.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.460+0000 7f04ef600640 1 -- 172.21.15.43:0/371618801 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f04c80a2800 msgr2=0x7f04c80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:19.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.460+0000 7f04ef600640 1 --2- 172.21.15.43:0/371618801 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f04c80a2800 0x7f04c80a4cc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f04d8005fd0 tx=0x7f04d8005850 comp rx=0 tx=0).stop 2024-03-26T08:40:19.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.460+0000 7f04ef600640 1 -- 172.21.15.43:0/371618801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04e81016b0 msgr2=0x7f04e8199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:19.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.460+0000 7f04ef600640 1 --2- 172.21.15.43:0/371618801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04e81016b0 0x7f04e8199ad0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f04d000e9c0 tx=0x7f04d000ee80 comp rx=0 tx=0).stop 2024-03-26T08:40:19.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.460+0000 7f04ef600640 1 -- 172.21.15.43:0/371618801 shutdown_connections 2024-03-26T08:40:19.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.460+0000 7f04ef600640 1 --2- 172.21.15.43:0/371618801 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f04c80a2800 0x7f04c80a4cc0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:19.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.460+0000 7f04ef600640 1 --2- 172.21.15.43:0/371618801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04e8103260 0x7f04e81a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:19.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.460+0000 7f04ef600640 1 --2- 172.21.15.43:0/371618801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04e81028a0 0x7f04e819a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:19.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.460+0000 7f04ef600640 1 --2- 172.21.15.43:0/371618801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04e81016b0 0x7f04e8199ad0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:19.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.460+0000 7f04ef600640 1 -- 172.21.15.43:0/371618801 >> 172.21.15.43:0/371618801 conn(0x7f04e80fcc40 msgr2=0x7f04e80738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:19.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.461+0000 7f04ef600640 1 -- 172.21.15.43:0/371618801 shutdown_connections 2024-03-26T08:40:19.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.461+0000 7f04ef600640 1 -- 172.21.15.43:0/371618801 wait complete. 2024-03-26T08:40:19.472 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-26T08:40:19.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.696+0000 7f257be00640 1 -- 172.21.15.43:0/2981465373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f257c102790 msgr2=0x7f257c104c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:19.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.696+0000 7f257be00640 1 --2- 172.21.15.43:0/2981465373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f257c102790 0x7f257c104c00 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2568009900 tx=0x7f256802bc50 comp rx=0 tx=0).stop 2024-03-26T08:40:19.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.696+0000 7f257be00640 1 -- 172.21.15.43:0/2981465373 shutdown_connections 2024-03-26T08:40:19.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.696+0000 7f257be00640 1 --2- 172.21.15.43:0/2981465373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f257c105140 0x7f257c1055c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:19.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.696+0000 7f257be00640 1 --2- 172.21.15.43:0/2981465373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f257c102790 0x7f257c104c00 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:19.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.696+0000 7f257be00640 1 --2- 172.21.15.43:0/2981465373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f257c0ffe20 0x7f257c102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:19.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.696+0000 7f257be00640 1 -- 172.21.15.43:0/2981465373 >> 172.21.15.43:0/2981465373 conn(0x7f257c0fba20 msgr2=0x7f257c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:19.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.696+0000 7f257be00640 1 -- 172.21.15.43:0/2981465373 shutdown_connections 2024-03-26T08:40:19.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.696+0000 7f257be00640 1 -- 172.21.15.43:0/2981465373 wait complete. 2024-03-26T08:40:19.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.697+0000 7f257be00640 1 Processor -- start 2024-03-26T08:40:19.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.697+0000 7f257be00640 1 -- start start 2024-03-26T08:40:19.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.697+0000 7f257be00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f257c0ffe20 0x7f257c19ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:19.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.697+0000 7f257be00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f257c102790 0x7f257c19e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:19.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.697+0000 7f257be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f257c105140 0x7f257c1a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:19.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.697+0000 7f257be00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257c113720 con 0x7f257c0ffe20 2024-03-26T08:40:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.697+0000 7f257be00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257c1135a0 con 0x7f257c102790 2024-03-26T08:40:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.697+0000 7f257be00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257c1138a0 con 0x7f257c105140 2024-03-26T08:40:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.698+0000 7f257aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f257c0ffe20 0x7f257c19ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.698+0000 7f257aa00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f257c0ffe20 0x7f257c19ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59322/0 (socket says 172.21.15.43:59322) 2024-03-26T08:40:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.698+0000 7f257aa00640 1 -- 172.21.15.43:0/3627775847 learned_addr learned my addr 172.21.15.43:0/3627775847 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:40:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.698+0000 7f257a000640 1 --2- 172.21.15.43:0/3627775847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f257c102790 0x7f257c19e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.698+0000 7f257aa00640 1 -- 172.21.15.43:0/3627775847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f257c105140 msgr2=0x7f257c1a5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.698+0000 7f257aa00640 1 --2- 172.21.15.43:0/3627775847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f257c105140 0x7f257c1a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.698+0000 7f257aa00640 1 -- 172.21.15.43:0/3627775847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f257c102790 msgr2=0x7f257c19e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.698+0000 7f257aa00640 1 --2- 172.21.15.43:0/3627775847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f257c102790 0x7f257c19e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.698+0000 7f257aa00640 1 -- 172.21.15.43:0/3627775847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257c1a59d0 con 0x7f257c0ffe20 2024-03-26T08:40:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.698+0000 7f257aa00640 1 --2- 172.21.15.43:0/3627775847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f257c0ffe20 0x7f257c19ddc0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f256c00d770 tx=0x7f256c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.699+0000 7f2573400640 1 -- 172.21.15.43:0/3627775847 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256c017070 con 0x7f257c0ffe20 2024-03-26T08:40:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.699+0000 7f257be00640 1 -- 172.21.15.43:0/3627775847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f257c1a5cc0 con 0x7f257c0ffe20 2024-03-26T08:40:19.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.699+0000 7f2573400640 1 -- 172.21.15.43:0/3627775847 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f256c0054d0 con 0x7f257c0ffe20 2024-03-26T08:40:19.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.699+0000 7f2573400640 1 -- 172.21.15.43:0/3627775847 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256c004cf0 con 0x7f257c0ffe20 2024-03-26T08:40:19.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.699+0000 7f257be00640 1 -- 172.21.15.43:0/3627775847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f257c1a61c0 con 0x7f257c0ffe20 2024-03-26T08:40:19.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.701+0000 7f257be00640 1 -- 172.21.15.43:0/3627775847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f257c0fd510 con 0x7f257c0ffe20 2024-03-26T08:40:19.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.702+0000 7f2573400640 1 -- 172.21.15.43:0/3627775847 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f256c004e90 con 0x7f257c0ffe20 2024-03-26T08:40:19.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.704+0000 7f2573400640 1 --2- 172.21.15.43:0/3627775847 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f25600a2800 0x7f25600a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:19.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.704+0000 7f2573400640 1 -- 172.21.15.43:0/3627775847 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9215+0+0 (secure 0 0 0) 0x7f256c0cea30 con 0x7f257c0ffe20 2024-03-26T08:40:19.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.705+0000 7f257a000640 1 --2- 172.21.15.43:0/3627775847 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f25600a2800 0x7f25600a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:19.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.705+0000 7f257a000640 1 --2- 172.21.15.43:0/3627775847 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f25600a2800 0x7f25600a4cc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2568006010 tx=0x7f2568000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:19.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.709+0000 7f2573400640 1 -- 172.21.15.43:0/3627775847 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f256c09b6a0 con 0x7f257c0ffe20 2024-03-26T08:40:19.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:19.908+0000 7f257be00640 1 -- 172.21.15.43:0/3627775847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f257c0622a0 con 0x7f257c0ffe20 2024-03-26T08:40:20.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:19 smithi183 ceph-mon[35818]: pgmap v646: 129 pgs: 129 active+clean; 993 KiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 614 B/s wr, 3 op/s 2024-03-26T08:40:20.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:19 smithi183 ceph-mon[35818]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-26T08:40:20.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:19 smithi183 ceph-mon[35818]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-26T08:40:20.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:19 smithi183 ceph-mon[35818]: Cluster is now healthy 2024-03-26T08:40:20.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:19 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/371618801' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-26T08:40:20.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:19 smithi183 ceph-mon[35818]: mds.? [v2:172.21.15.52:6836/2330347727,v1:172.21.15.52:6837/2330347727] up:boot 2024-03-26T08:40:20.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:19 smithi183 ceph-mon[35818]: mds.? [v2:172.21.15.52:6838/483482953,v1:172.21.15.52:6839/483482953] up:boot 2024-03-26T08:40:20.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:19 smithi183 ceph-mon[35818]: fsmap 12 up:standby 2024-03-26T08:40:20.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:19 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T08:40:20.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:19 smithi183 ceph-mon[35818]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T08:40:20.298 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:19 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3627775847' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T08:40:20.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:19 smithi052 ceph-mon[35071]: pgmap v646: 129 pgs: 129 active+clean; 993 KiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 614 B/s wr, 3 op/s 2024-03-26T08:40:20.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:19 smithi052 ceph-mon[35071]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-26T08:40:20.321 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:19 smithi052 ceph-mon[35071]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-26T08:40:20.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:19 smithi052 ceph-mon[35071]: Cluster is now healthy 2024-03-26T08:40:20.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:19 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/371618801' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-26T08:40:20.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:19 smithi052 ceph-mon[35071]: mds.? [v2:172.21.15.52:6836/2330347727,v1:172.21.15.52:6837/2330347727] up:boot 2024-03-26T08:40:20.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:19 smithi052 ceph-mon[35071]: mds.? [v2:172.21.15.52:6838/483482953,v1:172.21.15.52:6839/483482953] up:boot 2024-03-26T08:40:20.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:19 smithi052 ceph-mon[35071]: fsmap 12 up:standby 2024-03-26T08:40:20.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:19 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T08:40:20.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:19 smithi052 ceph-mon[35071]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T08:40:20.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:19 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3627775847' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T08:40:20.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:19 smithi043 ceph-mon[31929]: pgmap v646: 129 pgs: 129 active+clean; 993 KiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 614 B/s wr, 3 op/s 2024-03-26T08:40:20.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:19 smithi043 ceph-mon[31929]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-26T08:40:20.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:19 smithi043 ceph-mon[31929]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-26T08:40:20.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:19 smithi043 ceph-mon[31929]: Cluster is now healthy 2024-03-26T08:40:20.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:19 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/371618801' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-26T08:40:20.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:19 smithi043 ceph-mon[31929]: mds.? [v2:172.21.15.52:6836/2330347727,v1:172.21.15.52:6837/2330347727] up:boot 2024-03-26T08:40:20.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:19 smithi043 ceph-mon[31929]: mds.? [v2:172.21.15.52:6838/483482953,v1:172.21.15.52:6839/483482953] up:boot 2024-03-26T08:40:20.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:19 smithi043 ceph-mon[31929]: fsmap 12 up:standby 2024-03-26T08:40:20.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:19 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T08:40:20.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:19 smithi043 ceph-mon[31929]: from='mgr.14152 172.21.15.43:0/4290427301' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T08:40:20.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:19 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3627775847' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T08:40:20.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.461+0000 7f2573400640 1 -- 172.21.15.43:0/3627775847 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v96) v1 ==== 181+0+0 (secure 0 0 0) 0x7f256c09afa0 con 0x7f257c0ffe20 2024-03-26T08:40:20.454 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2024-03-26T08:40:20.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.465+0000 7f257be00640 1 -- 172.21.15.43:0/3627775847 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f25600a2800 msgr2=0x7f25600a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:20.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.466+0000 7f257be00640 1 --2- 172.21.15.43:0/3627775847 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f25600a2800 0x7f25600a4cc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2568006010 tx=0x7f2568000bc0 comp rx=0 tx=0).stop 2024-03-26T08:40:20.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.466+0000 7f257be00640 1 -- 172.21.15.43:0/3627775847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f257c0ffe20 msgr2=0x7f257c19ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:20.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.466+0000 7f257be00640 1 --2- 172.21.15.43:0/3627775847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f257c0ffe20 0x7f257c19ddc0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f256c00d770 tx=0x7f256c00dc30 comp rx=0 tx=0).stop 2024-03-26T08:40:20.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.467+0000 7f257be00640 1 -- 172.21.15.43:0/3627775847 shutdown_connections 2024-03-26T08:40:20.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.467+0000 7f257be00640 1 --2- 172.21.15.43:0/3627775847 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f25600a2800 0x7f25600a4cc0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:20.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.467+0000 7f257be00640 1 --2- 172.21.15.43:0/3627775847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f257c105140 0x7f257c1a5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:20.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.467+0000 7f257be00640 1 --2- 172.21.15.43:0/3627775847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f257c102790 0x7f257c19e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:20.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.467+0000 7f257be00640 1 --2- 172.21.15.43:0/3627775847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f257c0ffe20 0x7f257c19ddc0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:20.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.467+0000 7f257be00640 1 -- 172.21.15.43:0/3627775847 >> 172.21.15.43:0/3627775847 conn(0x7f257c0fba20 msgr2=0x7f257c1007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:20.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.468+0000 7f257be00640 1 -- 172.21.15.43:0/3627775847 shutdown_connections 2024-03-26T08:40:20.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.468+0000 7f257be00640 1 -- 172.21.15.43:0/3627775847 wait complete. 2024-03-26T08:40:20.483 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-26T08:40:20.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.715+0000 7f0f51600640 1 -- 172.21.15.43:0/3615186386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f4c0772a0 msgr2=0x7f0f4c077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:20.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.715+0000 7f0f51600640 1 --2- 172.21.15.43:0/3615186386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f4c0772a0 0x7f0f4c077700 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0f38009900 tx=0x7f0f3802bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:20.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.715+0000 7f0f51600640 1 -- 172.21.15.43:0/3615186386 shutdown_connections 2024-03-26T08:40:20.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.715+0000 7f0f51600640 1 --2- 172.21.15.43:0/3615186386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f4c077c40 0x7f0f4c0784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:20.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.715+0000 7f0f51600640 1 --2- 172.21.15.43:0/3615186386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f4c0772a0 0x7f0f4c077700 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:20.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.715+0000 7f0f51600640 1 --2- 172.21.15.43:0/3615186386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f4c076060 0x7f0f4c076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:20.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.715+0000 7f0f51600640 1 -- 172.21.15.43:0/3615186386 >> 172.21.15.43:0/3615186386 conn(0x7f0f4c0fee50 msgr2=0x7f0f4c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:20.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.715+0000 7f0f51600640 1 -- 172.21.15.43:0/3615186386 shutdown_connections 2024-03-26T08:40:20.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.715+0000 7f0f51600640 1 -- 172.21.15.43:0/3615186386 wait complete. 2024-03-26T08:40:20.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.716+0000 7f0f51600640 1 Processor -- start 2024-03-26T08:40:20.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.716+0000 7f0f51600640 1 -- start start 2024-03-26T08:40:20.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.717+0000 7f0f51600640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f4c076060 0x7f0f4c19df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:20.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.717+0000 7f0f51600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f4c0772a0 0x7f0f4c19e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:20.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.717+0000 7f0f51600640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f4c077c40 0x7f0f4c1a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:20.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.717+0000 7f0f51600640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c113860 con 0x7f0f4c077c40 2024-03-26T08:40:20.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.717+0000 7f0f51600640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c1136e0 con 0x7f0f4c076060 2024-03-26T08:40:20.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.717+0000 7f0f51600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c1139e0 con 0x7f0f4c0772a0 2024-03-26T08:40:20.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.717+0000 7f0f4b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f4c077c40 0x7f0f4c1a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:20.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.717+0000 7f0f4b400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f4c077c40 0x7f0f4c1a54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59332/0 (socket says 172.21.15.43:59332) 2024-03-26T08:40:20.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.717+0000 7f0f4b400640 1 -- 172.21.15.43:0/1501929681 learned_addr learned my addr 172.21.15.43:0/1501929681 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:40:20.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.717+0000 7f0f4aa00640 1 --2- 172.21.15.43:0/1501929681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f4c076060 0x7f0f4c19df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:20.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.717+0000 7f0f4a000640 1 --2- 172.21.15.43:0/1501929681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f4c0772a0 0x7f0f4c19e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:20.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.717+0000 7f0f4b400640 1 -- 172.21.15.43:0/1501929681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f4c0772a0 msgr2=0x7f0f4c19e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:20.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.717+0000 7f0f4b400640 1 --2- 172.21.15.43:0/1501929681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f4c0772a0 0x7f0f4c19e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:20.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.718+0000 7f0f4b400640 1 -- 172.21.15.43:0/1501929681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f4c076060 msgr2=0x7f0f4c19df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:20.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.718+0000 7f0f4b400640 1 --2- 172.21.15.43:0/1501929681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f4c076060 0x7f0f4c19df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:20.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.718+0000 7f0f4b400640 1 -- 172.21.15.43:0/1501929681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4c1a5b50 con 0x7f0f4c077c40 2024-03-26T08:40:20.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.718+0000 7f0f4b400640 1 --2- 172.21.15.43:0/1501929681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f4c077c40 0x7f0f4c1a54e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400eaa0 tx=0x7f0f4400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:20.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.718+0000 7f0f43400640 1 -- 172.21.15.43:0/1501929681 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4400cdb0 con 0x7f0f4c077c40 2024-03-26T08:40:20.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.718+0000 7f0f43400640 1 -- 172.21.15.43:0/1501929681 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0f44005020 con 0x7f0f4c077c40 2024-03-26T08:40:20.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.718+0000 7f0f51600640 1 -- 172.21.15.43:0/1501929681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f4c1a5e40 con 0x7f0f4c077c40 2024-03-26T08:40:20.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.718+0000 7f0f43400640 1 -- 172.21.15.43:0/1501929681 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f440116d0 con 0x7f0f4c077c40 2024-03-26T08:40:20.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.718+0000 7f0f51600640 1 -- 172.21.15.43:0/1501929681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f4c1a6300 con 0x7f0f4c077c40 2024-03-26T08:40:20.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.719+0000 7f0f51600640 1 -- 172.21.15.43:0/1501929681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f4c110250 con 0x7f0f4c077c40 2024-03-26T08:40:20.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.722+0000 7f0f43400640 1 -- 172.21.15.43:0/1501929681 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f0f440051d0 con 0x7f0f4c077c40 2024-03-26T08:40:20.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.723+0000 7f0f43400640 1 --2- 172.21.15.43:0/1501929681 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0f300a2800 0x7f0f300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:20.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.723+0000 7f0f43400640 1 -- 172.21.15.43:0/1501929681 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8719+0+0 (secure 0 0 0) 0x7f0f440cf270 con 0x7f0f4c077c40 2024-03-26T08:40:20.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.723+0000 7f0f4aa00640 1 --2- 172.21.15.43:0/1501929681 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0f300a2800 0x7f0f300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:20.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.724+0000 7f0f4aa00640 1 --2- 172.21.15.43:0/1501929681 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0f300a2800 0x7f0f300a4cc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c0058c0 tx=0x7f0f3c00a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:20.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.726+0000 7f0f43400640 1 -- 172.21.15.43:0/1501929681 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0f44016030 con 0x7f0f4c077c40 2024-03-26T08:40:20.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:20.927+0000 7f0f51600640 1 -- 172.21.15.43:0/1501929681 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0f4c02d070 con 0x7f0f4c077c40 2024-03-26T08:40:21.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.474+0000 7f0f43400640 1 -- 172.21.15.43:0/1501929681 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v97) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0f4409c0d0 con 0x7f0f4c077c40 2024-03-26T08:40:21.466 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2024-03-26T08:40:21.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.478+0000 7f0f51600640 1 -- 172.21.15.43:0/1501929681 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0f300a2800 msgr2=0x7f0f300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:21.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.478+0000 7f0f51600640 1 --2- 172.21.15.43:0/1501929681 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0f300a2800 0x7f0f300a4cc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c0058c0 tx=0x7f0f3c00a4c0 comp rx=0 tx=0).stop 2024-03-26T08:40:21.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.478+0000 7f0f51600640 1 -- 172.21.15.43:0/1501929681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f4c077c40 msgr2=0x7f0f4c1a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:21.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.478+0000 7f0f51600640 1 --2- 172.21.15.43:0/1501929681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f4c077c40 0x7f0f4c1a54e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400eaa0 tx=0x7f0f4400ef60 comp rx=0 tx=0).stop 2024-03-26T08:40:21.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.480+0000 7f0f51600640 1 -- 172.21.15.43:0/1501929681 shutdown_connections 2024-03-26T08:40:21.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.480+0000 7f0f51600640 1 --2- 172.21.15.43:0/1501929681 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f0f300a2800 0x7f0f300a4cc0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:21.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.480+0000 7f0f51600640 1 --2- 172.21.15.43:0/1501929681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f4c077c40 0x7f0f4c1a54e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:21.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.480+0000 7f0f51600640 1 --2- 172.21.15.43:0/1501929681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f4c0772a0 0x7f0f4c19e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:21.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.480+0000 7f0f51600640 1 --2- 172.21.15.43:0/1501929681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f4c076060 0x7f0f4c19df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:21.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.480+0000 7f0f51600640 1 -- 172.21.15.43:0/1501929681 >> 172.21.15.43:0/1501929681 conn(0x7f0f4c0fee50 msgr2=0x7f0f4c100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:21.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.480+0000 7f0f51600640 1 -- 172.21.15.43:0/1501929681 shutdown_connections 2024-03-26T08:40:21.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.480+0000 7f0f51600640 1 -- 172.21.15.43:0/1501929681 wait complete. 2024-03-26T08:40:21.497 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-26T08:40:21.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.733+0000 7f625de00640 1 -- 172.21.15.43:0/57771729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62581015a0 msgr2=0x7f62581019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:21.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.733+0000 7f625de00640 1 --2- 172.21.15.43:0/57771729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62581015a0 0x7f62581019c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f624c009900 tx=0x7f624c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:21.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.733+0000 7f625de00640 1 -- 172.21.15.43:0/57771729 shutdown_connections 2024-03-26T08:40:21.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.733+0000 7f625de00640 1 --2- 172.21.15.43:0/57771729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6258103150 0x7f62581099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:21.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.733+0000 7f625de00640 1 --2- 172.21.15.43:0/57771729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6258102790 0x7f6258102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:21.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.733+0000 7f625de00640 1 --2- 172.21.15.43:0/57771729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62581015a0 0x7f62581019c0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:21.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.733+0000 7f625de00640 1 -- 172.21.15.43:0/57771729 >> 172.21.15.43:0/57771729 conn(0x7f62580fcb10 msgr2=0x7f62580fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:21.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.733+0000 7f625de00640 1 -- 172.21.15.43:0/57771729 shutdown_connections 2024-03-26T08:40:21.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.734+0000 7f625de00640 1 -- 172.21.15.43:0/57771729 wait complete. 2024-03-26T08:40:21.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.734+0000 7f625de00640 1 Processor -- start 2024-03-26T08:40:21.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.735+0000 7f625de00640 1 -- start start 2024-03-26T08:40:21.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.735+0000 7f625de00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62581015a0 0x7f62581999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:21.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.735+0000 7f625de00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6258102790 0x7f6258199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:21.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.735+0000 7f625de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6258103150 0x7f62581a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:21.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.735+0000 7f625de00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625810d110 con 0x7f62581015a0 2024-03-26T08:40:21.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.735+0000 7f625de00640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625810cf90 con 0x7f6258102790 2024-03-26T08:40:21.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.735+0000 7f625de00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625810d290 con 0x7f6258103150 2024-03-26T08:40:21.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.735+0000 7f625ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62581015a0 0x7f62581999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:21.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.735+0000 7f625ca00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62581015a0 0x7f62581999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59346/0 (socket says 172.21.15.43:59346) 2024-03-26T08:40:21.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.735+0000 7f625ca00640 1 -- 172.21.15.43:0/2419747076 learned_addr learned my addr 172.21.15.43:0/2419747076 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-26T08:40:21.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.736+0000 7f625d400640 1 --2- 172.21.15.43:0/2419747076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6258103150 0x7f62581a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:21.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.736+0000 7f6257e00640 1 --2- 172.21.15.43:0/2419747076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6258102790 0x7f6258199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:21.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.736+0000 7f625ca00640 1 -- 172.21.15.43:0/2419747076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6258103150 msgr2=0x7f62581a0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:21.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.736+0000 7f625ca00640 1 --2- 172.21.15.43:0/2419747076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6258103150 0x7f62581a0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:21.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.736+0000 7f625ca00640 1 -- 172.21.15.43:0/2419747076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6258102790 msgr2=0x7f6258199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:21.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.736+0000 7f625ca00640 1 --2- 172.21.15.43:0/2419747076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6258102790 0x7f6258199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:21.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.736+0000 7f625ca00640 1 -- 172.21.15.43:0/2419747076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62581a15c0 con 0x7f62581015a0 2024-03-26T08:40:21.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.736+0000 7f6257e00640 1 --2- 172.21.15.43:0/2419747076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6258102790 0x7f6258199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:40:21.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.736+0000 7f625ca00640 1 --2- 172.21.15.43:0/2419747076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62581015a0 0x7f62581999a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f624c000c00 tx=0x7f624c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:21.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.737+0000 7f6255600640 1 -- 172.21.15.43:0/2419747076 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c043020 con 0x7f62581015a0 2024-03-26T08:40:21.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.737+0000 7f6255600640 1 -- 172.21.15.43:0/2419747076 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f624c0044c0 con 0x7f62581015a0 2024-03-26T08:40:21.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.737+0000 7f6255600640 1 -- 172.21.15.43:0/2419747076 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c003c90 con 0x7f62581015a0 2024-03-26T08:40:21.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.737+0000 7f625de00640 1 -- 172.21.15.43:0/2419747076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62581a1850 con 0x7f62581015a0 2024-03-26T08:40:21.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.737+0000 7f625de00640 1 -- 172.21.15.43:0/2419747076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62581a1da0 con 0x7f62581015a0 2024-03-26T08:40:21.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.738+0000 7f625de00640 1 -- 172.21.15.43:0/2419747076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62581a1fa0 con 0x7f62581015a0 2024-03-26T08:40:21.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.740+0000 7f6255600640 1 -- 172.21.15.43:0/2419747076 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 137290+0+0 (secure 0 0 0) 0x7f624c046020 con 0x7f62581015a0 2024-03-26T08:40:21.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.743+0000 7f6255600640 1 --2- 172.21.15.43:0/2419747076 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f62400a2800 0x7f62400a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:21.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.743+0000 7f6255600640 1 -- 172.21.15.43:0/2419747076 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8152+0+0 (secure 0 0 0) 0x7f624c0f0c80 con 0x7f62581015a0 2024-03-26T08:40:21.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.743+0000 7f6257e00640 1 --2- 172.21.15.43:0/2419747076 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f62400a2800 0x7f62400a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:21.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.744+0000 7f6257e00640 1 --2- 172.21.15.43:0/2419747076 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f62400a2800 0x7f62400a4cc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f625819aeb0 tx=0x7f6244005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:21.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.747+0000 7f6255600640 1 -- 172.21.15.43:0/2419747076 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f624c04d020 con 0x7f62581015a0 2024-03-26T08:40:21.797 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:21 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/3627775847' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T08:40:21.797 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:21 smithi183 ceph-mon[35818]: osdmap e96: 12 total, 12 up, 12 in 2024-03-26T08:40:21.797 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:21 smithi183 ceph-mon[35818]: pgmap v648: 97 pgs: 97 active+clean; 577 KiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:40:21.797 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:21 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1501929681' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T08:40:21.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:21 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/3627775847' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T08:40:21.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:21 smithi052 ceph-mon[35071]: osdmap e96: 12 total, 12 up, 12 in 2024-03-26T08:40:21.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:21 smithi052 ceph-mon[35071]: pgmap v648: 97 pgs: 97 active+clean; 577 KiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:40:21.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:21 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1501929681' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T08:40:21.900 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:21 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/3627775847' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T08:40:21.900 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:21 smithi043 ceph-mon[31929]: osdmap e96: 12 total, 12 up, 12 in 2024-03-26T08:40:21.900 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:21 smithi043 ceph-mon[31929]: pgmap v648: 97 pgs: 97 active+clean; 577 KiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:40:21.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:21 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1501929681' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T08:40:21.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:21.953+0000 7f625de00640 1 -- 172.21.15.43:0/2419747076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f62581a25f0 con 0x7f62581015a0 2024-03-26T08:40:22.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:22.478+0000 7f6255600640 1 -- 172.21.15.43:0/2419747076 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v98) v1 ==== 178+0+0 (secure 0 0 0) 0x7f624c0bddc0 con 0x7f62581015a0 2024-03-26T08:40:22.470 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data_ec' removed 2024-03-26T08:40:22.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:22.483+0000 7f625de00640 1 -- 172.21.15.43:0/2419747076 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f62400a2800 msgr2=0x7f62400a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:22.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:22.483+0000 7f625de00640 1 --2- 172.21.15.43:0/2419747076 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f62400a2800 0x7f62400a4cc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f625819aeb0 tx=0x7f6244005f90 comp rx=0 tx=0).stop 2024-03-26T08:40:22.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:22.483+0000 7f625de00640 1 -- 172.21.15.43:0/2419747076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62581015a0 msgr2=0x7f62581999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:22.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:22.483+0000 7f625de00640 1 --2- 172.21.15.43:0/2419747076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62581015a0 0x7f62581999a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f624c000c00 tx=0x7f624c0040c0 comp rx=0 tx=0).stop 2024-03-26T08:40:22.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:22.483+0000 7f625de00640 1 -- 172.21.15.43:0/2419747076 shutdown_connections 2024-03-26T08:40:22.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:22.483+0000 7f625de00640 1 --2- 172.21.15.43:0/2419747076 >> [v2:172.21.15.43:6800/3390816575,v1:172.21.15.43:6801/3390816575] conn(0x7f62400a2800 0x7f62400a4cc0 secure :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f625819aeb0 tx=0x7f6244005f90 comp rx=0 tx=0).stop 2024-03-26T08:40:22.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:22.483+0000 7f625de00640 1 --2- 172.21.15.43:0/2419747076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6258103150 0x7f62581a0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:22.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:22.484+0000 7f625de00640 1 --2- 172.21.15.43:0/2419747076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6258102790 0x7f6258199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:22.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:22.484+0000 7f625de00640 1 --2- 172.21.15.43:0/2419747076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62581015a0 0x7f62581999a0 secure :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f624c000c00 tx=0x7f624c0040c0 comp rx=0 tx=0).stop 2024-03-26T08:40:22.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:22.484+0000 7f625de00640 1 -- 172.21.15.43:0/2419747076 >> 172.21.15.43:0/2419747076 conn(0x7f62580fcb10 msgr2=0x7f62580738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:22.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:22.486+0000 7f625de00640 1 -- 172.21.15.43:0/2419747076 shutdown_connections 2024-03-26T08:40:22.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-26T08:40:22.486+0000 7f625de00640 1 -- 172.21.15.43:0/2419747076 wait complete. 2024-03-26T08:40:22.512 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T08:40:22.584 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T08:40:22.677 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T08:40:22.766 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:22 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/1501929681' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T08:40:22.766 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:22 smithi183 ceph-mon[35818]: osdmap e97: 12 total, 12 up, 12 in 2024-03-26T08:40:22.766 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:22 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2419747076' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T08:40:22.778 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-26T08:40:22.778 DEBUG:teuthology.orchestra.run.smithi043:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-26T08:40:22.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:22 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/1501929681' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T08:40:22.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:22 smithi052 ceph-mon[35071]: osdmap e97: 12 total, 12 up, 12 in 2024-03-26T08:40:22.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:22 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2419747076' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T08:40:22.836 DEBUG:teuthology.orchestra.run.smithi052:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-26T08:40:22.897 DEBUG:teuthology.orchestra.run.smithi183:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-26T08:40:22.900 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:22 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/1501929681' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T08:40:22.900 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:22 smithi043 ceph-mon[31929]: osdmap e97: 12 total, 12 up, 12 in 2024-03-26T08:40:22.900 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:22 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2419747076' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T08:40:22.959 INFO:tasks.cephadm:Stopping all daemons... 2024-03-26T08:40:22.959 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-26T08:40:22.959 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@mon.a 2024-03-26T08:40:23.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:23 smithi043 systemd[1]: Stopping Ceph mon.a for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:40:23.797 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:23 smithi183 ceph-mon[35818]: from='client.? 172.21.15.43:0/2419747076' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T08:40:23.797 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:23 smithi183 ceph-mon[35818]: osdmap e98: 12 total, 12 up, 12 in 2024-03-26T08:40:23.797 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:23 smithi183 ceph-mon[35818]: pgmap v651: 1 pgs: 1 active+clean; 577 KiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:40:23.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:23 smithi052 ceph-mon[35071]: from='client.? 172.21.15.43:0/2419747076' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T08:40:23.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:23 smithi052 ceph-mon[35071]: osdmap e98: 12 total, 12 up, 12 in 2024-03-26T08:40:23.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:23 smithi052 ceph-mon[35071]: pgmap v651: 1 pgs: 1 active+clean; 577 KiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:40:23.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:23 smithi043 ceph-mon[31929]: from='client.? 172.21.15.43:0/2419747076' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T08:40:23.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:23 smithi043 ceph-mon[31929]: osdmap e98: 12 total, 12 up, 12 in 2024-03-26T08:40:23.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:23 smithi043 ceph-mon[31929]: pgmap v651: 1 pgs: 1 active+clean; 577 KiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:40:23.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:23 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-a[31924]: 2024-03-26T08:40:23.596+0000 7f85adc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:40:23.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:23 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-a[31924]: 2024-03-26T08:40:23.596+0000 7f85adc00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-26T08:40:23.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:23 smithi043 podman[85791]: 2024-03-26 08:40:23.658690541 +0000 UTC m=+0.221760872 container died b43fc9922941a01d411d5cd0e4a643048d766ce35443039ac012e1128202736f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-a, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T08:40:24.688 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:24 smithi043 podman[85791]: 2024-03-26 08:40:24.690560345 +0000 UTC m=+1.253630642 container cleanup b43fc9922941a01d411d5cd0e4a643048d766ce35443039ac012e1128202736f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-a, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319) 2024-03-26T08:40:24.689 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:24 smithi043 bash[85791]: ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-a 2024-03-26T08:40:25.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 26 08:40:24 smithi043 podman[85805]: 2024-03-26 08:40:24.992492336 +0000 UTC m=+1.314302709 container remove b43fc9922941a01d411d5cd0e4a643048d766ce35443039ac012e1128202736f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-26T08:40:25.500 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mon.a.service' 2024-03-26T08:40:25.645 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:40:25.645 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-26T08:40:25.646 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-26T08:40:25.646 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@mon.b 2024-03-26T08:40:26.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:25 smithi052 systemd[1]: Stopping Ceph mon.b for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:40:26.571 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:26 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-b[35067]: 2024-03-26T08:40:26.240+0000 7f85f3a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:40:26.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:26 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-b[35067]: 2024-03-26T08:40:26.240+0000 7f85f3a00700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-26T08:40:26.572 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:26 smithi052 podman[68381]: 2024-03-26 08:40:26.272236422 +0000 UTC m=+0.171591940 container died 02f8c34e593dcc8d9af24481d097371b060bc446b1bf1671b837eb1a0e460b66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-26T08:40:27.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:26 smithi052 podman[68381]: 2024-03-26 08:40:26.975272295 +0000 UTC m=+0.874627824 container cleanup 02f8c34e593dcc8d9af24481d097371b060bc446b1bf1671b837eb1a0e460b66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-26T08:40:27.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:26 smithi052 bash[68381]: ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-b 2024-03-26T08:40:27.322 INFO:journalctl@ceph.mon.b.smithi052.stdout:Mar 26 08:40:27 smithi052 podman[68394]: 2024-03-26 08:40:27.210582179 +0000 UTC m=+0.924147119 container remove 02f8c34e593dcc8d9af24481d097371b060bc446b1bf1671b837eb1a0e460b66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-26T08:40:27.705 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mon.b.service' 2024-03-26T08:40:27.841 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:40:27.841 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-26T08:40:27.842 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-26T08:40:27.842 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@mon.c 2024-03-26T08:40:28.297 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:27 smithi183 systemd[1]: Stopping Ceph mon.c for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:40:28.797 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:28 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-c[35812]: 2024-03-26T08:40:28.371+0000 7f7c25c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:40:28.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:28 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-c[35812]: 2024-03-26T08:40:28.371+0000 7f7c25c00700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-26T08:40:28.798 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:28 smithi183 podman[69519]: 2024-03-26 08:40:28.5272678 +0000 UTC m=+0.300656684 container died fc6a11acaf9f95d107e7bf0c94c13f337e6d46e9944411d7415380b9a318575a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-c, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319) 2024-03-26T08:40:29.421 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:29 smithi183 podman[69519]: 2024-03-26 08:40:29.149278397 +0000 UTC m=+0.922667277 container cleanup fc6a11acaf9f95d107e7bf0c94c13f337e6d46e9944411d7415380b9a318575a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-c, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T08:40:29.421 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:29 smithi183 bash[69519]: ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-c 2024-03-26T08:40:29.797 INFO:journalctl@ceph.mon.c.smithi183.stdout:Mar 26 08:40:29 smithi183 podman[69533]: 2024-03-26 08:40:29.429322031 +0000 UTC m=+0.892118414 container remove fc6a11acaf9f95d107e7bf0c94c13f337e6d46e9944411d7415380b9a318575a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-26T08:40:29.901 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mon.c.service' 2024-03-26T08:40:30.040 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:40:30.041 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-26T08:40:30.041 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-26T08:40:30.041 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@mgr.x 2024-03-26T08:40:30.400 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:40:30 smithi043 systemd[1]: Stopping Ceph mgr.x for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:40:31.097 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:40:30 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:40:30.670+0000 7fbf75c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:40:31.097 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:40:30 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x[32169]: 2024-03-26T08:40:30.670+0000 7fbf75c00700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-26T08:40:31.097 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:40:30 smithi043 podman[85918]: 2024-03-26 08:40:30.763895727 +0000 UTC m=+0.263356246 container died ec1554ba1ef064986344b7d812fd120cb0f19e3bee06d4d80db09202028f184b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T08:40:31.348 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:40:31 smithi043 podman[85918]: 2024-03-26 08:40:31.330596799 +0000 UTC m=+0.830057299 container cleanup ec1554ba1ef064986344b7d812fd120cb0f19e3bee06d4d80db09202028f184b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-26T08:40:31.348 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:40:31 smithi043 bash[85918]: ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x 2024-03-26T08:40:31.900 INFO:journalctl@ceph.mgr.x.smithi043.stdout:Mar 26 08:40:31 smithi043 podman[85932]: 2024-03-26 08:40:31.515537409 +0000 UTC m=+0.734917359 container remove ec1554ba1ef064986344b7d812fd120cb0f19e3bee06d4d80db09202028f184b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319) 2024-03-26T08:40:32.035 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mgr.x.service' 2024-03-26T08:40:32.159 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:40:32.159 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-26T08:40:32.159 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-26T08:40:32.159 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@mgr.y 2024-03-26T08:40:32.571 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:40:32 smithi052 systemd[1]: Stopping Ceph mgr.y for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:40:33.311 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:40:32 smithi052 podman[68515]: 2024-03-26 08:40:32.856403147 +0000 UTC m=+0.246442616 container died 1bdae692a3392dbad6199655b98007919cc8b0ee99a1266ecbce97cb0feefb43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T08:40:33.571 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:40:33 smithi052 podman[68515]: 2024-03-26 08:40:33.481261364 +0000 UTC m=+0.871300813 container cleanup 1bdae692a3392dbad6199655b98007919cc8b0ee99a1266ecbce97cb0feefb43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-26T08:40:33.572 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:40:33 smithi052 bash[68515]: ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y 2024-03-26T08:40:34.071 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:40:33 smithi052 podman[68529]: 2024-03-26 08:40:33.707742608 +0000 UTC m=+0.835960022 container remove 1bdae692a3392dbad6199655b98007919cc8b0ee99a1266ecbce97cb0feefb43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-26T08:40:34.071 INFO:journalctl@ceph.mgr.y.smithi052.stdout:Mar 26 08:40:33 smithi052 systemd[1]: ceph-88759420-eb49-11ee-95d0-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-03-26T08:40:34.194 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mgr.y.service' 2024-03-26T08:40:34.322 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:40:34.322 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-26T08:40:34.322 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-26T08:40:34.322 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@mgr.z 2024-03-26T08:40:34.797 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:40:34 smithi183 systemd[1]: Stopping Ceph mgr.z for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:40:35.226 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:40:34 smithi183 podman[69640]: 2024-03-26 08:40:34.967114984 +0000 UTC m=+0.226043085 container died 5eb7852de1c7988488694a3d235bf39ba8a382b50065652d7d13118809e61c20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-26T08:40:35.547 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:40:35 smithi183 podman[69640]: 2024-03-26 08:40:35.412640983 +0000 UTC m=+0.671569077 container cleanup 5eb7852de1c7988488694a3d235bf39ba8a382b50065652d7d13118809e61c20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T08:40:35.547 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:40:35 smithi183 bash[69640]: ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z 2024-03-26T08:40:36.047 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:40:35 smithi183 podman[69653]: 2024-03-26 08:40:35.672762115 +0000 UTC m=+0.690389148 container remove 5eb7852de1c7988488694a3d235bf39ba8a382b50065652d7d13118809e61c20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mgr-z, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-26T08:40:36.047 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Mar 26 08:40:35 smithi183 systemd[1]: ceph-88759420-eb49-11ee-95d0-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-03-26T08:40:36.156 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mgr.z.service' 2024-03-26T08:40:36.268 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:40:36.268 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-26T08:40:36.268 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-26T08:40:36.268 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.0 2024-03-26T08:40:36.650 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 26 08:40:36 smithi043 systemd[1]: Stopping Ceph osd.0 for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:40:37.151 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 26 08:40:36 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-0[43905]: 2024-03-26T08:40:36.900+0000 7f5083200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:40:37.151 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 26 08:40:36 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-0[43905]: 2024-03-26T08:40:36.900+0000 7f5083200700 -1 osd.0 98 *** Got signal Terminated *** 2024-03-26T08:40:37.151 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 26 08:40:36 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-0[43905]: 2024-03-26T08:40:36.900+0000 7f5083200700 -1 osd.0 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T08:40:42.765 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 26 08:40:42 smithi043 podman[86041]: 2024-03-26 08:40:42.461514 +0000 UTC m=+5.717609379 container died 259b59fc83dd9c3a80f50a046d39630a43bd52be35ec37c40c58ac636626aae5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T08:40:43.060 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 26 08:40:42 smithi043 podman[86041]: 2024-03-26 08:40:42.927340439 +0000 UTC m=+6.183435853 container cleanup 259b59fc83dd9c3a80f50a046d39630a43bd52be35ec37c40c58ac636626aae5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-0, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-26T08:40:43.061 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 26 08:40:42 smithi043 bash[86041]: ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-0 2024-03-26T08:40:43.400 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 26 08:40:43 smithi043 podman[86074]: 2024-03-26 08:40:43.287230216 +0000 UTC m=+0.818317648 container remove 259b59fc83dd9c3a80f50a046d39630a43bd52be35ec37c40c58ac636626aae5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T08:40:44.151 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 26 08:40:43 smithi043 podman[86162]: 2024-03-26 08:40:43.797845038 +0000 UTC m=+0.040921759 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:40:44.151 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 26 08:40:43 smithi043 podman[86162]: 2024-03-26 08:40:43.921673512 +0000 UTC m=+0.164750227 container create ec64f1545015e70a686fa1de32a48d81fde6c8d5958069d31b416c35ef014358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-26T08:40:44.651 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 26 08:40:44 smithi043 podman[86162]: 2024-03-26 08:40:44.220175554 +0000 UTC m=+0.463252272 container init ec64f1545015e70a686fa1de32a48d81fde6c8d5958069d31b416c35ef014358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-0-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-26T08:40:44.651 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 26 08:40:44 smithi043 podman[86162]: 2024-03-26 08:40:44.23310049 +0000 UTC m=+0.476177185 container start ec64f1545015e70a686fa1de32a48d81fde6c8d5958069d31b416c35ef014358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-0-deactivate, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-26T08:40:44.651 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 26 08:40:44 smithi043 podman[86162]: 2024-03-26 08:40:44.332721031 +0000 UTC m=+0.575797752 container attach ec64f1545015e70a686fa1de32a48d81fde6c8d5958069d31b416c35ef014358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T08:40:45.065 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 26 08:40:44 smithi043 podman[86188]: 2024-03-26 08:40:44.73494885 +0000 UTC m=+0.032925175 container died ec64f1545015e70a686fa1de32a48d81fde6c8d5958069d31b416c35ef014358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-0-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-26T08:40:45.642 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.0.service' 2024-03-26T08:40:45.767 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:40:45.768 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-26T08:40:45.768 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-26T08:40:45.768 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.1 2024-03-26T08:40:46.321 INFO:journalctl@ceph.osd.1.smithi052.stdout:Mar 26 08:40:45 smithi052 systemd[1]: Stopping Ceph osd.1 for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:40:46.822 INFO:journalctl@ceph.osd.1.smithi052.stdout:Mar 26 08:40:46 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-1[39666]: 2024-03-26T08:40:46.369+0000 7efc5ce00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:40:46.822 INFO:journalctl@ceph.osd.1.smithi052.stdout:Mar 26 08:40:46 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-1[39666]: 2024-03-26T08:40:46.369+0000 7efc5ce00700 -1 osd.1 98 *** Got signal Terminated *** 2024-03-26T08:40:46.822 INFO:journalctl@ceph.osd.1.smithi052.stdout:Mar 26 08:40:46 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-1[39666]: 2024-03-26T08:40:46.369+0000 7efc5ce00700 -1 osd.1 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T08:40:52.252 INFO:journalctl@ceph.osd.1.smithi052.stdout:Mar 26 08:40:51 smithi052 podman[68639]: 2024-03-26 08:40:51.947183616 +0000 UTC m=+5.737013679 container died 08dd3012fb58a23f22d3305b07def235bba8988c0cdb37b0653aa3c39bdf68be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-1, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T08:40:52.571 INFO:journalctl@ceph.osd.1.smithi052.stdout:Mar 26 08:40:52 smithi052 podman[68639]: 2024-03-26 08:40:52.419614567 +0000 UTC m=+6.209444623 container cleanup 08dd3012fb58a23f22d3305b07def235bba8988c0cdb37b0653aa3c39bdf68be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-1, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True) 2024-03-26T08:40:52.571 INFO:journalctl@ceph.osd.1.smithi052.stdout:Mar 26 08:40:52 smithi052 bash[68639]: ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-1 2024-03-26T08:40:53.071 INFO:journalctl@ceph.osd.1.smithi052.stdout:Mar 26 08:40:52 smithi052 podman[68656]: 2024-03-26 08:40:52.679492034 +0000 UTC m=+0.712602887 container remove 08dd3012fb58a23f22d3305b07def235bba8988c0cdb37b0653aa3c39bdf68be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-26T08:40:53.821 INFO:journalctl@ceph.osd.1.smithi052.stdout:Mar 26 08:40:53 smithi052 podman[68746]: 2024-03-26 08:40:53.449624521 +0000 UTC m=+0.035059353 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:40:53.822 INFO:journalctl@ceph.osd.1.smithi052.stdout:Mar 26 08:40:53 smithi052 podman[68746]: 2024-03-26 08:40:53.563792069 +0000 UTC m=+0.149226932 container create cf2ee318e79709b7769c70f7b7a68064632b4f612cc83cc3c76b79397f8bff4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-26T08:40:54.315 INFO:journalctl@ceph.osd.1.smithi052.stdout:Mar 26 08:40:53 smithi052 podman[68746]: 2024-03-26 08:40:53.867990216 +0000 UTC m=+0.453425099 container init cf2ee318e79709b7769c70f7b7a68064632b4f612cc83cc3c76b79397f8bff4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-1-deactivate, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-26T08:40:54.315 INFO:journalctl@ceph.osd.1.smithi052.stdout:Mar 26 08:40:53 smithi052 podman[68746]: 2024-03-26 08:40:53.880621848 +0000 UTC m=+0.466056684 container start cf2ee318e79709b7769c70f7b7a68064632b4f612cc83cc3c76b79397f8bff4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-26T08:40:54.315 INFO:journalctl@ceph.osd.1.smithi052.stdout:Mar 26 08:40:53 smithi052 podman[68746]: 2024-03-26 08:40:53.983556341 +0000 UTC m=+0.568991247 container attach cf2ee318e79709b7769c70f7b7a68064632b4f612cc83cc3c76b79397f8bff4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-26T08:40:54.571 INFO:journalctl@ceph.osd.1.smithi052.stdout:Mar 26 08:40:54 smithi052 podman[68746]: 2024-03-26 08:40:54.324497155 +0000 UTC m=+0.909932043 container died cf2ee318e79709b7769c70f7b7a68064632b4f612cc83cc3c76b79397f8bff4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-1-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-26T08:40:55.113 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.1.service' 2024-03-26T08:40:55.241 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:40:55.241 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-26T08:40:55.242 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-26T08:40:55.242 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.2 2024-03-26T08:40:55.797 INFO:journalctl@ceph.osd.2.smithi183.stdout:Mar 26 08:40:55 smithi183 systemd[1]: Stopping Ceph osd.2 for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:40:56.297 INFO:journalctl@ceph.osd.2.smithi183.stdout:Mar 26 08:40:55 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-2[40992]: 2024-03-26T08:40:55.839+0000 7f4717000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:40:56.298 INFO:journalctl@ceph.osd.2.smithi183.stdout:Mar 26 08:40:55 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-2[40992]: 2024-03-26T08:40:55.839+0000 7f4717000700 -1 osd.2 98 *** Got signal Terminated *** 2024-03-26T08:40:56.298 INFO:journalctl@ceph.osd.2.smithi183.stdout:Mar 26 08:40:55 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-2[40992]: 2024-03-26T08:40:55.839+0000 7f4717000700 -1 osd.2 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T08:41:01.620 INFO:journalctl@ceph.osd.2.smithi183.stdout:Mar 26 08:41:01 smithi183 podman[69763]: 2024-03-26 08:41:01.376745937 +0000 UTC m=+5.699830302 container died 9820bbd3b51340cb5e466a140866f76e24336a9d7cf850d3742d3a9d89fb850c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-03-26T08:41:01.984 INFO:journalctl@ceph.osd.2.smithi183.stdout:Mar 26 08:41:01 smithi183 podman[69763]: 2024-03-26 08:41:01.782650641 +0000 UTC m=+6.105735024 container cleanup 9820bbd3b51340cb5e466a140866f76e24336a9d7cf850d3742d3a9d89fb850c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2) 2024-03-26T08:41:01.984 INFO:journalctl@ceph.osd.2.smithi183.stdout:Mar 26 08:41:01 smithi183 bash[69763]: ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-2 2024-03-26T08:41:02.297 INFO:journalctl@ceph.osd.2.smithi183.stdout:Mar 26 08:41:01 smithi183 podman[69781]: 2024-03-26 08:41:01.992535509 +0000 UTC m=+0.602746551 container remove 9820bbd3b51340cb5e466a140866f76e24336a9d7cf850d3742d3a9d89fb850c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319) 2024-03-26T08:41:03.148 INFO:journalctl@ceph.osd.2.smithi183.stdout:Mar 26 08:41:02 smithi183 podman[69869]: 2024-03-26 08:41:02.730094574 +0000 UTC m=+0.039312223 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:41:03.149 INFO:journalctl@ceph.osd.2.smithi183.stdout:Mar 26 08:41:02 smithi183 podman[69869]: 2024-03-26 08:41:02.877007718 +0000 UTC m=+0.186225364 container create f35a0268e68d8015b93b28e56495ebaf0743f8b54146d9785b3e81505272eb00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-03-26T08:41:03.547 INFO:journalctl@ceph.osd.2.smithi183.stdout:Mar 26 08:41:03 smithi183 podman[69869]: 2024-03-26 08:41:03.156500751 +0000 UTC m=+0.465718367 container init f35a0268e68d8015b93b28e56495ebaf0743f8b54146d9785b3e81505272eb00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-26T08:41:03.547 INFO:journalctl@ceph.osd.2.smithi183.stdout:Mar 26 08:41:03 smithi183 podman[69869]: 2024-03-26 08:41:03.168082437 +0000 UTC m=+0.477300061 container start f35a0268e68d8015b93b28e56495ebaf0743f8b54146d9785b3e81505272eb00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319) 2024-03-26T08:41:03.547 INFO:journalctl@ceph.osd.2.smithi183.stdout:Mar 26 08:41:03 smithi183 podman[69869]: 2024-03-26 08:41:03.270885605 +0000 UTC m=+0.580103315 container attach f35a0268e68d8015b93b28e56495ebaf0743f8b54146d9785b3e81505272eb00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-2-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-26T08:41:03.971 INFO:journalctl@ceph.osd.2.smithi183.stdout:Mar 26 08:41:03 smithi183 podman[69869]: 2024-03-26 08:41:03.600329703 +0000 UTC m=+0.909547314 container died f35a0268e68d8015b93b28e56495ebaf0743f8b54146d9785b3e81505272eb00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True) 2024-03-26T08:41:04.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:03 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:03.702+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:04.391 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.2.service' 2024-03-26T08:41:04.501 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:41:04.501 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-26T08:41:04.501 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-26T08:41:04.502 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.3 2024-03-26T08:41:04.900 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:04 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:04.663+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:04.900 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 26 08:41:04 smithi043 systemd[1]: Stopping Ceph osd.3 for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:41:05.280 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:04 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:04.995+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:05.401 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 26 08:41:05 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-3[50456]: 2024-03-26T08:41:05.139+0000 7f92bcc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:41:05.401 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 26 08:41:05 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-3[50456]: 2024-03-26T08:41:05.139+0000 7f92bcc00700 -1 osd.3 98 *** Got signal Terminated *** 2024-03-26T08:41:05.401 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 26 08:41:05 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-3[50456]: 2024-03-26T08:41:05.139+0000 7f92bcc00700 -1 osd.3 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T08:41:05.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:05 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:05.286+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:06.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:05 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:05.677+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:06.264 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:06 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:06.005+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:06.264 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:05 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:05.941+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.462806+0000 front 2024-03-26T08:40:40.462545+0000 (oldest deadline 2024-03-26T08:41:05.162732+0000) 2024-03-26T08:41:06.321 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:05 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:05.910+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:06.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:06 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:06.272+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:07.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:06 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:06.679+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:07.278 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:06 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:06.964+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:07.278 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:06 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:06.925+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.462806+0000 front 2024-03-26T08:40:40.462545+0000 (oldest deadline 2024-03-26T08:41:05.162732+0000) 2024-03-26T08:41:07.321 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:06 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:06.865+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:07.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:07 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:07.286+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:07.900 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 26 08:41:07 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-3[50456]: 2024-03-26T08:41:07.587+0000 7f92b3400700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.684882+0000 front 2024-03-26T08:40:41.684826+0000 (oldest deadline 2024-03-26T08:41:07.585038+0000) 2024-03-26T08:41:07.900 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:07 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:07.658+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:08.255 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:07 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:07.993+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:08.255 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:07 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:07.898+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.462806+0000 front 2024-03-26T08:40:40.462545+0000 (oldest deadline 2024-03-26T08:41:05.162732+0000) 2024-03-26T08:41:08.321 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:07 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:07.868+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:08.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:08 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:08.263+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:08.900 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:08 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:08.642+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:08.900 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 26 08:41:08 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-3[50456]: 2024-03-26T08:41:08.560+0000 7f92b3400700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.684882+0000 front 2024-03-26T08:40:41.684826+0000 (oldest deadline 2024-03-26T08:41:07.585038+0000) 2024-03-26T08:41:09.223 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:08 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:08.872+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:09.275 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:09 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:09.014+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:09.275 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:08 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:08.855+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.462806+0000 front 2024-03-26T08:40:40.462545+0000 (oldest deadline 2024-03-26T08:41:05.162732+0000) 2024-03-26T08:41:09.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:09 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:09.283+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:09.900 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 26 08:41:09 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-3[50456]: 2024-03-26T08:41:09.601+0000 7f92b3400700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.684882+0000 front 2024-03-26T08:40:41.684826+0000 (oldest deadline 2024-03-26T08:41:07.585038+0000) 2024-03-26T08:41:09.900 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:09 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:09.654+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:10.251 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:10 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:10.057+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:10.251 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:09 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:09.816+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.462806+0000 front 2024-03-26T08:40:40.462545+0000 (oldest deadline 2024-03-26T08:41:05.162732+0000) 2024-03-26T08:41:10.321 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:09 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:09.890+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:10.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:10 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:10.259+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:10.900 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:10 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:10.606+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:10.901 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 26 08:41:10 smithi043 podman[86269]: 2024-03-26 08:41:10.630420759 +0000 UTC m=+5.682405425 container died 3ae00888cfc22ffa711ecf42a7e48e75c0aa45827f14e017c493bf79305957c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-26T08:41:11.045 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:10 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:10.772+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.462806+0000 front 2024-03-26T08:40:40.462545+0000 (oldest deadline 2024-03-26T08:41:05.162732+0000) 2024-03-26T08:41:11.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:11 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:11.231+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:11.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:11 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:11.053+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:11.321 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:10 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:10.845+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:11.400 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 26 08:41:11 smithi043 podman[86269]: 2024-03-26 08:41:11.106354194 +0000 UTC m=+6.158338872 container cleanup 3ae00888cfc22ffa711ecf42a7e48e75c0aa45827f14e017c493bf79305957c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-3, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2) 2024-03-26T08:41:11.400 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 26 08:41:11 smithi043 bash[86269]: ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-3 2024-03-26T08:41:11.900 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 26 08:41:11 smithi043 podman[86287]: 2024-03-26 08:41:11.582726261 +0000 UTC m=+0.937380478 container remove 3ae00888cfc22ffa711ecf42a7e48e75c0aa45827f14e017c493bf79305957c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T08:41:11.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:11 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:11.650+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:12.023 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:11 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:11.769+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.462806+0000 front 2024-03-26T08:40:40.462545+0000 (oldest deadline 2024-03-26T08:41:05.162732+0000) 2024-03-26T08:41:12.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:12 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:12.209+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:12.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:12 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:12.030+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:12.321 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:11 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:11.848+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:12.604 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 26 08:41:12 smithi043 podman[86374]: 2024-03-26 08:41:12.226085861 +0000 UTC m=+0.035199122 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:41:12.604 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 26 08:41:12 smithi043 podman[86374]: 2024-03-26 08:41:12.383710454 +0000 UTC m=+0.192823737 container create 23373b40b0f869792dab45ea6da231ca07cd8d5da47202c4e78ad836bb7c16df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-26T08:41:12.900 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:12 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:12.696+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:12.900 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 26 08:41:12 smithi043 podman[86374]: 2024-03-26 08:41:12.612368962 +0000 UTC m=+0.421482252 container init 23373b40b0f869792dab45ea6da231ca07cd8d5da47202c4e78ad836bb7c16df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T08:41:12.901 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 26 08:41:12 smithi043 podman[86374]: 2024-03-26 08:41:12.623585679 +0000 UTC m=+0.432698932 container start 23373b40b0f869792dab45ea6da231ca07cd8d5da47202c4e78ad836bb7c16df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T08:41:12.901 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 26 08:41:12 smithi043 podman[86374]: 2024-03-26 08:41:12.778023937 +0000 UTC m=+0.587137190 container attach 23373b40b0f869792dab45ea6da231ca07cd8d5da47202c4e78ad836bb7c16df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-26T08:41:13.047 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:12 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:12.745+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.462806+0000 front 2024-03-26T08:40:40.462545+0000 (oldest deadline 2024-03-26T08:41:05.162732+0000) 2024-03-26T08:41:13.321 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:12 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:12.862+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:13.400 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 26 08:41:13 smithi043 podman[86374]: 2024-03-26 08:41:13.083256294 +0000 UTC m=+0.892369584 container died 23373b40b0f869792dab45ea6da231ca07cd8d5da47202c4e78ad836bb7c16df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-03-26T08:41:13.547 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:13 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:13.069+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:13.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:13 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:13.203+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:13.724 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:13 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:13.661+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:13.724 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:13 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:13.661+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:13.759 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.3.service' 2024-03-26T08:41:13.876 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:41:13.876 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-26T08:41:13.876 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-26T08:41:13.877 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.4 2024-03-26T08:41:13.971 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:13 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:13.653+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:13.971 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:13 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:13.832+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:13.972 INFO:journalctl@ceph.osd.4.smithi052.stdout:Mar 26 08:41:13 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-4[46638]: 2024-03-26T08:41:13.798+0000 7f3e0d000700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:47.354991+0000 front 2024-03-26T08:40:47.354508+0000 (oldest deadline 2024-03-26T08:41:13.156421+0000) 2024-03-26T08:41:14.047 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:13 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:13.730+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.462806+0000 front 2024-03-26T08:40:40.462545+0000 (oldest deadline 2024-03-26T08:41:05.162732+0000) 2024-03-26T08:41:14.321 INFO:journalctl@ceph.osd.4.smithi052.stdout:Mar 26 08:41:14 smithi052 systemd[1]: Stopping Ceph osd.4 for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:41:14.547 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:14 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:14.054+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:14.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:14 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:14.224+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:14.548 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:14 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:14.224+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:14.764 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:14 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:14.629+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:14.764 INFO:journalctl@ceph.osd.4.smithi052.stdout:Mar 26 08:41:14 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-4[46638]: 2024-03-26T08:41:14.507+0000 7f3e16800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:41:14.764 INFO:journalctl@ceph.osd.4.smithi052.stdout:Mar 26 08:41:14 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-4[46638]: 2024-03-26T08:41:14.507+0000 7f3e16800700 -1 osd.4 98 *** Got signal Terminated *** 2024-03-26T08:41:14.764 INFO:journalctl@ceph.osd.4.smithi052.stdout:Mar 26 08:41:14 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-4[46638]: 2024-03-26T08:41:14.507+0000 7f3e16800700 -1 osd.4 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T08:41:14.764 INFO:journalctl@ceph.osd.4.smithi052.stdout:Mar 26 08:41:14 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-4[46638]: 2024-03-26T08:41:14.772+0000 7f3e0d000700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:47.354991+0000 front 2024-03-26T08:40:47.354508+0000 (oldest deadline 2024-03-26T08:41:13.156421+0000) 2024-03-26T08:41:15.033 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:14 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:14.684+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.462806+0000 front 2024-03-26T08:40:40.462545+0000 (oldest deadline 2024-03-26T08:41:05.162732+0000) 2024-03-26T08:41:15.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:14 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:14.809+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:15.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:14 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:14.693+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:15.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:14 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:14.693+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:15.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:15 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:15.042+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:15.308 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:15 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:15.223+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:15.308 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:15 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:15.223+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:16.047 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:15 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:15.659+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.462806+0000 front 2024-03-26T08:40:40.462545+0000 (oldest deadline 2024-03-26T08:41:05.162732+0000) 2024-03-26T08:41:16.071 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:15 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:15.610+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:16.071 INFO:journalctl@ceph.osd.4.smithi052.stdout:Mar 26 08:41:15 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-4[46638]: 2024-03-26T08:41:15.797+0000 7f3e0d000700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:47.354991+0000 front 2024-03-26T08:40:47.354508+0000 (oldest deadline 2024-03-26T08:41:13.156421+0000) 2024-03-26T08:41:16.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:15 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:15.767+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:16.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:15 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:15.767+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:16.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:15 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:15.681+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:16.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:15 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:15.681+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:16.547 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:16 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:16.061+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:16.547 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:16 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:16.061+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:16.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:16 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:16.214+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:16.548 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:16 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:16.214+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:17.029 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:16 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:16.632+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.462806+0000 front 2024-03-26T08:40:40.462545+0000 (oldest deadline 2024-03-26T08:41:05.162732+0000) 2024-03-26T08:41:17.029 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:16 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:16.632+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:51.368128+0000 front 2024-03-26T08:40:51.368555+0000 (oldest deadline 2024-03-26T08:41:16.068227+0000) 2024-03-26T08:41:17.071 INFO:journalctl@ceph.osd.4.smithi052.stdout:Mar 26 08:41:16 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-4[46638]: 2024-03-26T08:41:16.830+0000 7f3e0d000700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:47.354991+0000 front 2024-03-26T08:40:47.354508+0000 (oldest deadline 2024-03-26T08:41:13.156421+0000) 2024-03-26T08:41:17.071 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:16 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:16.640+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:17.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:16 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:16.779+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:17.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:16 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:16.779+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:17.150 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:16 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:16.701+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.585253+0000 front 2024-03-26T08:40:50.585181+0000 (oldest deadline 2024-03-26T08:41:16.485418+0000) 2024-03-26T08:41:17.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:16 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:16.714+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:17.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:16 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:16.714+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:17.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:17 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:17.037+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:17.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:17 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:17.037+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:17.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:17 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:17.222+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:17.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:17 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:17.222+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:18.040 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:17 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:17.644+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.462806+0000 front 2024-03-26T08:40:40.462545+0000 (oldest deadline 2024-03-26T08:41:05.162732+0000) 2024-03-26T08:41:18.040 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:17 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:17.644+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:51.368128+0000 front 2024-03-26T08:40:51.368555+0000 (oldest deadline 2024-03-26T08:41:16.068227+0000) 2024-03-26T08:41:18.071 INFO:journalctl@ceph.osd.4.smithi052.stdout:Mar 26 08:41:17 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-4[46638]: 2024-03-26T08:41:17.789+0000 7f3e0d000700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:47.354991+0000 front 2024-03-26T08:40:47.354508+0000 (oldest deadline 2024-03-26T08:41:13.156421+0000) 2024-03-26T08:41:18.071 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:17 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:17.605+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:18.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:17 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:17.729+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:18.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:17 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:17.729+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:18.150 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:17 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:17.702+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.585253+0000 front 2024-03-26T08:40:50.585181+0000 (oldest deadline 2024-03-26T08:41:16.485418+0000) 2024-03-26T08:41:18.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:17 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:17.731+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:18.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:17 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:17.731+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:18.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:18 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:18.048+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:18.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:18 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:18.048+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:18.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:18 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:18.259+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:18.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:18 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:18.259+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:18.900 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:18 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:18.653+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.585253+0000 front 2024-03-26T08:40:50.585181+0000 (oldest deadline 2024-03-26T08:41:16.485418+0000) 2024-03-26T08:41:18.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:18 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:18.720+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:18.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:18 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:18.720+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:19.044 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:18 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:18.675+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.462806+0000 front 2024-03-26T08:40:40.462545+0000 (oldest deadline 2024-03-26T08:41:05.162732+0000) 2024-03-26T08:41:19.044 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:18 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:18.675+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:51.368128+0000 front 2024-03-26T08:40:51.368555+0000 (oldest deadline 2024-03-26T08:41:16.068227+0000) 2024-03-26T08:41:19.071 INFO:journalctl@ceph.osd.4.smithi052.stdout:Mar 26 08:41:18 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-4[46638]: 2024-03-26T08:41:18.764+0000 7f3e0d000700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:47.354991+0000 front 2024-03-26T08:40:47.354508+0000 (oldest deadline 2024-03-26T08:41:13.156421+0000) 2024-03-26T08:41:19.071 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:18 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:18.630+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:19.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:18 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:18.694+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:19.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:18 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:18.694+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:19.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:19 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:19.259+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:19.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:19 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:19.259+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:19.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:19 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:19.052+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:19.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:19 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:19.052+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:19.882 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:19 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:19.690+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:19.882 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:19 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:19.690+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:19.882 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:19 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:19.630+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:19.900 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:19 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:19.650+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.585253+0000 front 2024-03-26T08:40:50.585181+0000 (oldest deadline 2024-03-26T08:41:16.485418+0000) 2024-03-26T08:41:19.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:19 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:19.686+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:19.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:19 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:19.686+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:20.047 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:19 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:19.654+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.462806+0000 front 2024-03-26T08:40:40.462545+0000 (oldest deadline 2024-03-26T08:41:05.162732+0000) 2024-03-26T08:41:20.047 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:19 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:19.654+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:51.368128+0000 front 2024-03-26T08:40:51.368555+0000 (oldest deadline 2024-03-26T08:41:16.068227+0000) 2024-03-26T08:41:20.195 INFO:journalctl@ceph.osd.4.smithi052.stdout:Mar 26 08:41:19 smithi052 podman[68864]: 2024-03-26 08:41:19.889962298 +0000 UTC m=+5.551702438 container died 25370584c031d52afd58b650b6f4be4f2d8091bf7c66867afe1d54947db5c639 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-4, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T08:41:20.516 INFO:journalctl@ceph.osd.4.smithi052.stdout:Mar 26 08:41:20 smithi052 podman[68864]: 2024-03-26 08:41:20.315382697 +0000 UTC m=+5.977122850 container cleanup 25370584c031d52afd58b650b6f4be4f2d8091bf7c66867afe1d54947db5c639 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-4, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-26T08:41:20.517 INFO:journalctl@ceph.osd.4.smithi052.stdout:Mar 26 08:41:20 smithi052 bash[68864]: ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-4 2024-03-26T08:41:20.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:20 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:20.240+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:20.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:20 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:20.240+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:20.547 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:20 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:20.091+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:20.547 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:20 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:20.091+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:20.821 INFO:journalctl@ceph.osd.4.smithi052.stdout:Mar 26 08:41:20 smithi052 podman[68881]: 2024-03-26 08:41:20.524936838 +0000 UTC m=+0.617113568 container remove 25370584c031d52afd58b650b6f4be4f2d8091bf7c66867afe1d54947db5c639 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD) 2024-03-26T08:41:20.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:20 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:20.690+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:20.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:20 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:20.690+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:20.822 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:20 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:20.611+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:21.047 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:20 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:20.641+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.462806+0000 front 2024-03-26T08:40:40.462545+0000 (oldest deadline 2024-03-26T08:41:05.162732+0000) 2024-03-26T08:41:21.047 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:20 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:20.641+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:51.368128+0000 front 2024-03-26T08:40:51.368555+0000 (oldest deadline 2024-03-26T08:41:16.068227+0000) 2024-03-26T08:41:21.150 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:20 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:20.686+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.585253+0000 front 2024-03-26T08:40:50.585181+0000 (oldest deadline 2024-03-26T08:41:16.485418+0000) 2024-03-26T08:41:21.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:20 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:20.699+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:21.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:20 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:20.699+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:21.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:21 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:21.194+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:21.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:21 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:21.194+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:21.547 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:21 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:21.065+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:21.547 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:21 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:21.065+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:21.663 INFO:journalctl@ceph.osd.4.smithi052.stdout:Mar 26 08:41:21 smithi052 podman[68971]: 2024-03-26 08:41:21.322380809 +0000 UTC m=+0.043267601 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:41:21.663 INFO:journalctl@ceph.osd.4.smithi052.stdout:Mar 26 08:41:21 smithi052 podman[68971]: 2024-03-26 08:41:21.45928602 +0000 UTC m=+0.180172826 container create 1b0f40b23842c0c2ffc5ca2ce8f2477cf336ad4689a15585b14c6f9e31c4ac9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T08:41:21.664 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:21 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:21.564+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:22.003 INFO:journalctl@ceph.osd.4.smithi052.stdout:Mar 26 08:41:21 smithi052 podman[68971]: 2024-03-26 08:41:21.896750102 +0000 UTC m=+0.617636911 container init 1b0f40b23842c0c2ffc5ca2ce8f2477cf336ad4689a15585b14c6f9e31c4ac9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True) 2024-03-26T08:41:22.003 INFO:journalctl@ceph.osd.4.smithi052.stdout:Mar 26 08:41:21 smithi052 podman[68971]: 2024-03-26 08:41:21.907846136 +0000 UTC m=+0.628732941 container start 1b0f40b23842c0c2ffc5ca2ce8f2477cf336ad4689a15585b14c6f9e31c4ac9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319) 2024-03-26T08:41:22.004 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:21 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:21.693+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:22.004 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:21 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:21.693+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:22.004 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:21 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:21.693+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:22.042 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:21 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:21.674+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.462806+0000 front 2024-03-26T08:40:40.462545+0000 (oldest deadline 2024-03-26T08:41:05.162732+0000) 2024-03-26T08:41:22.042 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:21 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:21.674+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:51.368128+0000 front 2024-03-26T08:40:51.368555+0000 (oldest deadline 2024-03-26T08:41:16.068227+0000) 2024-03-26T08:41:22.150 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:21 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:21.734+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.585253+0000 front 2024-03-26T08:40:50.585181+0000 (oldest deadline 2024-03-26T08:41:16.485418+0000) 2024-03-26T08:41:22.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:21 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:21.743+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:22.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:21 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:21.743+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:22.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:22 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:22.051+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:22.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:22 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:22.051+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:22.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:22 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:22.051+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:22.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:22 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:22.215+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:22.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:22 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:22.215+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:22.321 INFO:journalctl@ceph.osd.4.smithi052.stdout:Mar 26 08:41:22 smithi052 podman[68971]: 2024-03-26 08:41:22.012078835 +0000 UTC m=+0.732965694 container attach 1b0f40b23842c0c2ffc5ca2ce8f2477cf336ad4689a15585b14c6f9e31c4ac9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-26T08:41:22.655 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:22 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:22.563+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:22.655 INFO:journalctl@ceph.osd.4.smithi052.stdout:Mar 26 08:41:22 smithi052 podman[68971]: 2024-03-26 08:41:22.357281552 +0000 UTC m=+1.078168361 container died 1b0f40b23842c0c2ffc5ca2ce8f2477cf336ad4689a15585b14c6f9e31c4ac9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-26T08:41:23.039 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:22 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:22.698+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.462806+0000 front 2024-03-26T08:40:40.462545+0000 (oldest deadline 2024-03-26T08:41:05.162732+0000) 2024-03-26T08:41:23.039 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:22 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:22.699+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:51.368128+0000 front 2024-03-26T08:40:51.368555+0000 (oldest deadline 2024-03-26T08:41:16.068227+0000) 2024-03-26T08:41:23.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:22 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:22.663+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:23.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:22 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:22.663+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:23.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:22 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:22.663+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:23.150 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:22 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:22.726+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.585253+0000 front 2024-03-26T08:40:50.585181+0000 (oldest deadline 2024-03-26T08:41:16.485418+0000) 2024-03-26T08:41:23.150 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:22 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:22.726+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.486558+0000 front 2024-03-26T08:40:56.486670+0000 (oldest deadline 2024-03-26T08:41:21.786776+0000) 2024-03-26T08:41:23.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:22 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:22.701+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:23.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:22 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:22.701+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:23.297 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.4.service' 2024-03-26T08:41:23.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:23 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:23.179+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:23.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:23 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:23.179+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:23.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:23 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:23.048+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:23.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:23 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:23.048+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:23.299 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:23 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:23.048+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:23.422 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:41:23.422 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-26T08:41:23.422 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-26T08:41:23.422 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.5 2024-03-26T08:41:23.797 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:23 smithi183 systemd[1]: Stopping Ceph osd.5 for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:41:23.797 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:23 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:23.742+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.462806+0000 front 2024-03-26T08:40:40.462545+0000 (oldest deadline 2024-03-26T08:41:05.162732+0000) 2024-03-26T08:41:23.797 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:23 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:23.742+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:51.368128+0000 front 2024-03-26T08:40:51.368555+0000 (oldest deadline 2024-03-26T08:41:16.068227+0000) 2024-03-26T08:41:23.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:23 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:23.546+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:23.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:23 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:23.546+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.154120+0000 front 2024-03-26T08:40:57.154394+0000 (oldest deadline 2024-03-26T08:41:23.053964+0000) 2024-03-26T08:41:23.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:23 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:23.641+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:23.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:23 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:23.641+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:23.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:23 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:23.641+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:23.900 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:23 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:23.682+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.585253+0000 front 2024-03-26T08:40:50.585181+0000 (oldest deadline 2024-03-26T08:41:16.485418+0000) 2024-03-26T08:41:23.901 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:23 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:23.682+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.486558+0000 front 2024-03-26T08:40:56.486670+0000 (oldest deadline 2024-03-26T08:41:21.786776+0000) 2024-03-26T08:41:23.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:23 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:23.691+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:23.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:23 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:23.691+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:23.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:23 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:23.691+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:24.297 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:23 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:23.975+0000 7f32fbc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:41:24.297 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:23 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:23.975+0000 7f32fbc00700 -1 osd.5 98 *** Got signal Terminated *** 2024-03-26T08:41:24.297 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:23 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:23.975+0000 7f32fbc00700 -1 osd.5 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T08:41:24.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:24 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:24.079+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:24.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:24 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:24.079+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:24.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:24 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:24.079+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:24.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:24 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:24.203+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:24.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:24 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:24.203+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:24.900 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:24 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:24.674+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.585253+0000 front 2024-03-26T08:40:50.585181+0000 (oldest deadline 2024-03-26T08:41:16.485418+0000) 2024-03-26T08:41:24.901 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:24 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:24.674+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.486558+0000 front 2024-03-26T08:40:56.486670+0000 (oldest deadline 2024-03-26T08:41:21.786776+0000) 2024-03-26T08:41:24.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:24 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:24.654+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:24.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:24 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:24.654+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:24.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:24 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:24.654+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:25.047 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:24 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:24.791+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.462806+0000 front 2024-03-26T08:40:40.462545+0000 (oldest deadline 2024-03-26T08:41:05.162732+0000) 2024-03-26T08:41:25.047 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:24 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:24.791+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:51.368128+0000 front 2024-03-26T08:40:51.368555+0000 (oldest deadline 2024-03-26T08:41:16.068227+0000) 2024-03-26T08:41:25.047 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:24 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:24.791+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.571541+0000 front 2024-03-26T08:41:00.571210+0000 (oldest deadline 2024-03-26T08:41:24.071519+0000) 2024-03-26T08:41:25.071 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:24 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:24.588+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:25.071 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:24 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:24.588+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.154120+0000 front 2024-03-26T08:40:57.154394+0000 (oldest deadline 2024-03-26T08:41:23.053964+0000) 2024-03-26T08:41:25.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:24 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:24.642+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:25.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:24 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:24.642+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:25.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:24 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:24.642+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:25.547 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:25 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:25.096+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:25.547 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:25 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:25.096+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:25.547 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:25 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:25.096+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:25.548 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:25 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:25.160+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:25.548 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:25 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:25.160+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:25.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:25 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:25.576+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:25.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:25 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:25.576+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.154120+0000 front 2024-03-26T08:40:57.154394+0000 (oldest deadline 2024-03-26T08:41:23.053964+0000) 2024-03-26T08:41:25.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:25 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:25.634+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:25.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:25 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:25.634+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:25.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:25 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:25.634+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:25.900 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:25 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:25.694+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.585253+0000 front 2024-03-26T08:40:50.585181+0000 (oldest deadline 2024-03-26T08:41:16.485418+0000) 2024-03-26T08:41:25.900 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:25 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:25.694+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.486558+0000 front 2024-03-26T08:40:56.486670+0000 (oldest deadline 2024-03-26T08:41:21.786776+0000) 2024-03-26T08:41:25.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:25 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:25.613+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:25.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:25 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:25.613+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:25.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:25 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:25.613+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:26.047 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:25 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:25.778+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.462806+0000 front 2024-03-26T08:40:40.462545+0000 (oldest deadline 2024-03-26T08:41:05.162732+0000) 2024-03-26T08:41:26.047 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:25 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:25.778+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:51.368128+0000 front 2024-03-26T08:40:51.368555+0000 (oldest deadline 2024-03-26T08:41:16.068227+0000) 2024-03-26T08:41:26.047 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:25 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:25.778+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.571541+0000 front 2024-03-26T08:41:00.571210+0000 (oldest deadline 2024-03-26T08:41:24.071519+0000) 2024-03-26T08:41:26.547 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:26 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:26.092+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:26.547 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:26 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:26.092+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:26.547 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:26 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:26.092+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:26.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:26 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:26.131+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:26.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:26 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:26.131+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:26.548 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:26 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:26.131+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:26.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:26 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:26.566+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:26.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:26 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:26.566+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.154120+0000 front 2024-03-26T08:40:57.154394+0000 (oldest deadline 2024-03-26T08:41:23.053964+0000) 2024-03-26T08:41:26.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:26 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:26.615+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:26.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:26 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:26.615+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:26.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:26 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:26.615+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:26.900 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:26 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:26.726+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.585253+0000 front 2024-03-26T08:40:50.585181+0000 (oldest deadline 2024-03-26T08:41:16.485418+0000) 2024-03-26T08:41:26.901 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:26 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:26.726+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.486558+0000 front 2024-03-26T08:40:56.486670+0000 (oldest deadline 2024-03-26T08:41:21.786776+0000) 2024-03-26T08:41:26.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:26 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:26.619+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:26.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:26 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:26.619+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:26.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:26 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:26.619+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:27.047 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:26 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:26.804+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.462806+0000 front 2024-03-26T08:40:40.462545+0000 (oldest deadline 2024-03-26T08:41:05.162732+0000) 2024-03-26T08:41:27.047 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:26 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:26.804+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:51.368128+0000 front 2024-03-26T08:40:51.368555+0000 (oldest deadline 2024-03-26T08:41:16.068227+0000) 2024-03-26T08:41:27.047 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:26 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:26.804+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.571541+0000 front 2024-03-26T08:41:00.571210+0000 (oldest deadline 2024-03-26T08:41:24.071519+0000) 2024-03-26T08:41:27.547 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:27 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:27.111+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:27.547 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:27 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:27.111+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:27.548 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:27 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:27.111+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:27.548 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:27 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:27.107+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:27.548 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:27 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:27.107+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:27.548 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:27 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:27.107+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:27.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:27 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:27.534+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:27.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:27 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:27.534+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.154120+0000 front 2024-03-26T08:40:57.154394+0000 (oldest deadline 2024-03-26T08:41:23.053964+0000) 2024-03-26T08:41:27.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:27 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:27.627+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:27.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:27 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:27.627+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:27.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:27 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:27.627+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:28.047 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:27 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:27.782+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.462806+0000 front 2024-03-26T08:40:40.462545+0000 (oldest deadline 2024-03-26T08:41:05.162732+0000) 2024-03-26T08:41:28.047 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:27 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:27.782+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:51.368128+0000 front 2024-03-26T08:40:51.368555+0000 (oldest deadline 2024-03-26T08:41:16.068227+0000) 2024-03-26T08:41:28.047 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:27 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:27.782+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.571541+0000 front 2024-03-26T08:41:00.571210+0000 (oldest deadline 2024-03-26T08:41:24.071519+0000) 2024-03-26T08:41:28.150 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:27 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:27.728+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.585253+0000 front 2024-03-26T08:40:50.585181+0000 (oldest deadline 2024-03-26T08:41:16.485418+0000) 2024-03-26T08:41:28.150 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:27 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:27.728+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.486558+0000 front 2024-03-26T08:40:56.486670+0000 (oldest deadline 2024-03-26T08:41:21.786776+0000) 2024-03-26T08:41:28.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:27 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:27.666+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:28.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:27 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:27.666+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:28.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:27 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:27.666+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:28.547 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:28 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:28.070+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:28.547 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:28 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:28.070+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:28.548 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:28 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:28.070+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:28.548 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:28 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:28.155+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:28.548 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:28 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:28.155+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:28.548 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:28 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:28.155+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:28.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:28 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:28.539+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:28.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:28 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:28.539+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.154120+0000 front 2024-03-26T08:40:57.154394+0000 (oldest deadline 2024-03-26T08:41:23.053964+0000) 2024-03-26T08:41:28.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:28 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:28.633+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:28.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:28 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:28.633+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:28.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:28 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:28.633+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:28.900 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:28 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:28.698+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.585253+0000 front 2024-03-26T08:40:50.585181+0000 (oldest deadline 2024-03-26T08:41:16.485418+0000) 2024-03-26T08:41:28.900 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:28 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:28.698+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.486558+0000 front 2024-03-26T08:40:56.486670+0000 (oldest deadline 2024-03-26T08:41:21.786776+0000) 2024-03-26T08:41:28.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:28 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:28.618+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:28.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:28 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:28.618+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:28.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:28 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:28.618+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:29.035 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:28 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:28.788+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.462806+0000 front 2024-03-26T08:40:40.462545+0000 (oldest deadline 2024-03-26T08:41:05.162732+0000) 2024-03-26T08:41:29.036 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:28 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:28.788+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:51.368128+0000 front 2024-03-26T08:40:51.368555+0000 (oldest deadline 2024-03-26T08:41:16.068227+0000) 2024-03-26T08:41:29.036 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:28 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5[47874]: 2024-03-26T08:41:28.788+0000 7f32f2400700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.571541+0000 front 2024-03-26T08:41:00.571210+0000 (oldest deadline 2024-03-26T08:41:24.071519+0000) 2024-03-26T08:41:29.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:29 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:29.146+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:29.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:29 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:29.146+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:29.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:29 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:29.146+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:29.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:29 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:29.043+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:29.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:29 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:29.043+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:29.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:29 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:29.043+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:29.797 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:29 smithi183 podman[69985]: 2024-03-26 08:41:29.55130662 +0000 UTC m=+5.726168497 container died f6e9d594387f5774b72db19e8dc2a1e598fa6accc5702f3c175eccda3c3ecf35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, CEPH_POINT_RELEASE=) 2024-03-26T08:41:29.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:29 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:29.515+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:29.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:29 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:29.515+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.154120+0000 front 2024-03-26T08:40:57.154394+0000 (oldest deadline 2024-03-26T08:41:23.053964+0000) 2024-03-26T08:41:29.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:29 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:29.601+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:29.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:29 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:29.601+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:29.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:29 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:29.601+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:29.900 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:29 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:29.736+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.585253+0000 front 2024-03-26T08:40:50.585181+0000 (oldest deadline 2024-03-26T08:41:16.485418+0000) 2024-03-26T08:41:29.901 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:29 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:29.736+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.486558+0000 front 2024-03-26T08:40:56.486670+0000 (oldest deadline 2024-03-26T08:41:21.786776+0000) 2024-03-26T08:41:29.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:29 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:29.596+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:29.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:29 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:29.596+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:29.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:29 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:29.596+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:30.074 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:29 smithi183 podman[69985]: 2024-03-26 08:41:29.918416488 +0000 UTC m=+6.093278390 container cleanup f6e9d594387f5774b72db19e8dc2a1e598fa6accc5702f3c175eccda3c3ecf35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-26T08:41:30.075 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:29 smithi183 bash[69985]: ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5 2024-03-26T08:41:30.547 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:30 smithi183 podman[70003]: 2024-03-26 08:41:30.178032682 +0000 UTC m=+0.611766082 container remove f6e9d594387f5774b72db19e8dc2a1e598fa6accc5702f3c175eccda3c3ecf35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-26T08:41:30.548 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:30 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:30.082+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:30.548 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:30 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:30.082+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:30.548 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:30 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:30.082+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:30.548 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:30 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:30.181+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:30.548 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:30 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:30.181+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:30.548 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:30 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:30.181+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:30.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:30 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:30.529+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:30.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:30 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:30.529+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.154120+0000 front 2024-03-26T08:40:57.154394+0000 (oldest deadline 2024-03-26T08:41:23.053964+0000) 2024-03-26T08:41:30.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:30 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:30.615+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:30.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:30 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:30.615+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:30.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:30 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:30.615+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:30.900 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:30 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:30.766+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.585253+0000 front 2024-03-26T08:40:50.585181+0000 (oldest deadline 2024-03-26T08:41:16.485418+0000) 2024-03-26T08:41:30.900 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:30 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:30.766+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.486558+0000 front 2024-03-26T08:40:56.486670+0000 (oldest deadline 2024-03-26T08:41:21.786776+0000) 2024-03-26T08:41:30.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:30 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:30.631+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:30.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:30 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:30.631+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:30.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:30 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:30.631+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:31.215 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:31 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:31.133+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:31.216 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:31 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:31.133+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:31.216 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:31 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:31.133+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:31.216 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:30 smithi183 podman[70089]: 2024-03-26 08:41:30.834447054 +0000 UTC m=+0.032332689 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:41:31.216 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:30 smithi183 podman[70089]: 2024-03-26 08:41:30.945838604 +0000 UTC m=+0.143724276 container create 4bc3f5bb1c21e244294c3e5c8d70d8975af6f64ac8a643579e63f9fab913649e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T08:41:31.216 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:31 smithi183 podman[70089]: 2024-03-26 08:41:31.224431622 +0000 UTC m=+0.422317300 container init 4bc3f5bb1c21e244294c3e5c8d70d8975af6f64ac8a643579e63f9fab913649e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-26T08:41:31.216 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:31 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:31.131+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:31.217 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:31 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:31.131+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:31.217 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:31 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:31.131+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:31.549 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:31 smithi183 podman[70089]: 2024-03-26 08:41:31.236960471 +0000 UTC m=+0.434846150 container start 4bc3f5bb1c21e244294c3e5c8d70d8975af6f64ac8a643579e63f9fab913649e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True) 2024-03-26T08:41:31.550 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:31 smithi183 podman[70089]: 2024-03-26 08:41:31.323385758 +0000 UTC m=+0.521271437 container attach 4bc3f5bb1c21e244294c3e5c8d70d8975af6f64ac8a643579e63f9fab913649e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5-deactivate, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T08:41:31.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:31 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:31.502+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:31.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:31 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:31.502+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.154120+0000 front 2024-03-26T08:40:57.154394+0000 (oldest deadline 2024-03-26T08:41:23.053964+0000) 2024-03-26T08:41:31.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:31 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:31.649+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:31.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:31 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:31.649+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:31.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:31 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:31.649+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:32.017 INFO:journalctl@ceph.osd.5.smithi183.stdout:Mar 26 08:41:31 smithi183 podman[70089]: 2024-03-26 08:41:31.658795888 +0000 UTC m=+0.856681603 container died 4bc3f5bb1c21e244294c3e5c8d70d8975af6f64ac8a643579e63f9fab913649e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T08:41:32.150 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:31 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:31.770+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.585253+0000 front 2024-03-26T08:40:50.585181+0000 (oldest deadline 2024-03-26T08:41:16.485418+0000) 2024-03-26T08:41:32.150 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:31 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:31.770+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.486558+0000 front 2024-03-26T08:40:56.486670+0000 (oldest deadline 2024-03-26T08:41:21.786776+0000) 2024-03-26T08:41:32.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:31 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:31.673+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:32.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:31 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:31.673+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:32.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:31 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:31.673+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:32.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:31 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:31.673+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:41:32.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:32 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:32.117+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:32.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:32 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:32.117+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:32.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:32 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:32.117+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:32.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:32 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:32.154+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:32.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:32 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:32.154+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:32.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:32 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:32.154+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:32.426 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.5.service' 2024-03-26T08:41:32.532 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:41:32.532 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-26T08:41:32.532 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-26T08:41:32.533 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.6 2024-03-26T08:41:32.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:32 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:32.496+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:32.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:32 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:32.496+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.154120+0000 front 2024-03-26T08:40:57.154394+0000 (oldest deadline 2024-03-26T08:41:23.053964+0000) 2024-03-26T08:41:32.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:32 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:32.496+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:06.358054+0000 front 2024-03-26T08:41:06.357924+0000 (oldest deadline 2024-03-26T08:41:31.657483+0000) 2024-03-26T08:41:32.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:32 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:32.624+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:32.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:32 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:32.624+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:32.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:32 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:32.624+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:32.901 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:32 smithi043 systemd[1]: Stopping Ceph osd.6 for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:41:32.902 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:32 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:32.738+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.585253+0000 front 2024-03-26T08:40:50.585181+0000 (oldest deadline 2024-03-26T08:41:16.485418+0000) 2024-03-26T08:41:32.902 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:32 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:32.738+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.486558+0000 front 2024-03-26T08:40:56.486670+0000 (oldest deadline 2024-03-26T08:41:21.786776+0000) 2024-03-26T08:41:32.902 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:32 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:32.738+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.992269+0000 front 2024-03-26T08:41:07.992556+0000 (oldest deadline 2024-03-26T08:41:32.692332+0000) 2024-03-26T08:41:32.902 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:32 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:32.710+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:32.902 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:32 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:32.710+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:32.902 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:32 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:32.710+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:32.902 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:32 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:32.710+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:41:33.400 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:33 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:33.156+0000 7fefc1800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:41:33.401 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:33 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:33.156+0000 7fefc1800700 -1 osd.6 98 *** Got signal Terminated *** 2024-03-26T08:41:33.401 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:33 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:33.156+0000 7fefc1800700 -1 osd.6 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T08:41:33.547 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:33 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:33.097+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:33.547 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:33 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:33.097+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:33.547 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:33 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:33.097+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:33.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:33 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:33.115+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:33.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:33 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:33.115+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:33.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:33 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:33.115+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:33.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:33 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:33.460+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:33.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:33 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:33.460+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.154120+0000 front 2024-03-26T08:40:57.154394+0000 (oldest deadline 2024-03-26T08:41:23.053964+0000) 2024-03-26T08:41:33.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:33 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:33.460+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:06.358054+0000 front 2024-03-26T08:41:06.357924+0000 (oldest deadline 2024-03-26T08:41:31.657483+0000) 2024-03-26T08:41:33.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:33 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:33.672+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:33.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:33 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:33.672+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:33.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:33 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:33.672+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:33.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:33 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:33.672+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:41:34.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:33 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:33.700+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:34.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:33 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:33.700+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:34.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:33 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:33.700+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:34.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:33 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:33.700+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:41:34.151 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:33 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:33.758+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.585253+0000 front 2024-03-26T08:40:50.585181+0000 (oldest deadline 2024-03-26T08:41:16.485418+0000) 2024-03-26T08:41:34.151 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:33 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:33.758+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.486558+0000 front 2024-03-26T08:40:56.486670+0000 (oldest deadline 2024-03-26T08:41:21.786776+0000) 2024-03-26T08:41:34.151 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:33 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:33.758+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.992269+0000 front 2024-03-26T08:41:07.992556+0000 (oldest deadline 2024-03-26T08:41:32.692332+0000) 2024-03-26T08:41:34.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:34 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:34.104+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:34.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:34 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:34.104+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:34.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:34 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:34.104+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:34.547 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:34 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:34.060+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:34.547 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:34 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:34.060+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:34.548 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:34 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:34.060+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:34.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:34 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:34.510+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:34.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:34 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:34.510+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.154120+0000 front 2024-03-26T08:40:57.154394+0000 (oldest deadline 2024-03-26T08:41:23.053964+0000) 2024-03-26T08:41:34.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:34 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:34.510+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:06.358054+0000 front 2024-03-26T08:41:06.357924+0000 (oldest deadline 2024-03-26T08:41:31.657483+0000) 2024-03-26T08:41:34.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:34 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:34.700+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:34.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:34 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:34.700+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:34.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:34 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:34.700+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:34.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:34 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:34.700+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:41:34.901 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:34 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:34.714+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.585253+0000 front 2024-03-26T08:40:50.585181+0000 (oldest deadline 2024-03-26T08:41:16.485418+0000) 2024-03-26T08:41:34.901 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:34 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:34.714+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.486558+0000 front 2024-03-26T08:40:56.486670+0000 (oldest deadline 2024-03-26T08:41:21.786776+0000) 2024-03-26T08:41:34.901 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:34 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:34.714+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.992269+0000 front 2024-03-26T08:41:07.992556+0000 (oldest deadline 2024-03-26T08:41:32.692332+0000) 2024-03-26T08:41:34.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:34 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:34.656+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:34.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:34 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:34.656+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:34.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:34 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:34.656+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:34.902 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:34 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:34.656+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:41:35.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:35 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:35.078+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:35.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:35 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:35.078+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:35.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:35 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:35.078+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:35.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:35 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:35.015+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:35.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:35 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:35.015+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:35.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:35 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:35.015+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:35.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:35 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:35.531+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:35.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:35 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:35.531+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.154120+0000 front 2024-03-26T08:40:57.154394+0000 (oldest deadline 2024-03-26T08:41:23.053964+0000) 2024-03-26T08:41:35.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:35 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:35.531+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:06.358054+0000 front 2024-03-26T08:41:06.357924+0000 (oldest deadline 2024-03-26T08:41:31.657483+0000) 2024-03-26T08:41:35.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:35 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:35.683+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:35.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:35 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:35.683+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:35.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:35 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:35.683+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:35.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:35 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:35.683+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:41:36.150 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:35 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:35.739+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.585253+0000 front 2024-03-26T08:40:50.585181+0000 (oldest deadline 2024-03-26T08:41:16.485418+0000) 2024-03-26T08:41:36.150 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:35 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:35.739+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.486558+0000 front 2024-03-26T08:40:56.486670+0000 (oldest deadline 2024-03-26T08:41:21.786776+0000) 2024-03-26T08:41:36.151 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:35 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:35.739+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.992269+0000 front 2024-03-26T08:41:07.992556+0000 (oldest deadline 2024-03-26T08:41:32.692332+0000) 2024-03-26T08:41:36.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:35 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:35.693+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:36.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:35 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:35.693+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:36.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:35 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:35.693+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:36.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:35 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:35.693+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:41:36.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:36 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:36.110+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:36.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:36 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:36.110+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:36.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:36 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:36.110+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:36.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:35 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:35.974+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:36.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:35 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:35.974+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:36.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:35 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:35.974+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:36.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:35 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:35.974+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:10.090511+0000 front 2024-03-26T08:41:10.090486+0000 (oldest deadline 2024-03-26T08:41:35.389824+0000) 2024-03-26T08:41:36.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:36 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:36.494+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:36.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:36 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:36.494+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.154120+0000 front 2024-03-26T08:40:57.154394+0000 (oldest deadline 2024-03-26T08:41:23.053964+0000) 2024-03-26T08:41:36.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:36 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:36.494+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:06.358054+0000 front 2024-03-26T08:41:06.357924+0000 (oldest deadline 2024-03-26T08:41:31.657483+0000) 2024-03-26T08:41:36.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:36 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:36.726+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:36.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:36 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:36.726+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:36.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:36 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:36.726+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:36.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:36 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:36.726+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:41:37.150 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:36 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:36.725+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.585253+0000 front 2024-03-26T08:40:50.585181+0000 (oldest deadline 2024-03-26T08:41:16.485418+0000) 2024-03-26T08:41:37.151 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:36 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:36.725+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.486558+0000 front 2024-03-26T08:40:56.486670+0000 (oldest deadline 2024-03-26T08:41:21.786776+0000) 2024-03-26T08:41:37.151 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:36 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:36.725+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.992269+0000 front 2024-03-26T08:41:07.992556+0000 (oldest deadline 2024-03-26T08:41:32.692332+0000) 2024-03-26T08:41:37.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:36 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:36.681+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:37.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:36 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:36.681+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:37.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:36 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:36.681+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:37.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:36 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:36.681+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:41:37.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:37 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:37.091+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:37.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:37 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:37.091+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:37.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:37 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:37.091+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:37.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:37 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:37.011+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:37.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:37 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:37.011+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:37.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:37 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:37.011+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:37.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:37 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:37.011+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:10.090511+0000 front 2024-03-26T08:41:10.090486+0000 (oldest deadline 2024-03-26T08:41:35.389824+0000) 2024-03-26T08:41:37.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:37 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:37.517+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:37.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:37 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:37.517+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.154120+0000 front 2024-03-26T08:40:57.154394+0000 (oldest deadline 2024-03-26T08:41:23.053964+0000) 2024-03-26T08:41:37.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:37 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:37.517+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:06.358054+0000 front 2024-03-26T08:41:06.357924+0000 (oldest deadline 2024-03-26T08:41:31.657483+0000) 2024-03-26T08:41:37.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:37 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:37.695+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:37.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:37 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:37.695+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:37.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:37 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:37.695+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:37.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:37 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:37.695+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:41:38.152 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:37 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:37.729+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.585253+0000 front 2024-03-26T08:40:50.585181+0000 (oldest deadline 2024-03-26T08:41:16.485418+0000) 2024-03-26T08:41:38.152 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:37 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:37.729+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.486558+0000 front 2024-03-26T08:40:56.486670+0000 (oldest deadline 2024-03-26T08:41:21.786776+0000) 2024-03-26T08:41:38.152 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:37 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6[57363]: 2024-03-26T08:41:37.729+0000 7fefb8000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.992269+0000 front 2024-03-26T08:41:07.992556+0000 (oldest deadline 2024-03-26T08:41:32.692332+0000) 2024-03-26T08:41:38.152 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:37 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:37.703+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:38.152 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:37 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:37.703+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:38.153 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:37 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:37.703+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:38.153 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:37 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:37.703+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:41:38.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:38 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:38.086+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:38.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:38 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:38.086+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:38.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:38 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:38.086+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:38.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:38 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:38.007+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:38.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:38 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:38.007+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:38.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:38 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:38.007+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:38.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:38 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:38.007+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:10.090511+0000 front 2024-03-26T08:41:10.090486+0000 (oldest deadline 2024-03-26T08:41:35.389824+0000) 2024-03-26T08:41:38.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:38 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:38.481+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:38.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:38 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:38.481+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.154120+0000 front 2024-03-26T08:40:57.154394+0000 (oldest deadline 2024-03-26T08:41:23.053964+0000) 2024-03-26T08:41:38.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:38 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:38.481+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:06.358054+0000 front 2024-03-26T08:41:06.357924+0000 (oldest deadline 2024-03-26T08:41:31.657483+0000) 2024-03-26T08:41:38.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:38 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:38.671+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:38.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:38 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:38.671+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:38.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:38 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:38.671+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:38.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:38 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:38.671+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:41:39.092 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:38 smithi043 podman[86490]: 2024-03-26 08:41:38.780284421 +0000 UTC m=+5.762307151 container died 0b46c1b15612bb44dbda1c5e8e74dcf6b09dbd08c8f3d71362a2b1118c7fa62f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T08:41:39.093 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:38 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:38.660+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:39.093 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:38 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:38.660+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:39.093 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:38 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:38.660+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:39.093 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:38 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:38.660+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:41:39.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:39 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:39.109+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:39.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:39 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:39.109+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:39.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:39 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:39.109+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:39.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:38 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:38.970+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:39.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:38 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:38.970+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:39.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:38 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:38.970+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:39.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:38 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:38.970+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:10.090511+0000 front 2024-03-26T08:41:10.090486+0000 (oldest deadline 2024-03-26T08:41:35.389824+0000) 2024-03-26T08:41:39.400 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:39 smithi043 podman[86490]: 2024-03-26 08:41:39.209214409 +0000 UTC m=+6.191237139 container cleanup 0b46c1b15612bb44dbda1c5e8e74dcf6b09dbd08c8f3d71362a2b1118c7fa62f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T08:41:39.400 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:39 smithi043 bash[86490]: ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6 2024-03-26T08:41:39.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:39 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:39.512+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:39.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:39 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:39.512+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.154120+0000 front 2024-03-26T08:40:57.154394+0000 (oldest deadline 2024-03-26T08:41:23.053964+0000) 2024-03-26T08:41:39.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:39 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:39.512+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:06.358054+0000 front 2024-03-26T08:41:06.357924+0000 (oldest deadline 2024-03-26T08:41:31.657483+0000) 2024-03-26T08:41:39.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:39 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:39.661+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:39.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:39 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:39.661+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:39.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:39 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:39.661+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:39.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:39 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:39.661+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:41:39.900 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:39 smithi043 podman[86511]: 2024-03-26 08:41:39.477305188 +0000 UTC m=+0.681243239 container remove 0b46c1b15612bb44dbda1c5e8e74dcf6b09dbd08c8f3d71362a2b1118c7fa62f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T08:41:39.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:39 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:39.633+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:39.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:39 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:39.633+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:39.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:39 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:39.633+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:39.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:39 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:39.633+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:41:40.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:40 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:40.070+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:40.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:40 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:40.070+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:40.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:40 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:40.070+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:40.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:39 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:39.949+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:40.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:39 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:39.949+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:40.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:39 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:39.949+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:40.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:39 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:39.949+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:10.090511+0000 front 2024-03-26T08:41:10.090486+0000 (oldest deadline 2024-03-26T08:41:35.389824+0000) 2024-03-26T08:41:40.577 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:40 smithi043 podman[86601]: 2024-03-26 08:41:40.225765842 +0000 UTC m=+0.038819103 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:41:40.578 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:40 smithi043 podman[86601]: 2024-03-26 08:41:40.361395597 +0000 UTC m=+0.174448885 container create 120448b113df3a2f18e6cd5d7ea92b7cd80b84b823a3208f752d7ccdf62cae43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-26T08:41:40.748 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:40 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:40.473+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:40.748 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:40 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:40.473+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.154120+0000 front 2024-03-26T08:40:57.154394+0000 (oldest deadline 2024-03-26T08:41:23.053964+0000) 2024-03-26T08:41:40.748 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:40 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:40.473+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:06.358054+0000 front 2024-03-26T08:41:06.357924+0000 (oldest deadline 2024-03-26T08:41:31.657483+0000) 2024-03-26T08:41:40.749 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:40 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:40.473+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:17.961808+0000 front 2024-03-26T08:41:17.961893+0000 (oldest deadline 2024-03-26T08:41:40.261986+0000) 2024-03-26T08:41:40.749 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:40 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:40.707+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:40.749 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:40 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:40.707+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:40.749 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:40 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:40.707+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:40.749 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:40 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:40.707+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:41:40.749 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:40 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:40.707+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:41:40.900 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:40 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:40.585+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:40.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:40 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:40.585+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:40.902 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:40 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:40.585+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:40.902 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:40 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:40.585+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:41:40.903 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:40 smithi043 podman[86601]: 2024-03-26 08:41:40.673865601 +0000 UTC m=+0.486918896 container init 120448b113df3a2f18e6cd5d7ea92b7cd80b84b823a3208f752d7ccdf62cae43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-26T08:41:40.903 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:40 smithi043 podman[86601]: 2024-03-26 08:41:40.686214321 +0000 UTC m=+0.499267579 container start 120448b113df3a2f18e6cd5d7ea92b7cd80b84b823a3208f752d7ccdf62cae43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319) 2024-03-26T08:41:40.903 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:40 smithi043 podman[86601]: 2024-03-26 08:41:40.780560044 +0000 UTC m=+0.593613379 container attach 120448b113df3a2f18e6cd5d7ea92b7cd80b84b823a3208f752d7ccdf62cae43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2) 2024-03-26T08:41:41.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:41 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:41.076+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:41.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:41 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:41.076+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:41.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:41 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:41.076+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:41.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:40 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:40.900+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:41.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:40 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:40.900+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:41.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:40 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:40.900+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:41.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:40 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:40.900+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:10.090511+0000 front 2024-03-26T08:41:10.090486+0000 (oldest deadline 2024-03-26T08:41:35.389824+0000) 2024-03-26T08:41:41.400 INFO:journalctl@ceph.osd.6.smithi043.stdout:Mar 26 08:41:41 smithi043 podman[86601]: 2024-03-26 08:41:41.133411604 +0000 UTC m=+0.946464946 container died 120448b113df3a2f18e6cd5d7ea92b7cd80b84b823a3208f752d7ccdf62cae43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T08:41:41.712 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:41 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:41.505+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:41.712 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:41 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:41.505+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.154120+0000 front 2024-03-26T08:40:57.154394+0000 (oldest deadline 2024-03-26T08:41:23.053964+0000) 2024-03-26T08:41:41.712 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:41 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:41.505+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:06.358054+0000 front 2024-03-26T08:41:06.357924+0000 (oldest deadline 2024-03-26T08:41:31.657483+0000) 2024-03-26T08:41:41.712 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:41 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:41.505+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:17.961808+0000 front 2024-03-26T08:41:17.961893+0000 (oldest deadline 2024-03-26T08:41:40.261986+0000) 2024-03-26T08:41:41.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:41 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:41.600+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:41.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:41 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:41.600+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:41.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:41 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:41.600+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:41.860 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:41 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:41.600+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:41:41.904 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.6.service' 2024-03-26T08:41:42.038 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:41:42.038 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-26T08:41:42.038 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-26T08:41:42.038 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.7 2024-03-26T08:41:42.070 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:41 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:41.719+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:42.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:41 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:41.719+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:42.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:41 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:41.719+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:42.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:41 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:41.719+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:41:42.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:41 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:41.719+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:41:42.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:42 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:42.091+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:42.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:42 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:42.091+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:42.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:42 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:42.091+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:42.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:41 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:41.864+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:42.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:41 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:41.864+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:42.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:41 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:41.864+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:42.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:41 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:41.864+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:10.090511+0000 front 2024-03-26T08:41:10.090486+0000 (oldest deadline 2024-03-26T08:41:35.389824+0000) 2024-03-26T08:41:42.381 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:42 smithi052 systemd[1]: Stopping Ceph osd.7 for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:41:42.734 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:42 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:42.507+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:42.735 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:42 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:42.507+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.154120+0000 front 2024-03-26T08:40:57.154394+0000 (oldest deadline 2024-03-26T08:41:23.053964+0000) 2024-03-26T08:41:42.735 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:42 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:42.507+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:06.358054+0000 front 2024-03-26T08:41:06.357924+0000 (oldest deadline 2024-03-26T08:41:31.657483+0000) 2024-03-26T08:41:42.735 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:42 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:42.507+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:17.961808+0000 front 2024-03-26T08:41:17.961893+0000 (oldest deadline 2024-03-26T08:41:40.261986+0000) 2024-03-26T08:41:42.900 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:42 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:42.611+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:42.900 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:42 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:42.611+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:42.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:42 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:42.611+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:42.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:42 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:42.611+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:41:43.047 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:42 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:42.841+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:43.048 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:42 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:42.841+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:43.048 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:42 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:42.841+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:43.048 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:42 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:42.841+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:10.090511+0000 front 2024-03-26T08:41:10.090486+0000 (oldest deadline 2024-03-26T08:41:35.389824+0000) 2024-03-26T08:41:43.071 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:42 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:42.742+0000 7fdcd9c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:41:43.071 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:42 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:42.742+0000 7fdcd9c00700 -1 osd.7 98 *** Got signal Terminated *** 2024-03-26T08:41:43.071 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:42 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:42.742+0000 7fdcd9c00700 -1 osd.7 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T08:41:43.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:42 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:42.748+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:43.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:42 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:42.748+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:43.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:42 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:42.748+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:43.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:42 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:42.748+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:41:43.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:42 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:42.748+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:41:43.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:43 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:43.076+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:43.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:43 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:43.076+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:43.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:43 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:43.076+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:43.753 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:43 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:43.619+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:43.754 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:43 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:43.619+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:43.754 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:43 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:43.619+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:43.754 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:43 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:43.619+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:41:43.754 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:43 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:43.619+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.702863+0000 front 2024-03-26T08:41:18.702460+0000 (oldest deadline 2024-03-26T08:41:42.801933+0000) 2024-03-26T08:41:43.785 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:43 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:43.473+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:43.785 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:43 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:43.473+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.154120+0000 front 2024-03-26T08:40:57.154394+0000 (oldest deadline 2024-03-26T08:41:23.053964+0000) 2024-03-26T08:41:43.786 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:43 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:43.473+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:06.358054+0000 front 2024-03-26T08:41:06.357924+0000 (oldest deadline 2024-03-26T08:41:31.657483+0000) 2024-03-26T08:41:43.786 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:43 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:43.473+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:17.961808+0000 front 2024-03-26T08:41:17.961893+0000 (oldest deadline 2024-03-26T08:41:40.261986+0000) 2024-03-26T08:41:44.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:43 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:43.793+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:44.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:43 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:43.793+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:44.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:43 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:43.793+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:44.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:43 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:43.793+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:41:44.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:43 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:43.793+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:41:44.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:44 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:44.037+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:44.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:44 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:44.037+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:44.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:44 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:44.037+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:44.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:43 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:43.884+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:44.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:43 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:43.884+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:44.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:43 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:43.884+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:44.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:43 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:43.884+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:10.090511+0000 front 2024-03-26T08:41:10.090486+0000 (oldest deadline 2024-03-26T08:41:35.389824+0000) 2024-03-26T08:41:44.770 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:44 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:44.480+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:44.770 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:44 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:44.480+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.154120+0000 front 2024-03-26T08:40:57.154394+0000 (oldest deadline 2024-03-26T08:41:23.053964+0000) 2024-03-26T08:41:44.770 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:44 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:44.480+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:06.358054+0000 front 2024-03-26T08:41:06.357924+0000 (oldest deadline 2024-03-26T08:41:31.657483+0000) 2024-03-26T08:41:44.770 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:44 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:44.480+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:17.961808+0000 front 2024-03-26T08:41:17.961893+0000 (oldest deadline 2024-03-26T08:41:40.261986+0000) 2024-03-26T08:41:45.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:44 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:44.777+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:45.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:44 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:44.777+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:45.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:44 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:44.777+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:45.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:44 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:44.777+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:41:45.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:44 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:44.777+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:41:45.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:44 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:44.661+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:45.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:44 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:44.661+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:45.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:44 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:44.661+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:45.152 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:44 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:44.661+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:41:45.152 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:44 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:44.661+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.702863+0000 front 2024-03-26T08:41:18.702460+0000 (oldest deadline 2024-03-26T08:41:42.801933+0000) 2024-03-26T08:41:45.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:45 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:45.053+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:45.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:45 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:45.053+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:45.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:45 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:45.053+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:45.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:44 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:44.859+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:45.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:44 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:44.859+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:45.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:44 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:44.859+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:45.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:44 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:44.859+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:10.090511+0000 front 2024-03-26T08:41:10.090486+0000 (oldest deadline 2024-03-26T08:41:35.389824+0000) 2024-03-26T08:41:45.795 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:45 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:45.515+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:45.795 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:45 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:45.515+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.154120+0000 front 2024-03-26T08:40:57.154394+0000 (oldest deadline 2024-03-26T08:41:23.053964+0000) 2024-03-26T08:41:45.795 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:45 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:45.515+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:06.358054+0000 front 2024-03-26T08:41:06.357924+0000 (oldest deadline 2024-03-26T08:41:31.657483+0000) 2024-03-26T08:41:45.795 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:45 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:45.515+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:17.961808+0000 front 2024-03-26T08:41:17.961893+0000 (oldest deadline 2024-03-26T08:41:40.261986+0000) 2024-03-26T08:41:46.073 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:45 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:45.802+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:46.074 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:45 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:45.802+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:46.074 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:45 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:45.802+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:46.074 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:45 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:45.802+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:41:46.074 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:45 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:45.802+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:41:46.095 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:45 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:45.815+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:46.095 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:45 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:45.815+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:46.095 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:45 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:45.815+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:46.096 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:45 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:45.815+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:10.090511+0000 front 2024-03-26T08:41:10.090486+0000 (oldest deadline 2024-03-26T08:41:35.389824+0000) 2024-03-26T08:41:46.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:45 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:45.695+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:46.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:45 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:45.695+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:46.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:45 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:45.695+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:46.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:45 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:45.695+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:41:46.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:45 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:45.695+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.702863+0000 front 2024-03-26T08:41:18.702460+0000 (oldest deadline 2024-03-26T08:41:42.801933+0000) 2024-03-26T08:41:46.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:46 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:46.103+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:46.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:46 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:46.103+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:46.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:46 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:46.103+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:46.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:46 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:46.103+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:41:46.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:46 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:46.556+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:46.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:46 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:46.556+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.154120+0000 front 2024-03-26T08:40:57.154394+0000 (oldest deadline 2024-03-26T08:41:23.053964+0000) 2024-03-26T08:41:46.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:46 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:46.556+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:06.358054+0000 front 2024-03-26T08:41:06.357924+0000 (oldest deadline 2024-03-26T08:41:31.657483+0000) 2024-03-26T08:41:46.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:46 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:46.556+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:17.961808+0000 front 2024-03-26T08:41:17.961893+0000 (oldest deadline 2024-03-26T08:41:40.261986+0000) 2024-03-26T08:41:46.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:46 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:46.761+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:46.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:46 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:46.761+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:46.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:46 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:46.761+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:46.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:46 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:46.761+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:41:46.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:46 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:46.761+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:41:47.145 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:46 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:46.855+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:47.145 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:46 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:46.855+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:47.145 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:46 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:46.855+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:47.146 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:46 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:46.855+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:10.090511+0000 front 2024-03-26T08:41:10.090486+0000 (oldest deadline 2024-03-26T08:41:35.389824+0000) 2024-03-26T08:41:47.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:46 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:46.666+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:47.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:46 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:46.666+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:47.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:46 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:46.666+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:47.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:46 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:46.666+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:41:47.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:46 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:46.666+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.702863+0000 front 2024-03-26T08:41:18.702460+0000 (oldest deadline 2024-03-26T08:41:42.801933+0000) 2024-03-26T08:41:47.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:47 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:47.153+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:47.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:47 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:47.153+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:47.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:47 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:47.153+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:47.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:47 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:47.153+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:41:47.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:47 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:47.568+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.752321+0000 front 2024-03-26T08:40:50.752422+0000 (oldest deadline 2024-03-26T08:41:13.651743+0000) 2024-03-26T08:41:47.821 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:47 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:47.569+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.154120+0000 front 2024-03-26T08:40:57.154394+0000 (oldest deadline 2024-03-26T08:41:23.053964+0000) 2024-03-26T08:41:47.822 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:47 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:47.569+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:06.358054+0000 front 2024-03-26T08:41:06.357924+0000 (oldest deadline 2024-03-26T08:41:31.657483+0000) 2024-03-26T08:41:47.822 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:47 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7[53297]: 2024-03-26T08:41:47.569+0000 7fdcd0400700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:17.961808+0000 front 2024-03-26T08:41:17.961893+0000 (oldest deadline 2024-03-26T08:41:40.261986+0000) 2024-03-26T08:41:47.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:47 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:47.755+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:47.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:47 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:47.755+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:47.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:47 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:47.755+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:47.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:47 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:47.755+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:41:47.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:47 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:47.755+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:41:48.127 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:47 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:47.840+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:48.128 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:47 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:47.840+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:48.128 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:47 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:47.840+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:48.128 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:47 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:47.840+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:10.090511+0000 front 2024-03-26T08:41:10.090486+0000 (oldest deadline 2024-03-26T08:41:35.389824+0000) 2024-03-26T08:41:48.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:47 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:47.666+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:48.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:47 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:47.666+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:48.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:47 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:47.666+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:48.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:47 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:47.666+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:41:48.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:47 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:47.666+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.702863+0000 front 2024-03-26T08:41:18.702460+0000 (oldest deadline 2024-03-26T08:41:42.801933+0000) 2024-03-26T08:41:48.500 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:48 smithi052 podman[69088]: 2024-03-26 08:41:48.198187402 +0000 UTC m=+5.734569883 container died fb994d47de12d6957205089762f2656ef5bdd0264927e85e747d2e8b4a9487a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T08:41:48.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:48 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:48.135+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:48.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:48 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:48.135+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:48.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:48 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:48.135+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:48.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:48 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:48.135+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:41:48.784 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:48 smithi052 podman[69088]: 2024-03-26 08:41:48.626823181 +0000 UTC m=+6.163205655 container cleanup fb994d47de12d6957205089762f2656ef5bdd0264927e85e747d2e8b4a9487a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-26T08:41:48.785 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:48 smithi052 bash[69088]: ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7 2024-03-26T08:41:49.071 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:49 smithi052 podman[69106]: 2024-03-26 08:41:49.0279748 +0000 UTC m=+0.819851361 container remove fb994d47de12d6957205089762f2656ef5bdd0264927e85e747d2e8b4a9487a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-26T08:41:49.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:48 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:48.792+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:49.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:48 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:48.792+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:49.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:48 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:48.792+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:49.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:48 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:48.792+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:41:49.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:48 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:48.792+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:41:49.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:48 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:48.705+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:49.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:48 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:48.705+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:49.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:48 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:48.705+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:49.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:48 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:48.705+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:41:49.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:48 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:48.705+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.702863+0000 front 2024-03-26T08:41:18.702460+0000 (oldest deadline 2024-03-26T08:41:42.801933+0000) 2024-03-26T08:41:49.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:49 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:49.101+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:49.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:49 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:49.101+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:49.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:49 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:49.101+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:49.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:49 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:49.101+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:41:49.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:48 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:48.870+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:49.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:48 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:48.870+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:49.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:48 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:48.870+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:49.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:48 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:48.870+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:10.090511+0000 front 2024-03-26T08:41:10.090486+0000 (oldest deadline 2024-03-26T08:41:35.389824+0000) 2024-03-26T08:41:50.071 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:49 smithi052 podman[69190]: 2024-03-26 08:41:49.856532159 +0000 UTC m=+0.040736723 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:41:50.071 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:49 smithi052 podman[69190]: 2024-03-26 08:41:49.987364933 +0000 UTC m=+0.171569510 container create b21def99f05c241e31db4c29c077841355ef910d8a956c7a87721a3879a77746 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319) 2024-03-26T08:41:50.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:49 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:49.754+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:50.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:49 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:49.754+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:50.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:49 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:49.754+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:50.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:49 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:49.754+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:41:50.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:49 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:49.754+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:41:50.106 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:49 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:49.855+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:50.106 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:49 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:49.855+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:50.106 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:49 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:49.855+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:50.106 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:49 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:49.855+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:10.090511+0000 front 2024-03-26T08:41:10.090486+0000 (oldest deadline 2024-03-26T08:41:35.389824+0000) 2024-03-26T08:41:50.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:49 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:49.722+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:50.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:49 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:49.722+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:50.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:49 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:49.722+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:50.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:49 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:49.722+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:41:50.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:49 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:49.722+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.702863+0000 front 2024-03-26T08:41:18.702460+0000 (oldest deadline 2024-03-26T08:41:42.801933+0000) 2024-03-26T08:41:50.386 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:50 smithi052 podman[69190]: 2024-03-26 08:41:50.382606187 +0000 UTC m=+0.566810807 container init b21def99f05c241e31db4c29c077841355ef910d8a956c7a87721a3879a77746 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-26T08:41:50.386 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:50 smithi052 podman[69190]: 2024-03-26 08:41:50.394458233 +0000 UTC m=+0.578662790 container start b21def99f05c241e31db4c29c077841355ef910d8a956c7a87721a3879a77746 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2) 2024-03-26T08:41:50.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:50 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:50.114+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:50.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:50 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:50.114+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:50.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:50 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:50.114+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:50.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:50 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:50.114+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:41:50.714 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:50 smithi052 podman[69190]: 2024-03-26 08:41:50.506361426 +0000 UTC m=+0.690566046 container attach b21def99f05c241e31db4c29c077841355ef910d8a956c7a87721a3879a77746 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-26T08:41:51.071 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:50 smithi052 podman[69190]: 2024-03-26 08:41:50.844283954 +0000 UTC m=+1.028488568 container died b21def99f05c241e31db4c29c077841355ef910d8a956c7a87721a3879a77746 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7-deactivate, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T08:41:51.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:50 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:50.722+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:51.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:50 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:50.722+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:51.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:50 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:50.722+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:51.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:50 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:50.722+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:41:51.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:50 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:50.722+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:41:51.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:50 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:50.726+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:51.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:50 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:50.726+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:51.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:50 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:50.726+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:51.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:50 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:50.726+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:41:51.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:50 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:50.726+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.702863+0000 front 2024-03-26T08:41:18.702460+0000 (oldest deadline 2024-03-26T08:41:42.801933+0000) 2024-03-26T08:41:51.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:51 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:51.084+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:51.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:51 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:51.084+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:51.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:51 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:51.084+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:51.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:51 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:51.084+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:41:51.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:51 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:51.084+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:41:51.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:50 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:50.844+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:51.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:50 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:50.844+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:51.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:50 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:50.844+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:51.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:50 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:50.844+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:10.090511+0000 front 2024-03-26T08:41:10.090486+0000 (oldest deadline 2024-03-26T08:41:35.389824+0000) 2024-03-26T08:41:51.712 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.7.service' 2024-03-26T08:41:51.840 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:51 smithi052 podman[69215]: 2024-03-26 08:41:51.676877142 +0000 UTC m=+0.809108093 container remove b21def99f05c241e31db4c29c077841355ef910d8a956c7a87721a3879a77746 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-26T08:41:51.840 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:51 smithi052 systemd[1]: ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.7.service: Deactivated successfully. 2024-03-26T08:41:51.840 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:51 smithi052 systemd[1]: Stopped Ceph osd.7 for 88759420-eb49-11ee-95d0-87774f69a715. 2024-03-26T08:41:51.840 INFO:journalctl@ceph.osd.7.smithi052.stdout:Mar 26 08:41:51 smithi052 systemd[1]: ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.7.service: Consumed 1min 8.748s CPU time. 2024-03-26T08:41:51.841 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:51 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:51.736+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:51.841 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:51 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:51.736+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:51.841 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:51 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:51.736+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:51.841 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:51 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:51.736+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:41:51.841 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:51 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:51.736+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:41:51.841 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:51 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:51.736+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:28.194477+0000 front 2024-03-26T08:41:28.194279+0000 (oldest deadline 2024-03-26T08:41:51.094275+0000) 2024-03-26T08:41:51.880 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:41:51.880 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-26T08:41:51.880 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-26T08:41:51.880 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.8 2024-03-26T08:41:52.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:51 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:51.730+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:52.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:51 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:51.730+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:52.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:51 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:51.730+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:52.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:51 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:51.730+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:41:52.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:51 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:51.730+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.702863+0000 front 2024-03-26T08:41:18.702460+0000 (oldest deadline 2024-03-26T08:41:42.801933+0000) 2024-03-26T08:41:52.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:51 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:51.730+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:25.703813+0000 front 2024-03-26T08:41:25.703881+0000 (oldest deadline 2024-03-26T08:41:51.003449+0000) 2024-03-26T08:41:52.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:51 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:51.863+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:52.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:51 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:51.863+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:52.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:51 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:51.863+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:52.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:51 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:51.863+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:10.090511+0000 front 2024-03-26T08:41:10.090486+0000 (oldest deadline 2024-03-26T08:41:35.389824+0000) 2024-03-26T08:41:52.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:52 smithi183 systemd[1]: Stopping Ceph osd.8 for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:41:52.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:52 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:52.094+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:52.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:52 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:52.094+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:52.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:52 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:52.094+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:52.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:52 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:52.094+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:41:52.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:52 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:52.094+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:41:52.797 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:52 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:52.484+0000 7f5ef9400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:41:52.797 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:52 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:52.484+0000 7f5ef9400700 -1 osd.8 98 *** Got signal Terminated *** 2024-03-26T08:41:52.797 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:52 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:52.484+0000 7f5ef9400700 -1 osd.8 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T08:41:53.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:52 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:52.729+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:53.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:52 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:52.729+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:53.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:52 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:52.729+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:53.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:52 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:52.729+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:41:53.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:52 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:52.729+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:41:53.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:52 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:52.729+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:28.194477+0000 front 2024-03-26T08:41:28.194279+0000 (oldest deadline 2024-03-26T08:41:51.094275+0000) 2024-03-26T08:41:53.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:52 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:52.724+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:53.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:52 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:52.724+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:53.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:52 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:52.724+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:53.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:52 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:52.724+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:41:53.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:52 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:52.724+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.702863+0000 front 2024-03-26T08:41:18.702460+0000 (oldest deadline 2024-03-26T08:41:42.801933+0000) 2024-03-26T08:41:53.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:52 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:52.724+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:25.703813+0000 front 2024-03-26T08:41:25.703881+0000 (oldest deadline 2024-03-26T08:41:51.003449+0000) 2024-03-26T08:41:53.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:52 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:52.876+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:53.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:52 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:52.876+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:53.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:52 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:52.876+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:53.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:52 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:52.876+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:10.090511+0000 front 2024-03-26T08:41:10.090486+0000 (oldest deadline 2024-03-26T08:41:35.389824+0000) 2024-03-26T08:41:53.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:53 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:53.097+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:53.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:53 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:53.098+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:53.299 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:53 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:53.098+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:53.299 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:53 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:53.098+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:41:53.299 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:53 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:53.098+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:41:54.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:53 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:53.713+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:54.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:53 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:53.713+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:54.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:53 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:53.713+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:54.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:53 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:53.713+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:41:54.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:53 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:53.713+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:41:54.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:53 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:53.713+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:28.194477+0000 front 2024-03-26T08:41:28.194279+0000 (oldest deadline 2024-03-26T08:41:51.094275+0000) 2024-03-26T08:41:54.136 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:53 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:53.701+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:54.137 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:53 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:53.701+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:54.137 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:53 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:53.701+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:54.137 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:53 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:53.701+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:41:54.137 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:53 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:53.701+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.702863+0000 front 2024-03-26T08:41:18.702460+0000 (oldest deadline 2024-03-26T08:41:42.801933+0000) 2024-03-26T08:41:54.137 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:53 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:53.701+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:25.703813+0000 front 2024-03-26T08:41:25.703881+0000 (oldest deadline 2024-03-26T08:41:51.003449+0000) 2024-03-26T08:41:54.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:53 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:53.922+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:54.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:53 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:53.922+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:54.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:53 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:53.922+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:54.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:53 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:53.922+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:10.090511+0000 front 2024-03-26T08:41:10.090486+0000 (oldest deadline 2024-03-26T08:41:35.389824+0000) 2024-03-26T08:41:54.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:54 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:54.077+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:54.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:54 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:54.077+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:54.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:54 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:54.077+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:54.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:54 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:54.077+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:41:54.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:54 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:54.077+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:41:55.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:54 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:54.680+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:55.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:54 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:54.680+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:55.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:54 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:54.680+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:55.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:54 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:54.680+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:41:55.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:54 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:54.680+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:41:55.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:54 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:54.680+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:28.194477+0000 front 2024-03-26T08:41:28.194279+0000 (oldest deadline 2024-03-26T08:41:51.094275+0000) 2024-03-26T08:41:55.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:54 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:54.677+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:55.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:54 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:54.677+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:55.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:54 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:54.677+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:55.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:54 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:54.677+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:41:55.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:54 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:54.677+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.702863+0000 front 2024-03-26T08:41:18.702460+0000 (oldest deadline 2024-03-26T08:41:42.801933+0000) 2024-03-26T08:41:55.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:54 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:54.677+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:25.703813+0000 front 2024-03-26T08:41:25.703881+0000 (oldest deadline 2024-03-26T08:41:51.003449+0000) 2024-03-26T08:41:55.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:55 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:55.036+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:55.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:55 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:55.036+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:55.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:55 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:55.036+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:55.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:55 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:55.036+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:41:55.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:55 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:55.036+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:41:55.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:54 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:54.936+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:55.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:54 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:54.936+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:55.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:54 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:54.936+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:55.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:54 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:54.936+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:10.090511+0000 front 2024-03-26T08:41:10.090486+0000 (oldest deadline 2024-03-26T08:41:35.389824+0000) 2024-03-26T08:41:55.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:54 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:54.936+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:27.495314+0000 front 2024-03-26T08:41:27.495360+0000 (oldest deadline 2024-03-26T08:41:54.496139+0000) 2024-03-26T08:41:56.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:55 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:55.646+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:56.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:55 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:55.646+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:56.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:55 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:55.646+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:56.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:55 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:55.646+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:41:56.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:55 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:55.646+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:41:56.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:55 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:55.646+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:28.194477+0000 front 2024-03-26T08:41:28.194279+0000 (oldest deadline 2024-03-26T08:41:51.094275+0000) 2024-03-26T08:41:56.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:55 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:55.709+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:56.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:55 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:55.709+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:56.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:55 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:55.709+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:56.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:55 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:55.709+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:41:56.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:55 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:55.709+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.702863+0000 front 2024-03-26T08:41:18.702460+0000 (oldest deadline 2024-03-26T08:41:42.801933+0000) 2024-03-26T08:41:56.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:55 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:55.709+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:25.703813+0000 front 2024-03-26T08:41:25.703881+0000 (oldest deadline 2024-03-26T08:41:51.003449+0000) 2024-03-26T08:41:56.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:55 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:55.979+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:56.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:55 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:55.979+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:56.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:55 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:55.979+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:56.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:55 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:55.979+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:10.090511+0000 front 2024-03-26T08:41:10.090486+0000 (oldest deadline 2024-03-26T08:41:35.389824+0000) 2024-03-26T08:41:56.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:55 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:55.979+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:27.495314+0000 front 2024-03-26T08:41:27.495360+0000 (oldest deadline 2024-03-26T08:41:54.496139+0000) 2024-03-26T08:41:56.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:56 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:56.032+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:56.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:56 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:56.032+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:56.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:56 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:56.032+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:56.299 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:56 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:56.032+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:41:56.299 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:56 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:56.032+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:41:57.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:56 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:56.661+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:57.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:56 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:56.661+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:57.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:56 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:56.661+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:57.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:56 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:56.661+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:41:57.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:56 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:56.661+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:41:57.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:56 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:56.661+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:28.194477+0000 front 2024-03-26T08:41:28.194279+0000 (oldest deadline 2024-03-26T08:41:51.094275+0000) 2024-03-26T08:41:57.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:56 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:56.708+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:57.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:56 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:56.708+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:57.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:56 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:56.708+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:57.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:56 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:56.708+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:41:57.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:56 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:56.708+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.702863+0000 front 2024-03-26T08:41:18.702460+0000 (oldest deadline 2024-03-26T08:41:42.801933+0000) 2024-03-26T08:41:57.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:56 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:56.708+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:25.703813+0000 front 2024-03-26T08:41:25.703881+0000 (oldest deadline 2024-03-26T08:41:51.003449+0000) 2024-03-26T08:41:57.297 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:57 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:57.004+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.383833+0000 front 2024-03-26T08:40:41.384320+0000 (oldest deadline 2024-03-26T08:41:04.284086+0000) 2024-03-26T08:41:57.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:57 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:57.004+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:49.586017+0000 front 2024-03-26T08:40:49.586430+0000 (oldest deadline 2024-03-26T08:41:15.485884+0000) 2024-03-26T08:41:57.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:57 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:57.004+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:57.187697+0000 front 2024-03-26T08:40:57.187928+0000 (oldest deadline 2024-03-26T08:41:21.887312+0000) 2024-03-26T08:41:57.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:57 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:57.004+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:10.090511+0000 front 2024-03-26T08:41:10.090486+0000 (oldest deadline 2024-03-26T08:41:35.389824+0000) 2024-03-26T08:41:57.298 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:57 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8[54608]: 2024-03-26T08:41:57.004+0000 7f5ef4c00700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:27.495314+0000 front 2024-03-26T08:41:27.495360+0000 (oldest deadline 2024-03-26T08:41:54.496139+0000) 2024-03-26T08:41:57.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:57 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:57.005+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:57.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:57 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:57.005+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:57.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:57 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:57.005+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:57.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:57 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:57.005+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:41:57.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:57 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:57.005+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:41:58.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:57 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:57.628+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:58.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:57 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:57.628+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:58.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:57 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:57.628+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:58.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:57 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:57.628+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:41:58.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:57 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:57.628+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:41:58.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:57 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:57.628+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:28.194477+0000 front 2024-03-26T08:41:28.194279+0000 (oldest deadline 2024-03-26T08:41:51.094275+0000) 2024-03-26T08:41:58.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:57 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:57.700+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:58.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:57 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:57.700+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:58.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:57 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:57.700+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:58.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:57 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:57.700+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:41:58.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:57 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:57.700+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.702863+0000 front 2024-03-26T08:41:18.702460+0000 (oldest deadline 2024-03-26T08:41:42.801933+0000) 2024-03-26T08:41:58.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:57 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:57.700+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:25.703813+0000 front 2024-03-26T08:41:25.703881+0000 (oldest deadline 2024-03-26T08:41:51.003449+0000) 2024-03-26T08:41:58.218 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:57 smithi183 podman[70207]: 2024-03-26 08:41:57.920378362 +0000 UTC m=+5.571862192 container died 1fc6782bae0f5ff93426b675d1e47618fc9140ecbb83bbdaf0d62548e3509120 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-26T08:41:58.219 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:58 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:58.012+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:58.219 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:58 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:58.012+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:58.219 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:58 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:58.012+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:58.219 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:58 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:58.012+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:41:58.219 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:58 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:58.012+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:41:58.547 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:58 smithi183 podman[70207]: 2024-03-26 08:41:58.344705537 +0000 UTC m=+5.996189360 container cleanup 1fc6782bae0f5ff93426b675d1e47618fc9140ecbb83bbdaf0d62548e3509120 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T08:41:58.547 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:58 smithi183 bash[70207]: ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8 2024-03-26T08:41:59.033 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:58 smithi183 podman[70225]: 2024-03-26 08:41:58.620787077 +0000 UTC m=+0.688763634 container remove 1fc6782bae0f5ff93426b675d1e47618fc9140ecbb83bbdaf0d62548e3509120 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-26T08:41:59.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:58 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:58.592+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:59.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:58 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:58.592+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:59.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:58 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:58.592+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:59.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:58 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:58.592+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:41:59.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:58 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:58.592+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:41:59.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:58 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:58.592+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:28.194477+0000 front 2024-03-26T08:41:28.194279+0000 (oldest deadline 2024-03-26T08:41:51.094275+0000) 2024-03-26T08:41:59.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:58 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:58.663+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:41:59.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:58 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:58.663+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:41:59.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:58 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:58.663+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:41:59.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:58 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:58.663+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:41:59.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:58 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:58.663+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.702863+0000 front 2024-03-26T08:41:18.702460+0000 (oldest deadline 2024-03-26T08:41:42.801933+0000) 2024-03-26T08:41:59.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:58 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:58.663+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:25.703813+0000 front 2024-03-26T08:41:25.703881+0000 (oldest deadline 2024-03-26T08:41:51.003449+0000) 2024-03-26T08:41:59.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:59 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:59.041+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:41:59.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:59 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:59.041+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:41:59.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:59 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:59.041+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:41:59.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:59 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:59.041+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:41:59.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:59 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:59.041+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:41:59.735 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:59 smithi183 podman[70310]: 2024-03-26 08:41:59.363482944 +0000 UTC m=+0.035889393 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:41:59.736 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:59 smithi183 podman[70310]: 2024-03-26 08:41:59.479958502 +0000 UTC m=+0.152364957 container create a07d2e2e07fbc9d5636e6f1e2a5fd3c9edacec720673ac413652a2b57fdacab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8-deactivate, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-26T08:41:59.736 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:59 smithi183 podman[70310]: 2024-03-26 08:41:59.743939797 +0000 UTC m=+0.416346258 container init a07d2e2e07fbc9d5636e6f1e2a5fd3c9edacec720673ac413652a2b57fdacab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-03-26T08:41:59.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:59 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:59.558+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:41:59.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:59 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:59.558+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:41:59.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:59 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:59.558+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:41:59.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:59 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:59.558+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:41:59.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:59 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:59.558+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:41:59.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:41:59 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:41:59.558+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:28.194477+0000 front 2024-03-26T08:41:28.194279+0000 (oldest deadline 2024-03-26T08:41:51.094275+0000) 2024-03-26T08:42:00.047 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:59 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:59.993+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:42:00.047 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:59 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:59.993+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:42:00.047 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:59 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:59.993+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:42:00.047 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:59 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:59.993+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:42:00.047 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:41:59 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:41:59.993+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:42:00.048 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:59 smithi183 podman[70310]: 2024-03-26 08:41:59.75483348 +0000 UTC m=+0.427239938 container start a07d2e2e07fbc9d5636e6f1e2a5fd3c9edacec720673ac413652a2b57fdacab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319) 2024-03-26T08:42:00.048 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:41:59 smithi183 podman[70310]: 2024-03-26 08:41:59.857209253 +0000 UTC m=+0.529615755 container attach a07d2e2e07fbc9d5636e6f1e2a5fd3c9edacec720673ac413652a2b57fdacab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8-deactivate, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux ) 2024-03-26T08:42:00.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:59 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:59.671+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:42:00.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:59 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:59.671+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:42:00.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:59 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:59.671+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:42:00.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:59 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:59.671+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:42:00.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:59 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:59.671+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.702863+0000 front 2024-03-26T08:41:18.702460+0000 (oldest deadline 2024-03-26T08:41:42.801933+0000) 2024-03-26T08:42:00.151 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:41:59 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:41:59.671+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:25.703813+0000 front 2024-03-26T08:41:25.703881+0000 (oldest deadline 2024-03-26T08:41:51.003449+0000) 2024-03-26T08:42:00.547 INFO:journalctl@ceph.osd.8.smithi183.stdout:Mar 26 08:42:00 smithi183 podman[70310]: 2024-03-26 08:42:00.183243742 +0000 UTC m=+0.855650220 container died a07d2e2e07fbc9d5636e6f1e2a5fd3c9edacec720673ac413652a2b57fdacab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-8-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-26T08:42:00.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:00 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:00.556+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:42:00.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:00 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:00.556+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:42:00.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:00 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:00.556+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:42:00.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:00 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:00.556+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:42:00.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:00 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:00.556+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:42:00.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:00 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:00.556+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:28.194477+0000 front 2024-03-26T08:41:28.194279+0000 (oldest deadline 2024-03-26T08:41:51.094275+0000) 2024-03-26T08:42:00.944 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.8.service' 2024-03-26T08:42:01.053 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:42:01.053 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-26T08:42:01.054 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-26T08:42:01.054 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.9 2024-03-26T08:42:01.147 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:00 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:00.662+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:42:01.147 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:00 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:00.662+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:42:01.148 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:00 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:00.662+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:42:01.148 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:00 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:00.662+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:42:01.148 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:00 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:00.662+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.702863+0000 front 2024-03-26T08:41:18.702460+0000 (oldest deadline 2024-03-26T08:41:42.801933+0000) 2024-03-26T08:42:01.148 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:00 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:00.662+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:25.703813+0000 front 2024-03-26T08:41:25.703881+0000 (oldest deadline 2024-03-26T08:41:51.003449+0000) 2024-03-26T08:42:01.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:00 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:00.974+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:42:01.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:00 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:00.974+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:42:01.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:00 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:00.974+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:42:01.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:00 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:00.974+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:42:01.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:00 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:00.974+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:42:01.400 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:01 smithi043 systemd[1]: Stopping Ceph osd.9 for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:42:01.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:01 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:01.506+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:42:01.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:01 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:01.506+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:42:01.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:01 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:01.506+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:42:01.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:01 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:01.506+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:42:01.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:01 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:01.506+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:42:01.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:01 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:01.506+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:28.194477+0000 front 2024-03-26T08:41:28.194279+0000 (oldest deadline 2024-03-26T08:41:51.094275+0000) 2024-03-26T08:42:01.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:01 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:01.617+0000 7feb0b800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:42:01.902 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:01 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:01.617+0000 7feb0b800700 -1 osd.9 98 *** Got signal Terminated *** 2024-03-26T08:42:01.902 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:01 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:01.617+0000 7feb0b800700 -1 osd.9 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T08:42:01.902 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:01 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:01.647+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:42:01.902 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:01 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:01.647+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:42:01.902 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:01 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:01.647+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:42:01.902 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:01 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:01.647+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:42:01.902 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:01 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:01.647+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.702863+0000 front 2024-03-26T08:41:18.702460+0000 (oldest deadline 2024-03-26T08:41:42.801933+0000) 2024-03-26T08:42:01.902 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:01 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:01.647+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:25.703813+0000 front 2024-03-26T08:41:25.703881+0000 (oldest deadline 2024-03-26T08:41:51.003449+0000) 2024-03-26T08:42:02.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:01 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:01.940+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:42:02.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:01 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:01.940+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:42:02.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:01 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:01.940+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:42:02.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:01 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:01.940+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:42:02.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:01 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:01.940+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:42:02.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:02 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:02.503+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:42:02.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:02 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:02.503+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:42:02.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:02 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:02.503+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:42:02.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:02 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:02.503+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:42:02.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:02 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:02.503+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:42:02.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:02 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:02.503+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:28.194477+0000 front 2024-03-26T08:41:28.194279+0000 (oldest deadline 2024-03-26T08:41:51.094275+0000) 2024-03-26T08:42:02.900 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:02 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:02.644+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:42:02.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:02 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:02.644+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:42:02.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:02 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:02.644+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:42:02.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:02 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:02.644+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:42:02.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:02 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:02.644+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.702863+0000 front 2024-03-26T08:41:18.702460+0000 (oldest deadline 2024-03-26T08:41:42.801933+0000) 2024-03-26T08:42:02.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:02 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:02.644+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:25.703813+0000 front 2024-03-26T08:41:25.703881+0000 (oldest deadline 2024-03-26T08:41:51.003449+0000) 2024-03-26T08:42:02.978 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:02 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:02.893+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:42:02.979 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:02 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:02.893+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:42:02.979 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:02 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:02.893+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:42:02.979 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:02 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:02.893+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:42:02.979 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:02 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:02.893+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:42:03.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:03 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:03.494+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:42:03.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:03 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:03.494+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:42:03.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:03 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:03.494+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:42:03.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:03 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:03.494+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:42:03.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:03 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:03.494+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:42:03.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:03 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:03.494+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:28.194477+0000 front 2024-03-26T08:41:28.194279+0000 (oldest deadline 2024-03-26T08:41:51.094275+0000) 2024-03-26T08:42:03.900 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:03 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:03.596+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:42:03.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:03 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:03.596+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:42:03.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:03 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:03.596+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:42:03.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:03 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:03.596+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:42:03.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:03 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:03.596+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.702863+0000 front 2024-03-26T08:41:18.702460+0000 (oldest deadline 2024-03-26T08:41:42.801933+0000) 2024-03-26T08:42:03.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:03 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:03.596+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:25.703813+0000 front 2024-03-26T08:41:25.703881+0000 (oldest deadline 2024-03-26T08:41:51.003449+0000) 2024-03-26T08:42:04.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:03 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:03.891+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:42:04.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:03 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:03.891+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:42:04.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:03 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:03.891+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:42:04.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:03 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:03.891+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:42:04.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:03 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:03.891+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:42:04.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:03 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:03.891+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.114824+0000 front 2024-03-26T08:41:36.114917+0000 (oldest deadline 2024-03-26T08:42:03.715631+0000) 2024-03-26T08:42:04.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:04 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:04.540+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:42:04.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:04 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:04.540+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:42:04.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:04 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:04.540+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:42:04.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:04 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:04.540+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:42:04.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:04 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:04.540+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:42:04.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:04 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:04.540+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:28.194477+0000 front 2024-03-26T08:41:28.194279+0000 (oldest deadline 2024-03-26T08:41:51.094275+0000) 2024-03-26T08:42:04.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:04 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:04.633+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:42:04.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:04 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:04.633+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:42:04.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:04 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:04.633+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:42:04.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:04 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:04.633+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:42:04.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:04 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:04.633+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.702863+0000 front 2024-03-26T08:41:18.702460+0000 (oldest deadline 2024-03-26T08:41:42.801933+0000) 2024-03-26T08:42:04.902 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:04 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:04.633+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:25.703813+0000 front 2024-03-26T08:41:25.703881+0000 (oldest deadline 2024-03-26T08:41:51.003449+0000) 2024-03-26T08:42:05.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:04 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:04.907+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:42:05.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:04 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:04.907+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:42:05.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:04 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:04.907+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:42:05.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:04 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:04.907+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:42:05.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:04 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:04.907+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:42:05.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:04 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:04.907+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.114824+0000 front 2024-03-26T08:41:36.114917+0000 (oldest deadline 2024-03-26T08:42:03.715631+0000) 2024-03-26T08:42:05.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:05 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:05.561+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:42:05.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:05 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:05.561+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:42:05.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:05 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:05.561+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:42:05.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:05 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:05.561+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:42:05.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:05 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:05.561+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:42:05.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:05 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:05.561+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:28.194477+0000 front 2024-03-26T08:41:28.194279+0000 (oldest deadline 2024-03-26T08:41:51.094275+0000) 2024-03-26T08:42:05.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:05 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:05.561+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.995873+0000 front 2024-03-26T08:41:36.996171+0000 (oldest deadline 2024-03-26T08:42:04.596497+0000) 2024-03-26T08:42:05.900 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:05 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:05.623+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:37.487599+0000 front 2024-03-26T08:40:37.487565+0000 (oldest deadline 2024-03-26T08:41:03.387757+0000) 2024-03-26T08:42:05.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:05 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:05.623+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:48.392683+0000 front 2024-03-26T08:40:48.392587+0000 (oldest deadline 2024-03-26T08:41:13.092764+0000) 2024-03-26T08:42:05.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:05 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:05.623+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.696025+0000 front 2024-03-26T08:41:00.696123+0000 (oldest deadline 2024-03-26T08:41:22.996284+0000) 2024-03-26T08:42:05.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:05 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:05.623+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:07.698287+0000 front 2024-03-26T08:41:07.698138+0000 (oldest deadline 2024-03-26T08:41:31.198413+0000) 2024-03-26T08:42:05.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:05 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:05.623+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.702863+0000 front 2024-03-26T08:41:18.702460+0000 (oldest deadline 2024-03-26T08:41:42.801933+0000) 2024-03-26T08:42:05.901 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:05 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9[63982]: 2024-03-26T08:42:05.623+0000 7feb07000700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:25.703813+0000 front 2024-03-26T08:41:25.703881+0000 (oldest deadline 2024-03-26T08:41:51.003449+0000) 2024-03-26T08:42:06.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:05 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:05.914+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:42:06.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:05 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:05.914+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:42:06.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:05 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:05.914+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:42:06.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:05 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:05.914+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:42:06.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:05 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:05.914+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:42:06.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:05 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:05.914+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.114824+0000 front 2024-03-26T08:41:36.114917+0000 (oldest deadline 2024-03-26T08:42:03.715631+0000) 2024-03-26T08:42:06.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:06 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:06.533+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:42:06.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:06 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:06.533+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:42:06.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:06 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:06.533+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:42:06.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:06 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:06.533+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:42:06.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:06 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:06.533+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:42:06.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:06 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:06.533+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:28.194477+0000 front 2024-03-26T08:41:28.194279+0000 (oldest deadline 2024-03-26T08:41:51.094275+0000) 2024-03-26T08:42:06.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:06 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:06.533+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.995873+0000 front 2024-03-26T08:41:36.996171+0000 (oldest deadline 2024-03-26T08:42:04.596497+0000) 2024-03-26T08:42:07.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:06 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:06.959+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:42:07.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:06 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:06.959+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:42:07.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:06 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:06.959+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:42:07.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:06 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:06.959+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:42:07.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:06 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:06.959+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:42:07.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:06 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:06.959+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.114824+0000 front 2024-03-26T08:41:36.114917+0000 (oldest deadline 2024-03-26T08:42:03.715631+0000) 2024-03-26T08:42:07.650 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:07 smithi043 podman[86718]: 2024-03-26 08:42:07.209017952 +0000 UTC m=+5.728266460 container died fdddf9ad3486e60de8e4d3cb29f808f2c909c92347ee73e8f235b6ca1e0c1b12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-26T08:42:07.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:07 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:07.575+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:42:07.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:07 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:07.575+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:42:07.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:07 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:07.575+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:42:07.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:07 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:07.575+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:42:07.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:07 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:07.575+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:42:07.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:07 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:07.575+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:28.194477+0000 front 2024-03-26T08:41:28.194279+0000 (oldest deadline 2024-03-26T08:41:51.094275+0000) 2024-03-26T08:42:07.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:07 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:07.575+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.995873+0000 front 2024-03-26T08:41:36.996171+0000 (oldest deadline 2024-03-26T08:42:04.596497+0000) 2024-03-26T08:42:08.094 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:07 smithi043 podman[86718]: 2024-03-26 08:42:07.827034523 +0000 UTC m=+6.346283062 container cleanup fdddf9ad3486e60de8e4d3cb29f808f2c909c92347ee73e8f235b6ca1e0c1b12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T08:42:08.094 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:07 smithi043 bash[86718]: ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9 2024-03-26T08:42:08.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:07 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:07.946+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:42:08.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:07 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:07.946+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:42:08.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:07 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:07.946+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:42:08.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:07 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:07.946+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:42:08.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:07 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:07.946+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:42:08.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:07 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:07.946+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.114824+0000 front 2024-03-26T08:41:36.114917+0000 (oldest deadline 2024-03-26T08:42:03.715631+0000) 2024-03-26T08:42:08.400 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:08 smithi043 podman[86736]: 2024-03-26 08:42:08.103095189 +0000 UTC m=+0.878067753 container remove fdddf9ad3486e60de8e4d3cb29f808f2c909c92347ee73e8f235b6ca1e0c1b12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T08:42:09.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:08 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:08.599+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:42:09.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:08 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:08.599+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:42:09.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:08 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:08.599+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:42:09.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:08 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:08.599+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:42:09.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:08 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:08.599+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:42:09.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:08 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:08.599+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:28.194477+0000 front 2024-03-26T08:41:28.194279+0000 (oldest deadline 2024-03-26T08:41:51.094275+0000) 2024-03-26T08:42:09.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:08 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:08.599+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.995873+0000 front 2024-03-26T08:41:36.996171+0000 (oldest deadline 2024-03-26T08:42:04.596497+0000) 2024-03-26T08:42:09.271 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:08 smithi043 podman[86825]: 2024-03-26 08:42:08.995505341 +0000 UTC m=+0.130313843 container create 1e361bd784392959a394c68ef22032e647ff3830354371832c449f177084bc84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9-deactivate, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-26T08:42:09.271 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:08 smithi043 podman[86825]: 2024-03-26 08:42:08.902256756 +0000 UTC m=+0.037065244 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:42:09.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:08 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:08.938+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:42:09.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:08 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:08.938+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:42:09.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:08 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:08.938+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:42:09.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:08 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:08.938+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:42:09.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:08 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:08.938+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:42:09.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:08 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:08.938+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.114824+0000 front 2024-03-26T08:41:36.114917+0000 (oldest deadline 2024-03-26T08:42:03.715631+0000) 2024-03-26T08:42:09.651 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:09 smithi043 podman[86825]: 2024-03-26 08:42:09.382867141 +0000 UTC m=+0.517675690 container init 1e361bd784392959a394c68ef22032e647ff3830354371832c449f177084bc84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-26T08:42:09.651 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:09 smithi043 podman[86825]: 2024-03-26 08:42:09.394146141 +0000 UTC m=+0.528954653 container start 1e361bd784392959a394c68ef22032e647ff3830354371832c449f177084bc84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, ceph=True) 2024-03-26T08:42:09.651 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:09 smithi043 podman[86825]: 2024-03-26 08:42:09.456382401 +0000 UTC m=+0.591190963 container attach 1e361bd784392959a394c68ef22032e647ff3830354371832c449f177084bc84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-26T08:42:09.974 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:09 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:09.599+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:42:09.974 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:09 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:09.599+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:42:09.974 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:09 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:09.599+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:42:09.974 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:09 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:09.599+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:42:09.974 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:09 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:09.599+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:42:09.974 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:09 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:09.599+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:28.194477+0000 front 2024-03-26T08:41:28.194279+0000 (oldest deadline 2024-03-26T08:41:51.094275+0000) 2024-03-26T08:42:09.974 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:09 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:09.599+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.995873+0000 front 2024-03-26T08:41:36.996171+0000 (oldest deadline 2024-03-26T08:42:04.596497+0000) 2024-03-26T08:42:10.150 INFO:journalctl@ceph.osd.9.smithi043.stdout:Mar 26 08:42:09 smithi043 podman[86851]: 2024-03-26 08:42:09.897578156 +0000 UTC m=+0.037155108 container died 1e361bd784392959a394c68ef22032e647ff3830354371832c449f177084bc84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T08:42:10.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:09 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:09.908+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:42:10.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:09 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:09.908+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:42:10.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:09 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:09.908+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:42:10.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:09 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:09.908+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:42:10.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:09 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:09.908+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:42:10.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:09 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:09.908+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.114824+0000 front 2024-03-26T08:41:36.114917+0000 (oldest deadline 2024-03-26T08:42:03.715631+0000) 2024-03-26T08:42:10.598 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.9.service' 2024-03-26T08:42:10.729 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:42:10.730 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-26T08:42:10.730 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-26T08:42:10.730 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.10 2024-03-26T08:42:10.862 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:10 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:10.605+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:42:10.863 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:10 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:10.605+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:42:10.863 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:10 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:10.605+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:42:10.863 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:10 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:10.605+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:42:10.863 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:10 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:10.605+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:42:10.863 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:10 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:10.605+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:28.194477+0000 front 2024-03-26T08:41:28.194279+0000 (oldest deadline 2024-03-26T08:41:51.094275+0000) 2024-03-26T08:42:10.863 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:10 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:10.605+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.995873+0000 front 2024-03-26T08:41:36.996171+0000 (oldest deadline 2024-03-26T08:42:04.596497+0000) 2024-03-26T08:42:11.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:10 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:10.907+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:42:11.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:10 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:10.907+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:42:11.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:10 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:10.907+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:42:11.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:10 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:10.907+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:42:11.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:10 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:10.907+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:42:11.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:10 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:10.907+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.114824+0000 front 2024-03-26T08:41:36.114917+0000 (oldest deadline 2024-03-26T08:42:03.715631+0000) 2024-03-26T08:42:11.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:10 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:10.907+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6820 osd.7 since back 2024-03-26T08:41:44.817557+0000 front 2024-03-26T08:41:44.817587+0000 (oldest deadline 2024-03-26T08:42:10.717186+0000) 2024-03-26T08:42:11.320 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:10 smithi052 systemd[1]: Stopping Ceph osd.10 for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:42:11.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:11 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:11.414+0000 7f9d90c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:42:11.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:11 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:11.414+0000 7f9d90c00700 -1 osd.10 98 *** Got signal Terminated *** 2024-03-26T08:42:11.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:11 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:11.414+0000 7f9d90c00700 -1 osd.10 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T08:42:11.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:11 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:11.569+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:42:11.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:11 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:11.569+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:42:11.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:11 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:11.569+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:42:11.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:11 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:11.569+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:42:11.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:11 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:11.569+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:42:11.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:11 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:11.569+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:28.194477+0000 front 2024-03-26T08:41:28.194279+0000 (oldest deadline 2024-03-26T08:41:51.094275+0000) 2024-03-26T08:42:11.822 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:11 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:11.569+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.995873+0000 front 2024-03-26T08:41:36.996171+0000 (oldest deadline 2024-03-26T08:42:04.596497+0000) 2024-03-26T08:42:12.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:11 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:11.958+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:42:12.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:11 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:11.958+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:42:12.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:11 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:11.958+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:42:12.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:11 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:11.958+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:42:12.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:11 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:11.958+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:42:12.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:11 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:11.958+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.114824+0000 front 2024-03-26T08:41:36.114917+0000 (oldest deadline 2024-03-26T08:42:03.715631+0000) 2024-03-26T08:42:12.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:11 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:11.958+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6820 osd.7 since back 2024-03-26T08:41:44.817557+0000 front 2024-03-26T08:41:44.817587+0000 (oldest deadline 2024-03-26T08:42:10.717186+0000) 2024-03-26T08:42:13.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:12 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:12.604+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:42:13.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:12 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:12.604+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:42:13.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:12 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:12.604+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:42:13.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:12 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:12.604+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:42:13.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:12 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:12.604+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:42:13.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:12 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:12.604+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:28.194477+0000 front 2024-03-26T08:41:28.194279+0000 (oldest deadline 2024-03-26T08:41:51.094275+0000) 2024-03-26T08:42:13.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:12 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:12.604+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.995873+0000 front 2024-03-26T08:41:36.996171+0000 (oldest deadline 2024-03-26T08:42:04.596497+0000) 2024-03-26T08:42:13.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:12 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:12.921+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:42:13.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:12 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:12.921+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:42:13.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:12 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:12.921+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:42:13.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:12 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:12.921+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:42:13.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:12 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:12.921+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:42:13.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:12 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:12.921+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.114824+0000 front 2024-03-26T08:41:36.114917+0000 (oldest deadline 2024-03-26T08:42:03.715631+0000) 2024-03-26T08:42:13.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:12 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:12.921+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6820 osd.7 since back 2024-03-26T08:41:44.817557+0000 front 2024-03-26T08:41:44.817587+0000 (oldest deadline 2024-03-26T08:42:10.717186+0000) 2024-03-26T08:42:14.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:13 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:13.591+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:42:14.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:13 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:13.591+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:42:14.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:13 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:13.591+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:42:14.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:13 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:13.591+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:42:14.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:13 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:13.591+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:42:14.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:13 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:13.591+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:28.194477+0000 front 2024-03-26T08:41:28.194279+0000 (oldest deadline 2024-03-26T08:41:51.094275+0000) 2024-03-26T08:42:14.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:13 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:13.591+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.995873+0000 front 2024-03-26T08:41:36.996171+0000 (oldest deadline 2024-03-26T08:42:04.596497+0000) 2024-03-26T08:42:14.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:13 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:13.897+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:42:14.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:13 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:13.897+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:42:14.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:13 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:13.897+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:42:14.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:13 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:13.897+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:42:14.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:13 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:13.897+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:42:14.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:13 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:13.897+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.114824+0000 front 2024-03-26T08:41:36.114917+0000 (oldest deadline 2024-03-26T08:42:03.715631+0000) 2024-03-26T08:42:14.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:13 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:13.897+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6820 osd.7 since back 2024-03-26T08:41:44.817557+0000 front 2024-03-26T08:41:44.817587+0000 (oldest deadline 2024-03-26T08:42:10.717186+0000) 2024-03-26T08:42:15.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:14 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:14.632+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:42:15.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:14 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:14.632+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:42:15.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:14 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:14.632+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:42:15.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:14 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:14.632+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:42:15.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:14 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:14.632+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:42:15.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:14 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:14.632+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:28.194477+0000 front 2024-03-26T08:41:28.194279+0000 (oldest deadline 2024-03-26T08:41:51.094275+0000) 2024-03-26T08:42:15.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:14 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:14.632+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.995873+0000 front 2024-03-26T08:41:36.996171+0000 (oldest deadline 2024-03-26T08:42:04.596497+0000) 2024-03-26T08:42:15.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:14 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:14.911+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:42:15.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:14 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:14.911+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:42:15.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:14 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:14.911+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:42:15.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:14 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:14.911+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:42:15.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:14 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:14.911+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:42:15.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:14 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:14.911+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.114824+0000 front 2024-03-26T08:41:36.114917+0000 (oldest deadline 2024-03-26T08:42:03.715631+0000) 2024-03-26T08:42:15.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:14 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:14.911+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6820 osd.7 since back 2024-03-26T08:41:44.817557+0000 front 2024-03-26T08:41:44.817587+0000 (oldest deadline 2024-03-26T08:42:10.717186+0000) 2024-03-26T08:42:16.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:15 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:15.625+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:40.380362+0000 front 2024-03-26T08:40:40.380432+0000 (oldest deadline 2024-03-26T08:41:05.080599+0000) 2024-03-26T08:42:16.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:15 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:15.625+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.983007+0000 front 2024-03-26T08:40:50.983153+0000 (oldest deadline 2024-03-26T08:41:15.682995+0000) 2024-03-26T08:42:16.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:15 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:15.625+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:40:56.784771+0000 front 2024-03-26T08:40:56.785028+0000 (oldest deadline 2024-03-26T08:41:21.484204+0000) 2024-03-26T08:42:16.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:15 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:15.625+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6814 osd.3 since back 2024-03-26T08:41:09.588272+0000 front 2024-03-26T08:41:09.588429+0000 (oldest deadline 2024-03-26T08:41:33.088501+0000) 2024-03-26T08:42:16.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:15 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:15.625+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:18.390412+0000 front 2024-03-26T08:41:18.390322+0000 (oldest deadline 2024-03-26T08:41:40.089952+0000) 2024-03-26T08:42:16.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:15 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:15.625+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:28.194477+0000 front 2024-03-26T08:41:28.194279+0000 (oldest deadline 2024-03-26T08:41:51.094275+0000) 2024-03-26T08:42:16.072 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:15 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10[60354]: 2024-03-26T08:42:15.625+0000 7f9d87400700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.995873+0000 front 2024-03-26T08:41:36.996171+0000 (oldest deadline 2024-03-26T08:42:04.596497+0000) 2024-03-26T08:42:16.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:15 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:15.934+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:42:16.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:15 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:15.934+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:42:16.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:15 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:15.934+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:42:16.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:15 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:15.934+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:42:16.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:15 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:15.934+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:42:16.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:15 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:15.934+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.114824+0000 front 2024-03-26T08:41:36.114917+0000 (oldest deadline 2024-03-26T08:42:03.715631+0000) 2024-03-26T08:42:16.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:15 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:15.934+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6820 osd.7 since back 2024-03-26T08:41:44.817557+0000 front 2024-03-26T08:41:44.817587+0000 (oldest deadline 2024-03-26T08:42:10.717186+0000) 2024-03-26T08:42:17.174 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:16 smithi052 podman[69307]: 2024-03-26 08:42:16.888199891 +0000 UTC m=+5.629536176 container died d9204bafa8f7172bba44f4f7278d5f9554852f41d46892c424819feb202604b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD) 2024-03-26T08:42:17.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:16 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:16.954+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:42:17.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:16 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:16.954+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:42:17.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:16 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:16.954+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:42:17.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:16 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:16.954+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:42:17.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:16 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:16.954+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:42:17.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:16 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:16.954+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.114824+0000 front 2024-03-26T08:41:36.114917+0000 (oldest deadline 2024-03-26T08:42:03.715631+0000) 2024-03-26T08:42:17.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:16 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:16.954+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6820 osd.7 since back 2024-03-26T08:41:44.817557+0000 front 2024-03-26T08:41:44.817587+0000 (oldest deadline 2024-03-26T08:42:10.717186+0000) 2024-03-26T08:42:17.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:17 smithi052 podman[69307]: 2024-03-26 08:42:17.442459834 +0000 UTC m=+6.183796176 container cleanup d9204bafa8f7172bba44f4f7278d5f9554852f41d46892c424819feb202604b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-26T08:42:17.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:17 smithi052 bash[69307]: ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10 2024-03-26T08:42:17.821 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:17 smithi052 podman[69327]: 2024-03-26 08:42:17.664125159 +0000 UTC m=+0.757744344 container remove d9204bafa8f7172bba44f4f7278d5f9554852f41d46892c424819feb202604b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T08:42:18.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:17 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:17.982+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:42:18.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:17 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:17.982+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:42:18.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:17 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:17.982+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:42:18.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:17 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:17.982+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:42:18.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:17 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:17.982+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:42:18.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:17 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:17.982+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.114824+0000 front 2024-03-26T08:41:36.114917+0000 (oldest deadline 2024-03-26T08:42:03.715631+0000) 2024-03-26T08:42:18.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:17 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:17.982+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6820 osd.7 since back 2024-03-26T08:41:44.817557+0000 front 2024-03-26T08:41:44.817587+0000 (oldest deadline 2024-03-26T08:42:10.717186+0000) 2024-03-26T08:42:18.792 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:18 smithi052 podman[69415]: 2024-03-26 08:42:18.426555274 +0000 UTC m=+0.036942302 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:42:18.793 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:18 smithi052 podman[69415]: 2024-03-26 08:42:18.664979657 +0000 UTC m=+0.275366707 container create ace53529502998974bb1b81b4a45ec2e3bea8f9d4834c497e9cd2c834e7c20b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD) 2024-03-26T08:42:19.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:18 smithi052 podman[69415]: 2024-03-26 08:42:18.893791063 +0000 UTC m=+0.504178120 container init ace53529502998974bb1b81b4a45ec2e3bea8f9d4834c497e9cd2c834e7c20b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T08:42:19.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:18 smithi052 podman[69415]: 2024-03-26 08:42:18.905094063 +0000 UTC m=+0.515481115 container start ace53529502998974bb1b81b4a45ec2e3bea8f9d4834c497e9cd2c834e7c20b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, ceph=True, org.label-schema.schema-version=1.0) 2024-03-26T08:42:19.071 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:18 smithi052 podman[69415]: 2024-03-26 08:42:18.99221685 +0000 UTC m=+0.602603927 container attach ace53529502998974bb1b81b4a45ec2e3bea8f9d4834c497e9cd2c834e7c20b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T08:42:19.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:18 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:18.938+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:42:19.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:18 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:18.938+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:42:19.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:18 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:18.938+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:42:19.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:18 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:18.938+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:42:19.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:18 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:18.938+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:42:19.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:18 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:18.938+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.114824+0000 front 2024-03-26T08:41:36.114917+0000 (oldest deadline 2024-03-26T08:42:03.715631+0000) 2024-03-26T08:42:19.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:18 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:18.938+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6820 osd.7 since back 2024-03-26T08:41:44.817557+0000 front 2024-03-26T08:41:44.817587+0000 (oldest deadline 2024-03-26T08:42:10.717186+0000) 2024-03-26T08:42:19.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:18 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:18.938+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6820 osd.8 since back 2024-03-26T08:41:56.618884+0000 front 2024-03-26T08:41:56.619000+0000 (oldest deadline 2024-03-26T08:42:18.318633+0000) 2024-03-26T08:42:19.666 INFO:journalctl@ceph.osd.10.smithi052.stdout:Mar 26 08:42:19 smithi052 podman[69441]: 2024-03-26 08:42:19.405800579 +0000 UTC m=+0.035290895 container died ace53529502998974bb1b81b4a45ec2e3bea8f9d4834c497e9cd2c834e7c20b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-26T08:42:20.068 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.10.service' 2024-03-26T08:42:20.191 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:42:20.192 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-26T08:42:20.192 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-26T08:42:20.192 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.11 2024-03-26T08:42:20.281 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:19 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:19.935+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:42:20.281 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:19 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:19.935+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:42:20.281 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:19 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:19.935+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:42:20.281 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:19 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:19.935+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:42:20.281 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:19 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:19.935+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:42:20.281 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:19 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:19.935+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.114824+0000 front 2024-03-26T08:41:36.114917+0000 (oldest deadline 2024-03-26T08:42:03.715631+0000) 2024-03-26T08:42:20.281 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:19 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:19.935+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6820 osd.7 since back 2024-03-26T08:41:44.817557+0000 front 2024-03-26T08:41:44.817587+0000 (oldest deadline 2024-03-26T08:42:10.717186+0000) 2024-03-26T08:42:20.281 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:19 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:19.935+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6820 osd.8 since back 2024-03-26T08:41:56.618884+0000 front 2024-03-26T08:41:56.619000+0000 (oldest deadline 2024-03-26T08:42:18.318633+0000) 2024-03-26T08:42:20.547 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:20 smithi183 systemd[1]: Stopping Ceph osd.11 for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:42:21.048 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:20 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:20.725+0000 7f7d35400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:42:21.048 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:20 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:20.725+0000 7f7d35400700 -1 osd.11 98 *** Got signal Terminated *** 2024-03-26T08:42:21.048 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:20 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:20.725+0000 7f7d35400700 -1 osd.11 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T08:42:21.048 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:20 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:20.934+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:42:21.048 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:20 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:20.934+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:42:21.048 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:20 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:20.934+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:42:21.049 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:20 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:20.934+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:42:21.049 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:20 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:20.934+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:42:21.049 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:20 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:20.934+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.114824+0000 front 2024-03-26T08:41:36.114917+0000 (oldest deadline 2024-03-26T08:42:03.715631+0000) 2024-03-26T08:42:21.049 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:20 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:20.934+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6820 osd.7 since back 2024-03-26T08:41:44.817557+0000 front 2024-03-26T08:41:44.817587+0000 (oldest deadline 2024-03-26T08:42:10.717186+0000) 2024-03-26T08:42:21.049 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:20 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:20.934+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6820 osd.8 since back 2024-03-26T08:41:56.618884+0000 front 2024-03-26T08:41:56.619000+0000 (oldest deadline 2024-03-26T08:42:18.318633+0000) 2024-03-26T08:42:22.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:21 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:21.927+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:42:22.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:21 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:21.927+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:42:22.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:21 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:21.927+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:42:22.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:21 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:21.927+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:42:22.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:21 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:21.927+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:42:22.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:21 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:21.927+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.114824+0000 front 2024-03-26T08:41:36.114917+0000 (oldest deadline 2024-03-26T08:42:03.715631+0000) 2024-03-26T08:42:22.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:21 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:21.927+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6820 osd.7 since back 2024-03-26T08:41:44.817557+0000 front 2024-03-26T08:41:44.817587+0000 (oldest deadline 2024-03-26T08:42:10.717186+0000) 2024-03-26T08:42:22.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:21 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:21.927+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6820 osd.8 since back 2024-03-26T08:41:56.618884+0000 front 2024-03-26T08:41:56.619000+0000 (oldest deadline 2024-03-26T08:42:18.318633+0000) 2024-03-26T08:42:23.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:22 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:22.939+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:42:23.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:22 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:22.939+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:42:23.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:22 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:22.939+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:42:23.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:22 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:22.939+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:42:23.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:22 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:22.939+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:42:23.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:22 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:22.939+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.114824+0000 front 2024-03-26T08:41:36.114917+0000 (oldest deadline 2024-03-26T08:42:03.715631+0000) 2024-03-26T08:42:23.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:22 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:22.939+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6820 osd.7 since back 2024-03-26T08:41:44.817557+0000 front 2024-03-26T08:41:44.817587+0000 (oldest deadline 2024-03-26T08:42:10.717186+0000) 2024-03-26T08:42:23.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:22 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:22.939+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6820 osd.8 since back 2024-03-26T08:41:56.618884+0000 front 2024-03-26T08:41:56.619000+0000 (oldest deadline 2024-03-26T08:42:18.318633+0000) 2024-03-26T08:42:24.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:23 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:23.977+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:42:24.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:23 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:23.977+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:42:24.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:23 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:23.977+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:42:24.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:23 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:23.977+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:42:24.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:23 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:23.977+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:42:24.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:23 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:23.977+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.114824+0000 front 2024-03-26T08:41:36.114917+0000 (oldest deadline 2024-03-26T08:42:03.715631+0000) 2024-03-26T08:42:24.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:23 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:23.977+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6820 osd.7 since back 2024-03-26T08:41:44.817557+0000 front 2024-03-26T08:41:44.817587+0000 (oldest deadline 2024-03-26T08:42:10.717186+0000) 2024-03-26T08:42:24.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:23 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:23.977+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6820 osd.8 since back 2024-03-26T08:41:56.618884+0000 front 2024-03-26T08:41:56.619000+0000 (oldest deadline 2024-03-26T08:42:18.318633+0000) 2024-03-26T08:42:25.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:24 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:24.949+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-03-26T08:40:41.496672+0000 front 2024-03-26T08:40:41.496795+0000 (oldest deadline 2024-03-26T08:41:04.996621+0000) 2024-03-26T08:42:25.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:24 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:24.949+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-03-26T08:40:50.602274+0000 front 2024-03-26T08:40:50.602774+0000 (oldest deadline 2024-03-26T08:41:13.502415+0000) 2024-03-26T08:42:25.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:24 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:24.949+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-03-26T08:41:00.505367+0000 front 2024-03-26T08:41:00.505418+0000 (oldest deadline 2024-03-26T08:41:25.205538+0000) 2024-03-26T08:42:25.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:24 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:24.949+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-03-26T08:41:15.709854+0000 front 2024-03-26T08:41:15.709564+0000 (oldest deadline 2024-03-26T08:41:45.710022+0000) 2024-03-26T08:42:25.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:24 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:24.949+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-03-26T08:41:26.811877+0000 front 2024-03-26T08:41:26.812077+0000 (oldest deadline 2024-03-26T08:41:50.312038+0000) 2024-03-26T08:42:25.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:24 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:24.949+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.43:6822 osd.6 since back 2024-03-26T08:41:36.114824+0000 front 2024-03-26T08:41:36.114917+0000 (oldest deadline 2024-03-26T08:42:03.715631+0000) 2024-03-26T08:42:25.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:24 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:24.949+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.52:6820 osd.7 since back 2024-03-26T08:41:44.817557+0000 front 2024-03-26T08:41:44.817587+0000 (oldest deadline 2024-03-26T08:42:10.717186+0000) 2024-03-26T08:42:25.298 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:24 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11[61473]: 2024-03-26T08:42:24.949+0000 7f7d30c00700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.183:6820 osd.8 since back 2024-03-26T08:41:56.618884+0000 front 2024-03-26T08:41:56.619000+0000 (oldest deadline 2024-03-26T08:42:18.318633+0000) 2024-03-26T08:42:26.481 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:26 smithi183 podman[70427]: 2024-03-26 08:42:26.206860781 +0000 UTC m=+5.594974223 container died 08d09c25149a0f1feb823bd2bef6f8ca79b466313753cf1993b35e4469504b86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-26T08:42:26.797 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:26 smithi183 podman[70427]: 2024-03-26 08:42:26.644053356 +0000 UTC m=+6.032166835 container cleanup 08d09c25149a0f1feb823bd2bef6f8ca79b466313753cf1993b35e4469504b86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T08:42:26.797 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:26 smithi183 bash[70427]: ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11 2024-03-26T08:42:27.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:26 smithi183 podman[70446]: 2024-03-26 08:42:26.853755556 +0000 UTC m=+0.628013509 container remove 08d09c25149a0f1feb823bd2bef6f8ca79b466313753cf1993b35e4469504b86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-26T08:42:27.967 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:27 smithi183 podman[70535]: 2024-03-26 08:42:27.601234793 +0000 UTC m=+0.031639552 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:42:27.967 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:27 smithi183 podman[70535]: 2024-03-26 08:42:27.729523097 +0000 UTC m=+0.159927913 container create cbcc1f97ed9a16f8874e2a2dcfa3ddec102a46ae697e10630e6d4fe8663d2215 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T08:42:28.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:27 smithi183 podman[70535]: 2024-03-26 08:42:27.97565595 +0000 UTC m=+0.406060773 container init cbcc1f97ed9a16f8874e2a2dcfa3ddec102a46ae697e10630e6d4fe8663d2215 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T08:42:28.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:27 smithi183 podman[70535]: 2024-03-26 08:42:27.988524941 +0000 UTC m=+0.418929744 container start cbcc1f97ed9a16f8874e2a2dcfa3ddec102a46ae697e10630e6d4fe8663d2215 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T08:42:28.297 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:28 smithi183 podman[70535]: 2024-03-26 08:42:28.073169652 +0000 UTC m=+0.503574492 container attach cbcc1f97ed9a16f8874e2a2dcfa3ddec102a46ae697e10630e6d4fe8663d2215 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T08:42:28.731 INFO:journalctl@ceph.osd.11.smithi183.stdout:Mar 26 08:42:28 smithi183 podman[70535]: 2024-03-26 08:42:28.428599669 +0000 UTC m=+0.859004472 container died cbcc1f97ed9a16f8874e2a2dcfa3ddec102a46ae697e10630e6d4fe8663d2215 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-osd-11-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-26T08:42:29.026 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@osd.11.service' 2024-03-26T08:42:29.147 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:42:29.147 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-26T08:42:29.147 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-26T08:42:29.147 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.a 2024-03-26T08:42:29.650 INFO:journalctl@ceph.mds.a.smithi043.stdout:Mar 26 08:42:29 smithi043 systemd[1]: Stopping Ceph mds.a for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:42:30.025 INFO:journalctl@ceph.mds.a.smithi043.stdout:Mar 26 08:42:29 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-a[71041]: 2024-03-26T08:42:29.766+0000 7f33aee00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:42:30.025 INFO:journalctl@ceph.mds.a.smithi043.stdout:Mar 26 08:42:29 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-a[71041]: 2024-03-26T08:42:29.766+0000 7f33aee00700 -1 mds.a *** got signal Terminated *** 2024-03-26T08:42:34.106 INFO:journalctl@ceph.mds.a.smithi043.stdout:Mar 26 08:42:33 smithi043 podman[86929]: 2024-03-26 08:42:33.785499742 +0000 UTC m=+4.199660299 container died e7393de82697c3f3f2012ebbc3a63525f5f9f893d97d2ebbcb78d7e7c23b5cb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T08:42:34.400 INFO:journalctl@ceph.mds.a.smithi043.stdout:Mar 26 08:42:34 smithi043 podman[86929]: 2024-03-26 08:42:34.210145883 +0000 UTC m=+4.624306430 container cleanup e7393de82697c3f3f2012ebbc3a63525f5f9f893d97d2ebbcb78d7e7c23b5cb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-a, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319) 2024-03-26T08:42:34.400 INFO:journalctl@ceph.mds.a.smithi043.stdout:Mar 26 08:42:34 smithi043 bash[86929]: ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-a 2024-03-26T08:42:34.900 INFO:journalctl@ceph.mds.a.smithi043.stdout:Mar 26 08:42:34 smithi043 podman[86953]: 2024-03-26 08:42:34.511462994 +0000 UTC m=+0.709872498 container remove e7393de82697c3f3f2012ebbc3a63525f5f9f893d97d2ebbcb78d7e7c23b5cb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-26T08:42:35.009 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.a.service' 2024-03-26T08:42:35.139 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:42:35.139 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-26T08:42:35.139 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-26T08:42:35.139 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.d 2024-03-26T08:42:35.650 INFO:journalctl@ceph.mds.d.smithi043.stdout:Mar 26 08:42:35 smithi043 systemd[1]: Stopping Ceph mds.d for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:42:36.150 INFO:journalctl@ceph.mds.d.smithi043.stdout:Mar 26 08:42:35 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-d[69955]: 2024-03-26T08:42:35.761+0000 7f4959400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:42:36.150 INFO:journalctl@ceph.mds.d.smithi043.stdout:Mar 26 08:42:35 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-d[69955]: 2024-03-26T08:42:35.761+0000 7f4959400700 -1 mds.d *** got signal Terminated *** 2024-03-26T08:42:40.092 INFO:journalctl@ceph.mds.d.smithi043.stdout:Mar 26 08:42:39 smithi043 podman[87060]: 2024-03-26 08:42:39.775664747 +0000 UTC m=+4.135901066 container stop 68d36da8a4dcf745f6f321ffa1eea6292a5bfc8112a467d1848baa77de2ff385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, ceph=True, org.label-schema.license=GPLv2) 2024-03-26T08:42:40.093 INFO:journalctl@ceph.mds.d.smithi043.stdout:Mar 26 08:42:39 smithi043 podman[87060]: 2024-03-26 08:42:39.804223535 +0000 UTC m=+4.164459804 container died 68d36da8a4dcf745f6f321ffa1eea6292a5bfc8112a467d1848baa77de2ff385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True) 2024-03-26T08:42:40.400 INFO:journalctl@ceph.mds.d.smithi043.stdout:Mar 26 08:42:40 smithi043 podman[87060]: 2024-03-26 08:42:40.230285373 +0000 UTC m=+4.590521678 container cleanup 68d36da8a4dcf745f6f321ffa1eea6292a5bfc8112a467d1848baa77de2ff385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-d, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-26T08:42:40.400 INFO:journalctl@ceph.mds.d.smithi043.stdout:Mar 26 08:42:40 smithi043 bash[87060]: ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-d 2024-03-26T08:42:40.750 INFO:journalctl@ceph.mds.d.smithi043.stdout:Mar 26 08:42:40 smithi043 podman[87074]: 2024-03-26 08:42:40.448150354 +0000 UTC m=+0.652359457 container remove 68d36da8a4dcf745f6f321ffa1eea6292a5bfc8112a467d1848baa77de2ff385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-26T08:42:40.946 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.d.service' 2024-03-26T08:42:41.031 INFO:journalctl@ceph.mds.d.smithi043.stdout:Mar 26 08:42:40 smithi043 systemd[1]: ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.d.service: Deactivated successfully. 2024-03-26T08:42:41.031 INFO:journalctl@ceph.mds.d.smithi043.stdout:Mar 26 08:42:40 smithi043 systemd[1]: Stopped Ceph mds.d for 88759420-eb49-11ee-95d0-87774f69a715. 2024-03-26T08:42:41.031 INFO:journalctl@ceph.mds.d.smithi043.stdout:Mar 26 08:42:40 smithi043 systemd[1]: ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.d.service: Consumed 2.385s CPU time. 2024-03-26T08:42:41.066 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:42:41.067 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-26T08:42:41.067 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-26T08:42:41.067 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.g 2024-03-26T08:42:41.400 INFO:journalctl@ceph.mds.g.smithi043.stdout:Mar 26 08:42:41 smithi043 systemd[1]: Stopping Ceph mds.g for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:42:42.151 INFO:journalctl@ceph.mds.g.smithi043.stdout:Mar 26 08:42:41 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-g[70668]: 2024-03-26T08:42:41.732+0000 7fe7b7600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:42:42.151 INFO:journalctl@ceph.mds.g.smithi043.stdout:Mar 26 08:42:41 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-g[70668]: 2024-03-26T08:42:41.732+0000 7fe7b7600700 -1 mds.g *** got signal Terminated *** 2024-03-26T08:42:46.099 INFO:journalctl@ceph.mds.g.smithi043.stdout:Mar 26 08:42:45 smithi043 podman[87179]: 2024-03-26 08:42:45.744591798 +0000 UTC m=+4.172450767 container stop 27c24afd4d0533bedfb7e0211faa40f980b92603459768cad4de403d51b5a6a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-g, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T08:42:46.099 INFO:journalctl@ceph.mds.g.smithi043.stdout:Mar 26 08:42:45 smithi043 podman[87179]: 2024-03-26 08:42:45.773234066 +0000 UTC m=+4.201093037 container died 27c24afd4d0533bedfb7e0211faa40f980b92603459768cad4de403d51b5a6a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-g, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319) 2024-03-26T08:42:46.400 INFO:journalctl@ceph.mds.g.smithi043.stdout:Mar 26 08:42:46 smithi043 podman[87179]: 2024-03-26 08:42:46.242312218 +0000 UTC m=+4.670171230 container cleanup 27c24afd4d0533bedfb7e0211faa40f980b92603459768cad4de403d51b5a6a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-g, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T08:42:46.400 INFO:journalctl@ceph.mds.g.smithi043.stdout:Mar 26 08:42:46 smithi043 bash[87179]: ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-g 2024-03-26T08:42:46.900 INFO:journalctl@ceph.mds.g.smithi043.stdout:Mar 26 08:42:46 smithi043 podman[87193]: 2024-03-26 08:42:46.468440504 +0000 UTC m=+0.697983192 container remove 27c24afd4d0533bedfb7e0211faa40f980b92603459768cad4de403d51b5a6a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-g, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319) 2024-03-26T08:42:47.058 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.g.service' 2024-03-26T08:42:47.186 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:42:47.186 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-26T08:42:47.187 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-26T08:42:47.187 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.j 2024-03-26T08:42:47.650 INFO:journalctl@ceph.mds.j.smithi043.stdout:Mar 26 08:42:47 smithi043 systemd[1]: Stopping Ceph mds.j for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:42:48.150 INFO:journalctl@ceph.mds.j.smithi043.stdout:Mar 26 08:42:47 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-j[69564]: 2024-03-26T08:42:47.751+0000 7f7d52800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:42:48.150 INFO:journalctl@ceph.mds.j.smithi043.stdout:Mar 26 08:42:47 smithi043 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-j[69564]: 2024-03-26T08:42:47.751+0000 7f7d52800700 -1 mds.j *** got signal Terminated *** 2024-03-26T08:42:48.987 INFO:journalctl@ceph.mds.j.smithi043.stdout:Mar 26 08:42:48 smithi043 podman[87299]: 2024-03-26 08:42:48.743253162 +0000 UTC m=+1.115203637 container died de9f0ba937827eb60dd950c659f1178a3515948f928b001a90c954863d24f031 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-j, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-26T08:42:49.321 INFO:journalctl@ceph.mds.j.smithi043.stdout:Mar 26 08:42:49 smithi043 podman[87299]: 2024-03-26 08:42:49.129049719 +0000 UTC m=+1.501000190 container cleanup de9f0ba937827eb60dd950c659f1178a3515948f928b001a90c954863d24f031 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-j, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-26T08:42:49.321 INFO:journalctl@ceph.mds.j.smithi043.stdout:Mar 26 08:42:49 smithi043 bash[87299]: ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-j 2024-03-26T08:42:49.650 INFO:journalctl@ceph.mds.j.smithi043.stdout:Mar 26 08:42:49 smithi043 podman[87313]: 2024-03-26 08:42:49.329817404 +0000 UTC m=+0.568632905 container remove de9f0ba937827eb60dd950c659f1178a3515948f928b001a90c954863d24f031 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-j, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-03-26T08:42:49.822 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.j.service' 2024-03-26T08:42:49.941 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:42:49.941 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-26T08:42:49.941 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-26T08:42:49.941 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.b 2024-03-26T08:42:50.321 INFO:journalctl@ceph.mds.b.smithi052.stdout:Mar 26 08:42:50 smithi052 systemd[1]: Stopping Ceph mds.b for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:42:50.821 INFO:journalctl@ceph.mds.b.smithi052.stdout:Mar 26 08:42:50 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-b[65603]: 2024-03-26T08:42:50.512+0000 7f009e200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:42:50.821 INFO:journalctl@ceph.mds.b.smithi052.stdout:Mar 26 08:42:50 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-b[65603]: 2024-03-26T08:42:50.512+0000 7f009e200700 -1 mds.b *** got signal Terminated *** 2024-03-26T08:42:54.821 INFO:journalctl@ceph.mds.b.smithi052.stdout:Mar 26 08:42:54 smithi052 podman[69519]: 2024-03-26 08:42:54.527391801 +0000 UTC m=+4.129251880 container died 233fc0c0ef08c114f3418eb2e84b2474cc0f97f106577ebb6e2995081eada7f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T08:42:55.306 INFO:journalctl@ceph.mds.b.smithi052.stdout:Mar 26 08:42:55 smithi052 podman[69519]: 2024-03-26 08:42:55.056442028 +0000 UTC m=+4.658302103 container cleanup 233fc0c0ef08c114f3418eb2e84b2474cc0f97f106577ebb6e2995081eada7f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-b, org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-26T08:42:55.306 INFO:journalctl@ceph.mds.b.smithi052.stdout:Mar 26 08:42:55 smithi052 bash[69519]: ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-b 2024-03-26T08:42:55.570 INFO:journalctl@ceph.mds.b.smithi052.stdout:Mar 26 08:42:55 smithi052 podman[69545]: 2024-03-26 08:42:55.315098147 +0000 UTC m=+0.767538491 container remove 233fc0c0ef08c114f3418eb2e84b2474cc0f97f106577ebb6e2995081eada7f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-b, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-26T08:42:55.790 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.b.service' 2024-03-26T08:42:55.918 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:42:55.918 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-26T08:42:55.918 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-26T08:42:55.918 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.e 2024-03-26T08:42:56.321 INFO:journalctl@ceph.mds.e.smithi052.stdout:Mar 26 08:42:56 smithi052 systemd[1]: Stopping Ceph mds.e for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:42:56.821 INFO:journalctl@ceph.mds.e.smithi052.stdout:Mar 26 08:42:56 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-e[64984]: 2024-03-26T08:42:56.558+0000 7f8f7f400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:42:56.821 INFO:journalctl@ceph.mds.e.smithi052.stdout:Mar 26 08:42:56 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-e[64984]: 2024-03-26T08:42:56.558+0000 7f8f7f400700 -1 mds.e *** got signal Terminated *** 2024-03-26T08:43:00.820 INFO:journalctl@ceph.mds.e.smithi052.stdout:Mar 26 08:43:00 smithi052 podman[69651]: 2024-03-26 08:43:00.571881375 +0000 UTC m=+4.168296763 container died 7f77c1bd618d1e95025056e983f252626862ee2251c8ecd2732175f0f0f9916d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-26T08:43:01.133 INFO:journalctl@ceph.mds.e.smithi052.stdout:Mar 26 08:43:01 smithi052 podman[69651]: 2024-03-26 08:43:01.134528725 +0000 UTC m=+4.730944090 container cleanup 7f77c1bd618d1e95025056e983f252626862ee2251c8ecd2732175f0f0f9916d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-e, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-26T08:43:01.133 INFO:journalctl@ceph.mds.e.smithi052.stdout:Mar 26 08:43:01 smithi052 bash[69651]: ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-e 2024-03-26T08:43:01.821 INFO:journalctl@ceph.mds.e.smithi052.stdout:Mar 26 08:43:01 smithi052 podman[69665]: 2024-03-26 08:43:01.369125107 +0000 UTC m=+0.774726986 container remove 7f77c1bd618d1e95025056e983f252626862ee2251c8ecd2732175f0f0f9916d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-e, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319) 2024-03-26T08:43:01.859 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.e.service' 2024-03-26T08:43:01.982 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:43:01.983 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-26T08:43:01.983 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-26T08:43:01.983 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.h 2024-03-26T08:43:02.571 INFO:journalctl@ceph.mds.h.smithi052.stdout:Mar 26 08:43:02 smithi052 systemd[1]: Stopping Ceph mds.h for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:43:03.071 INFO:journalctl@ceph.mds.h.smithi052.stdout:Mar 26 08:43:02 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-h[65286]: 2024-03-26T08:43:02.627+0000 7f5b76c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:43:03.071 INFO:journalctl@ceph.mds.h.smithi052.stdout:Mar 26 08:43:02 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-h[65286]: 2024-03-26T08:43:02.627+0000 7f5b76c00700 -1 mds.h *** got signal Terminated *** 2024-03-26T08:43:06.983 INFO:journalctl@ceph.mds.h.smithi052.stdout:Mar 26 08:43:06 smithi052 podman[69773]: 2024-03-26 08:43:06.609561204 +0000 UTC m=+4.116030013 container died c09851e5d51e031a915391579f43c55527bf9a1ad5b82d615ecd060b5c74e5e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-h, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-26T08:43:07.266 INFO:journalctl@ceph.mds.h.smithi052.stdout:Mar 26 08:43:07 smithi052 podman[69773]: 2024-03-26 08:43:07.096195435 +0000 UTC m=+4.602664258 container cleanup c09851e5d51e031a915391579f43c55527bf9a1ad5b82d615ecd060b5c74e5e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-h, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-26T08:43:07.266 INFO:journalctl@ceph.mds.h.smithi052.stdout:Mar 26 08:43:07 smithi052 bash[69773]: ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-h 2024-03-26T08:43:07.570 INFO:journalctl@ceph.mds.h.smithi052.stdout:Mar 26 08:43:07 smithi052 podman[69787]: 2024-03-26 08:43:07.27202904 +0000 UTC m=+0.646591195 container remove c09851e5d51e031a915391579f43c55527bf9a1ad5b82d615ecd060b5c74e5e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-h, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-26T08:43:07.740 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.h.service' 2024-03-26T08:43:07.862 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:43:07.863 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-26T08:43:07.863 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-26T08:43:07.863 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.k 2024-03-26T08:43:08.321 INFO:journalctl@ceph.mds.k.smithi052.stdout:Mar 26 08:43:07 smithi052 systemd[1]: Stopping Ceph mds.k for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:43:08.821 INFO:journalctl@ceph.mds.k.smithi052.stdout:Mar 26 08:43:08 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-k[64679]: 2024-03-26T08:43:08.421+0000 7efc32800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:43:08.821 INFO:journalctl@ceph.mds.k.smithi052.stdout:Mar 26 08:43:08 smithi052 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-k[64679]: 2024-03-26T08:43:08.421+0000 7efc32800700 -1 mds.k *** got signal Terminated *** 2024-03-26T08:43:12.711 INFO:journalctl@ceph.mds.k.smithi052.stdout:Mar 26 08:43:12 smithi052 podman[69893]: 2024-03-26 08:43:12.421886584 +0000 UTC m=+4.116506027 container died b0d46f3ecabb0a95b540570e0d6117794c731f581c6676379e7b7ecc6a9bc050 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-k, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T08:43:13.066 INFO:journalctl@ceph.mds.k.smithi052.stdout:Mar 26 08:43:12 smithi052 podman[69893]: 2024-03-26 08:43:12.832881233 +0000 UTC m=+4.527500674 container cleanup b0d46f3ecabb0a95b540570e0d6117794c731f581c6676379e7b7ecc6a9bc050 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T08:43:13.067 INFO:journalctl@ceph.mds.k.smithi052.stdout:Mar 26 08:43:12 smithi052 bash[69893]: ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-k 2024-03-26T08:43:13.320 INFO:journalctl@ceph.mds.k.smithi052.stdout:Mar 26 08:43:13 smithi052 podman[69907]: 2024-03-26 08:43:13.075491199 +0000 UTC m=+0.641141338 container remove b0d46f3ecabb0a95b540570e0d6117794c731f581c6676379e7b7ecc6a9bc050 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-k, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T08:43:13.560 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.k.service' 2024-03-26T08:43:13.676 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:43:13.677 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-26T08:43:13.677 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-26T08:43:13.677 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.c 2024-03-26T08:43:14.047 INFO:journalctl@ceph.mds.c.smithi183.stdout:Mar 26 08:43:13 smithi183 systemd[1]: Stopping Ceph mds.c for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:43:14.424 INFO:journalctl@ceph.mds.c.smithi183.stdout:Mar 26 08:43:14 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-c[66535]: 2024-03-26T08:43:14.262+0000 7f66b7200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:43:14.424 INFO:journalctl@ceph.mds.c.smithi183.stdout:Mar 26 08:43:14 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-c[66535]: 2024-03-26T08:43:14.262+0000 7f66b7200700 -1 mds.c *** got signal Terminated *** 2024-03-26T08:43:17.418 INFO:journalctl@ceph.mds.c.smithi183.stdout:Mar 26 08:43:17 smithi183 podman[70641]: 2024-03-26 08:43:17.106315103 +0000 UTC m=+2.962979472 container died acfb6a572373cc49196183f38311a5ae1fe3b239aa504e30574431be71ff2e82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T08:43:17.797 INFO:journalctl@ceph.mds.c.smithi183.stdout:Mar 26 08:43:17 smithi183 podman[70641]: 2024-03-26 08:43:17.522251475 +0000 UTC m=+3.378915864 container cleanup acfb6a572373cc49196183f38311a5ae1fe3b239aa504e30574431be71ff2e82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319) 2024-03-26T08:43:17.797 INFO:journalctl@ceph.mds.c.smithi183.stdout:Mar 26 08:43:17 smithi183 bash[70641]: ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-c 2024-03-26T08:43:17.798 INFO:journalctl@ceph.mds.c.smithi183.stdout:Mar 26 08:43:17 smithi183 podman[70667]: 2024-03-26 08:43:17.672954136 +0000 UTC m=+0.544881973 container remove acfb6a572373cc49196183f38311a5ae1fe3b239aa504e30574431be71ff2e82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-c, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-26T08:43:18.148 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.c.service' 2024-03-26T08:43:18.257 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:43:18.258 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-26T08:43:18.258 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-26T08:43:18.258 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.f 2024-03-26T08:43:18.797 INFO:journalctl@ceph.mds.f.smithi183.stdout:Mar 26 08:43:18 smithi183 systemd[1]: Stopping Ceph mds.f for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:43:19.297 INFO:journalctl@ceph.mds.f.smithi183.stdout:Mar 26 08:43:18 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-f[66232]: 2024-03-26T08:43:18.890+0000 7f526b200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:43:19.297 INFO:journalctl@ceph.mds.f.smithi183.stdout:Mar 26 08:43:18 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-f[66232]: 2024-03-26T08:43:18.890+0000 7f526b200700 -1 mds.f *** got signal Terminated *** 2024-03-26T08:43:21.500 INFO:journalctl@ceph.mds.f.smithi183.stdout:Mar 26 08:43:21 smithi183 podman[70772]: 2024-03-26 08:43:21.249420235 +0000 UTC m=+2.524141312 container died 321fef0effb14b8de5d66fe5421c5e4391c01cc7acd8e34f859d70c02b2334ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-f, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5) 2024-03-26T08:43:21.759 INFO:journalctl@ceph.mds.f.smithi183.stdout:Mar 26 08:43:21 smithi183 podman[70772]: 2024-03-26 08:43:21.591799869 +0000 UTC m=+2.866520939 container cleanup 321fef0effb14b8de5d66fe5421c5e4391c01cc7acd8e34f859d70c02b2334ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319) 2024-03-26T08:43:21.759 INFO:journalctl@ceph.mds.f.smithi183.stdout:Mar 26 08:43:21 smithi183 bash[70772]: ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-f 2024-03-26T08:43:22.046 INFO:journalctl@ceph.mds.f.smithi183.stdout:Mar 26 08:43:21 smithi183 podman[70786]: 2024-03-26 08:43:21.767626787 +0000 UTC m=+0.502891276 container remove 321fef0effb14b8de5d66fe5421c5e4391c01cc7acd8e34f859d70c02b2334ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-26T08:43:22.220 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.f.service' 2024-03-26T08:43:22.336 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:43:22.336 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-26T08:43:22.336 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-26T08:43:22.336 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.i 2024-03-26T08:43:22.547 INFO:journalctl@ceph.mds.i.smithi183.stdout:Mar 26 08:43:22 smithi183 systemd[1]: Stopping Ceph mds.i for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:43:23.297 INFO:journalctl@ceph.mds.i.smithi183.stdout:Mar 26 08:43:22 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-i[66838]: 2024-03-26T08:43:22.893+0000 7f84e2200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:43:23.297 INFO:journalctl@ceph.mds.i.smithi183.stdout:Mar 26 08:43:22 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-i[66838]: 2024-03-26T08:43:22.893+0000 7f84e2200700 -1 mds.i *** got signal Terminated *** 2024-03-26T08:43:25.949 INFO:journalctl@ceph.mds.i.smithi183.stdout:Mar 26 08:43:25 smithi183 podman[70893]: 2024-03-26 08:43:25.679171152 +0000 UTC m=+2.907789668 container died 756510d7e47eff92ba184919269eab3c7f05ad11143039e0224aa8106134b5ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-i, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-26T08:43:26.245 INFO:journalctl@ceph.mds.i.smithi183.stdout:Mar 26 08:43:26 smithi183 podman[70893]: 2024-03-26 08:43:26.070079174 +0000 UTC m=+3.298697683 container cleanup 756510d7e47eff92ba184919269eab3c7f05ad11143039e0224aa8106134b5ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-i, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-26T08:43:26.245 INFO:journalctl@ceph.mds.i.smithi183.stdout:Mar 26 08:43:26 smithi183 bash[70893]: ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-i 2024-03-26T08:43:26.547 INFO:journalctl@ceph.mds.i.smithi183.stdout:Mar 26 08:43:26 smithi183 podman[70907]: 2024-03-26 08:43:26.25434545 +0000 UTC m=+0.554095887 container remove 756510d7e47eff92ba184919269eab3c7f05ad11143039e0224aa8106134b5ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-i, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-26T08:43:26.703 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.i.service' 2024-03-26T08:43:26.818 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:43:26.819 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-26T08:43:26.819 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-26T08:43:26.819 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.l 2024-03-26T08:43:27.296 INFO:journalctl@ceph.mds.l.smithi183.stdout:Mar 26 08:43:26 smithi183 systemd[1]: Stopping Ceph mds.l for 88759420-eb49-11ee-95d0-87774f69a715... 2024-03-26T08:43:27.796 INFO:journalctl@ceph.mds.l.smithi183.stdout:Mar 26 08:43:27 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-l[65924]: 2024-03-26T08:43:27.436+0000 7f18dc200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:43:27.797 INFO:journalctl@ceph.mds.l.smithi183.stdout:Mar 26 08:43:27 smithi183 ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-l[65924]: 2024-03-26T08:43:27.436+0000 7f18dc200700 -1 mds.l *** got signal Terminated *** 2024-03-26T08:43:31.725 INFO:journalctl@ceph.mds.l.smithi183.stdout:Mar 26 08:43:31 smithi183 podman[71015]: 2024-03-26 08:43:31.459222422 +0000 UTC m=+4.145862026 container died 476ecfbdfdd3cf3bc17c0e85e0cf2d9c2ae93115d8b38302f2785d6a6608b57f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T08:43:32.047 INFO:journalctl@ceph.mds.l.smithi183.stdout:Mar 26 08:43:31 smithi183 podman[71015]: 2024-03-26 08:43:31.880922909 +0000 UTC m=+4.567562514 container cleanup 476ecfbdfdd3cf3bc17c0e85e0cf2d9c2ae93115d8b38302f2785d6a6608b57f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T08:43:32.047 INFO:journalctl@ceph.mds.l.smithi183.stdout:Mar 26 08:43:31 smithi183 bash[71015]: ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-l 2024-03-26T08:43:32.540 INFO:journalctl@ceph.mds.l.smithi183.stdout:Mar 26 08:43:32 smithi183 podman[71030]: 2024-03-26 08:43:32.056606767 +0000 UTC m=+0.580135744 container remove 476ecfbdfdd3cf3bc17c0e85e0cf2d9c2ae93115d8b38302f2785d6a6608b57f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-88759420-eb49-11ee-95d0-87774f69a715-mds-l, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T08:43:32.568 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-88759420-eb49-11ee-95d0-87774f69a715@mds.l.service' 2024-03-26T08:43:32.677 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:43:32.677 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-26T08:43:32.677 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 88759420-eb49-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-26T08:43:33.019 INFO:teuthology.orchestra.run.smithi043.stdout:Deleting cluster with fsid: 88759420-eb49-11ee-95d0-87774f69a715 2024-03-26T08:43:38.206 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 88759420-eb49-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-26T08:43:38.551 INFO:teuthology.orchestra.run.smithi052.stdout:Deleting cluster with fsid: 88759420-eb49-11ee-95d0-87774f69a715 2024-03-26T08:43:43.437 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 88759420-eb49-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-26T08:43:43.764 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: 88759420-eb49-11ee-95d0-87774f69a715 2024-03-26T08:43:48.625 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T08:43:48.737 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T08:43:48.846 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T08:43:48.915 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-26T08:43:48.959 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622859/remote/smithi043/crash 2024-03-26T08:43:48.960 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/crash -- . 2024-03-26T08:43:49.082 INFO:teuthology.orchestra.run.smithi043.stderr:tar: /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-26T08:43:49.082 INFO:teuthology.orchestra.run.smithi043.stderr:tar: Error is not recoverable: exiting now 2024-03-26T08:43:49.089 DEBUG:teuthology.misc:Transferring archived files from smithi052:/var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622859/remote/smithi052/crash 2024-03-26T08:43:49.090 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/crash -- . 2024-03-26T08:43:49.345 INFO:teuthology.orchestra.run.smithi052.stderr:tar: /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-26T08:43:49.346 INFO:teuthology.orchestra.run.smithi052.stderr:tar: Error is not recoverable: exiting now 2024-03-26T08:43:49.353 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622859/remote/smithi183/crash 2024-03-26T08:43:49.354 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/crash -- . 2024-03-26T08:43:49.461 INFO:teuthology.orchestra.run.smithi183.stderr:tar: /var/lib/ceph/88759420-eb49-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-26T08:43:49.461 INFO:teuthology.orchestra.run.smithi183.stderr:tar: Error is not recoverable: exiting now 2024-03-26T08:43:49.465 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-26T08:43:49.465 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-26T08:43:49.574 INFO:teuthology.orchestra.run.smithi043.stdout:2024-03-26T08:27:51.056308+0000 mon.a (mon.0) 631 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-03-26T08:43:49.575 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-26T08:43:49.576 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[SEC\]' /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-26T08:43:49.665 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[ERR\]' /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-26T08:43:49.754 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[WRN\]' /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-26T08:43:49.841 INFO:teuthology.orchestra.run.smithi043.stdout:2024-03-26T08:27:51.056308+0000 mon.a (mon.0) 631 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-03-26T08:43:49.842 INFO:tasks.cephadm:Compressing logs... 2024-03-26T08:43:49.842 DEBUG:teuthology.orchestra.run.smithi043:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-26T08:43:49.846 DEBUG:teuthology.orchestra.run.smithi052:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-26T08:43:49.853 DEBUG:teuthology.orchestra.run.smithi183:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-26T08:43:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:find: ‘/var/log/rbd-target-api’gzip: No such file or directory -5 2024-03-26T08:43:49.910 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/cephadm.log 2024-03-26T08:43:49.910 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.6.log 2024-03-26T08:43:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mon.a.log 2024-03-26T08:43:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-26T08:43:49.914 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.g.log 2024-03-26T08:43:49.915 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.j.log 2024-03-26T08:43:49.917 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.3.log 2024-03-26T08:43:49.918 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.j.log: 93.2%gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mgr.x.log 2024-03-26T08:43:49.919 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-26T08:43:49.919 INFO:teuthology.orchestra.run.smithi043.stderr: 90.0% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-26T08:43:49.920 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.0.log 2024-03-26T08:43:49.921 INFO:teuthology.orchestra.run.smithi043.stderr: 94.7% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.g.log.gz 2024-03-26T08:43:49.921 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-26T08:43:49.923 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.log 2024-03-26T08:43:49.924 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.0.log: /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.cephadm.log: 82.3% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-26T08:43:49.926 INFO:teuthology.orchestra.run.smithi043.stderr:gzip 94.6% -5 -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.j.log.gz --verbose 2024-03-26T08:43:49.926 INFO:teuthology.orchestra.run.smithi043.stderr: -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-26T08:43:49.926 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.9.log 2024-03-26T08:43:49.929 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-volume.log: 87.1% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-26T08:43:49.929 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.d.log 2024-03-26T08:43:49.931 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.9.log: /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.a.log 2024-03-26T08:43:49.935 INFO:teuthology.orchestra.run.smithi052.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-26T08:43:49.935 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-26T08:43:49.936 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.10.log 2024-03-26T08:43:49.936 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-03-26T08:43:49.937 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-26T08:43:49.938 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.b.log 2024-03-26T08:43:49.939 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.4.log 2024-03-26T08:43:49.941 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-03-26T08:43:49.941 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.audit.log: /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.1.log 2024-03-26T08:43:49.942 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.4.log: 91.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-26T08:43:49.943 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.1.log: 90.2% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-26T08:43:49.943 INFO:teuthology.orchestra.run.smithi183.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-26T08:43:49.944 INFO:teuthology.orchestra.run.smithi043.stderr: 94.5% 94.6% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.d.log.gz 2024-03-26T08:43:49.945 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.k.log 2024-03-26T08:43:49.946 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.a.log.gz 2024-03-26T08:43:49.946 INFO:teuthology.orchestra.run.smithi043.stderr: 90.8% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-26T08:43:49.946 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-26T08:43:49.947 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.h.log 2024-03-26T08:43:49.948 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.l.log 2024-03-26T08:43:49.948 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-26T08:43:49.949 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- 92.1% /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.i.log -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-26T08:43:49.949 INFO:teuthology.orchestra.run.smithi183.stderr: 2024-03-26T08:43:49.949 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.11.log 2024-03-26T08:43:49.951 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-03-26T08:43:49.951 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mgr.z.log 2024-03-26T08:43:49.952 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.k.log: /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.7.log 2024-03-26T08:43:49.952 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.e.log 2024-03-26T08:43:49.953 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.8.log 2024-03-26T08:43:49.953 INFO:teuthology.orchestra.run.smithi183.stderr: 90.2% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-26T08:43:49.953 INFO:teuthology.orchestra.run.smithi183.stderr: 94.8% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.l.log.gz 2024-03-26T08:43:49.954 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.5.log 2024-03-26T08:43:49.955 INFO:teuthology.orchestra.run.smithi052.stderr: 94.6% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.k.log.gz 2024-03-26T08:43:49.955 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.e.log: /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mgr.y.log 2024-03-26T08:43:49.956 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.8.log: 94.7% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.i.log.gz 2024-03-26T08:43:49.956 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.f.log 2024-03-26T08:43:49.956 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-26T08:43:49.957 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.log 2024-03-26T08:43:49.959 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.2.log 2024-03-26T08:43:49.960 INFO:teuthology.orchestra.run.smithi183.stderr: 81.8% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-26T08:43:49.960 INFO:teuthology.orchestra.run.smithi183.stderr: 92.7% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mgr.z.log.gz 2024-03-26T08:43:49.961 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.c.log 2024-03-26T08:43:49.962 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-26T08:43:49.962 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.2.log: 94.7% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.f.log.gz 2024-03-26T08:43:49.962 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-26T08:43:49.963 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.c.log: 87.1% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-26T08:43:49.964 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mon.c.log 2024-03-26T08:43:49.965 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.cephadm.log: 81.8% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-26T08:43:49.966 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.log 2024-03-26T08:43:49.967 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-volume.log: /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mon.c.log: 94.7% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.c.log.gz 2024-03-26T08:43:49.968 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-26T08:43:49.971 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mon.b.log 2024-03-26T08:43:49.973 INFO:teuthology.orchestra.run.smithi052.stderr: 94.7% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.e.log.gz 2024-03-26T08:43:49.975 INFO:teuthology.orchestra.run.smithi183.stderr: 90.8% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-26T08:43:49.978 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mon.b.log: 87.2%/var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-volume.log: -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-26T08:43:49.982 INFO:teuthology.orchestra.run.smithi052.stderr: 92.5% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mgr.y.log.gz 2024-03-26T08:43:50.005 INFO:teuthology.orchestra.run.smithi052.stderr: 90.8% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-26T08:43:50.039 INFO:teuthology.orchestra.run.smithi052.stderr: 93.1% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.h.log.gz 2024-03-26T08:43:50.184 INFO:teuthology.orchestra.run.smithi043.stderr: 89.8% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mgr.x.log.gz 2024-03-26T08:43:50.235 INFO:teuthology.orchestra.run.smithi052.stderr: 92.6% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mds.b.log.gz 2024-03-26T08:43:50.339 INFO:teuthology.orchestra.run.smithi183.stderr: 91.8% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mon.c.log.gz 2024-03-26T08:43:50.372 INFO:teuthology.orchestra.run.smithi052.stderr: 91.9% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mon.b.log.gz 2024-03-26T08:43:50.754 INFO:teuthology.orchestra.run.smithi043.stderr: 90.4% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-mon.a.log.gz 2024-03-26T08:43:53.553 INFO:teuthology.orchestra.run.smithi183.stderr: 92.7% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.11.log.gz 2024-03-26T08:43:53.731 INFO:teuthology.orchestra.run.smithi183.stderr: 92.7% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.8.log.gz 2024-03-26T08:43:53.766 INFO:teuthology.orchestra.run.smithi052.stderr: 92.9% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.10.log.gz 2024-03-26T08:43:54.541 INFO:teuthology.orchestra.run.smithi052.stderr: 92.7% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.1.log.gz 2024-03-26T08:43:54.790 INFO:teuthology.orchestra.run.smithi043.stderr: 92.7% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.0.log.gz 2024-03-26T08:43:55.038 INFO:teuthology.orchestra.run.smithi183.stderr: 92.8% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.2.log.gz 2024-03-26T08:43:55.803 INFO:teuthology.orchestra.run.smithi052.stderr: 92.9% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.7.log.gz 2024-03-26T08:43:56.394 INFO:teuthology.orchestra.run.smithi052.stderr: 92.8% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.4.log.gz 2024-03-26T08:43:56.399 INFO:teuthology.orchestra.run.smithi052.stderr: 2024-03-26T08:43:56.399 INFO:teuthology.orchestra.run.smithi052.stderr:real 0m6.501s 2024-03-26T08:43:56.399 INFO:teuthology.orchestra.run.smithi052.stderr:user 0m19.881s 2024-03-26T08:43:56.399 INFO:teuthology.orchestra.run.smithi052.stderr:sys 0m1.669s 2024-03-26T08:43:56.578 INFO:teuthology.orchestra.run.smithi043.stderr: 92.7% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.9.log.gz 2024-03-26T08:43:56.758 INFO:teuthology.orchestra.run.smithi183.stderr: 92.7% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.5.log.gz 2024-03-26T08:43:56.763 INFO:teuthology.orchestra.run.smithi183.stderr: 2024-03-26T08:43:56.763 INFO:teuthology.orchestra.run.smithi183.stderr:real 0m6.856s 2024-03-26T08:43:56.763 INFO:teuthology.orchestra.run.smithi183.stderr:user 0m18.234s 2024-03-26T08:43:56.763 INFO:teuthology.orchestra.run.smithi183.stderr:sys 0m1.514s 2024-03-26T08:43:57.287 INFO:teuthology.orchestra.run.smithi043.stderr: 92.9% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.3.log.gz 2024-03-26T08:43:57.400 INFO:teuthology.orchestra.run.smithi043.stderr: 92.8% -- replaced with /var/log/ceph/88759420-eb49-11ee-95d0-87774f69a715/ceph-osd.6.log.gz 2024-03-26T08:43:59.035 INFO:teuthology.orchestra.run.smithi043.stderr: 96.3% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-03-26T08:43:59.039 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-03-26T08:43:59.039 INFO:teuthology.orchestra.run.smithi043.stderr:real 0m9.162s 2024-03-26T08:43:59.039 INFO:teuthology.orchestra.run.smithi043.stderr:user 0m34.022s 2024-03-26T08:43:59.039 INFO:teuthology.orchestra.run.smithi043.stderr:sys 0m2.705s 2024-03-26T08:43:59.040 INFO:tasks.cephadm:Archiving logs... 2024-03-26T08:43:59.041 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622859/remote/smithi043/log 2024-03-26T08:43:59.042 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-26T08:44:03.627 DEBUG:teuthology.misc:Transferring archived files from smithi052:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622859/remote/smithi052/log 2024-03-26T08:44:03.628 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-26T08:44:06.815 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622859/remote/smithi183/log 2024-03-26T08:44:06.816 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-26T08:44:09.938 INFO:tasks.cephadm:Removing cluster... 2024-03-26T08:44:09.938 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 88759420-eb49-11ee-95d0-87774f69a715 --force 2024-03-26T08:44:10.279 INFO:teuthology.orchestra.run.smithi043.stdout:Deleting cluster with fsid: 88759420-eb49-11ee-95d0-87774f69a715 2024-03-26T08:44:10.989 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 88759420-eb49-11ee-95d0-87774f69a715 --force 2024-03-26T08:44:11.331 INFO:teuthology.orchestra.run.smithi052.stdout:Deleting cluster with fsid: 88759420-eb49-11ee-95d0-87774f69a715 2024-03-26T08:44:12.031 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 88759420-eb49-11ee-95d0-87774f69a715 --force 2024-03-26T08:44:12.347 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: 88759420-eb49-11ee-95d0-87774f69a715 2024-03-26T08:44:13.027 INFO:tasks.cephadm:Removing cephadm ... 2024-03-26T08:44:13.028 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-26T08:44:13.091 DEBUG:teuthology.orchestra.run.smithi052:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-26T08:44:13.145 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-26T08:44:13.204 INFO:tasks.cephadm:Teardown complete 2024-03-26T08:44:13.204 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-26T08:44:13.216 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-26T08:44:13.216 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-26T08:44:13.224 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-26T08:44:13.232 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-26T08:44:13.406 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-26T08:44:13.416 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-26T08:44:13.416 DEBUG:teuthology.orchestra.run.smithi043:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T08:44:13.424 DEBUG:teuthology.orchestra.run.smithi052:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T08:44:13.430 DEBUG:teuthology.orchestra.run.smithi183:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T08:44:13.475 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-03-26T08:44:13.475 INFO:teuthology.orchestra.run.smithi052.stderr:bash: line 1: ntpq: command not found 2024-03-26T08:44:13.484 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-03-26T08:44:13.484 INFO:teuthology.orchestra.run.smithi052.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T08:44:13.484 INFO:teuthology.orchestra.run.smithi052.stdout:=============================================================================== 2024-03-26T08:44:13.484 INFO:teuthology.orchestra.run.smithi052.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 27 -301us[ -302us] +/- 83ms 2024-03-26T08:44:13.484 INFO:teuthology.orchestra.run.smithi052.stdout:^* hv02.front.sepia.ceph.com 3 6 377 22 -1901us[-1902us] +/- 66ms 2024-03-26T08:44:13.484 INFO:teuthology.orchestra.run.smithi052.stdout:^+ hv03.front.sepia.ceph.com 3 8 377 22 +2212us[+2211us] +/- 68ms 2024-03-26T08:44:13.484 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T08:44:13.485 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T08:44:13.485 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-03-26T08:44:13.485 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 24 -163us[ -175us] +/- 83ms 2024-03-26T08:44:13.485 INFO:teuthology.orchestra.run.smithi043.stdout:^* hv02.front.sepia.ceph.com 3 7 377 21 -1780us[-1792us] +/- 66ms 2024-03-26T08:44:13.485 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 +2258us[+2247us] +/- 68ms 2024-03-26T08:44:13.485 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T08:44:13.494 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T08:44:13.494 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-03-26T08:44:13.494 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 20 -230us[ -195us] +/- 83ms 2024-03-26T08:44:13.494 INFO:teuthology.orchestra.run.smithi183.stdout:^* hv02.front.sepia.ceph.com 3 6 377 19 -1825us[-1790us] +/- 66ms 2024-03-26T08:44:13.494 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 +2271us[+2306us] +/- 68ms 2024-03-26T08:44:13.494 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T08:44:13.495 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-26T08:44:13.505 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-26T08:44:13.506 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-26T08:44:13.528 DEBUG:teuthology.orchestra.run.smithi043:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-26T08:44:13.780 DEBUG:teuthology.orchestra.run.smithi052:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-26T08:44:14.012 DEBUG:teuthology.orchestra.run.smithi183:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-26T08:44:14.233 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T08:44:14.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T08:44:14.342 DEBUG:teuthology.orchestra.run.smithi052:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T08:44:14.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T08:44:14.449 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T08:44:14.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T08:44:14.550 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-26T08:44:14.562 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-26T08:44:14.590 INFO:teuthology.task.internal:Duration was 2024.926735 seconds 2024-03-26T08:44:14.590 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-26T08:44:14.620 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-26T08:44:14.621 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-26T08:44:14.629 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-26T08:44:14.637 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-26T08:44:14.776 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T08:44:14.797 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T08:44:14.804 INFO:teuthology.orchestra.run.smithi052.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T08:44:15.238 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-26T08:44:15.238 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi043.front.sepia.ceph.com 2024-03-26T08:44:15.238 DEBUG:teuthology.orchestra.run.smithi043:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-26T08:44:15.306 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi052.front.sepia.ceph.com 2024-03-26T08:44:15.306 DEBUG:teuthology.orchestra.run.smithi052:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-26T08:44:15.383 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi183.front.sepia.ceph.com 2024-03-26T08:44:15.383 DEBUG:teuthology.orchestra.run.smithi183:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-26T08:44:15.457 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-26T08:44:15.458 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-26T08:44:15.465 DEBUG:teuthology.orchestra.run.smithi052:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-26T08:44:15.473 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-26T08:44:15.574 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-26T08:44:15.574 DEBUG:teuthology.orchestra.run.smithi043:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-26T08:44:15.580 DEBUG:teuthology.orchestra.run.smithi052:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-26T08:44:15.586 DEBUG:teuthology.orchestra.run.smithi183:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-26T08:44:16.066 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-26T08:44:16.078 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-26T08:44:16.078 DEBUG:teuthology.orchestra.run.smithi043:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-26T08:44:16.175 DEBUG:teuthology.orchestra.run.smithi052:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-26T08:44:16.279 DEBUG:teuthology.orchestra.run.smithi183:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-26T08:44:16.379 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-26T08:44:16.393 DEBUG:teuthology.orchestra.run.smithi043:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-26T08:44:16.400 DEBUG:teuthology.orchestra.run.smithi052:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-26T08:44:16.408 DEBUG:teuthology.orchestra.run.smithi183:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-26T08:44:16.511 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = core 2024-03-26T08:44:16.512 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = core 2024-03-26T08:44:16.517 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern = core 2024-03-26T08:44:16.588 DEBUG:teuthology.orchestra.run.smithi043:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-26T08:44:16.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T08:44:16.644 DEBUG:teuthology.orchestra.run.smithi052:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-26T08:44:16.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T08:44:16.703 DEBUG:teuthology.orchestra.run.smithi183:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-26T08:44:16.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T08:44:16.758 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-26T08:44:16.769 INFO:teuthology.task.internal:Transferring archived files... 2024-03-26T08:44:16.770 DEBUG:teuthology.misc:Transferring archived files from smithi043:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622859/remote/smithi043 2024-03-26T08:44:16.771 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-26T08:44:16.893 DEBUG:teuthology.misc:Transferring archived files from smithi052:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622859/remote/smithi052 2024-03-26T08:44:16.894 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-26T08:44:17.007 DEBUG:teuthology.misc:Transferring archived files from smithi183:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622859/remote/smithi183 2024-03-26T08:44:17.007 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-26T08:44:17.116 INFO:teuthology.task.internal:Removing archive directory... 2024-03-26T08:44:17.116 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-26T08:44:17.124 DEBUG:teuthology.orchestra.run.smithi052:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-26T08:44:17.133 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-26T08:44:17.188 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-26T08:44:17.202 INFO:teuthology.task.internal:Not uploading archives. 2024-03-26T08:44:17.202 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-26T08:44:17.238 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-26T08:44:17.238 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-26T08:44:17.246 DEBUG:teuthology.orchestra.run.smithi052:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-26T08:44:17.254 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-26T08:44:17.299 INFO:teuthology.orchestra.run.smithi043.stdout: 393251 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 26 08:44 /home/ubuntu/cephtest 2024-03-26T08:44:17.307 INFO:teuthology.orchestra.run.smithi052.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 26 08:44 /home/ubuntu/cephtest 2024-03-26T08:44:17.311 INFO:teuthology.orchestra.run.smithi183.stdout: 393268 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 26 08:44 /home/ubuntu/cephtest 2024-03-26T08:44:17.314 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-26T08:44:17.326 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-26T08:44:17.388 INFO:teuthology.nuke:Checking targets against current locks 2024-03-26T08:44:17.412 DEBUG:teuthology.nuke:shortname: smithi043 2024-03-26T08:44:17.412 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T08:44:17.437 DEBUG:teuthology.nuke:shortname: smithi052 2024-03-26T08:44:17.437 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T08:44:17.443 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622859', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 08:01:41.334318', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMH1jU9U0sU6ZzsJUSlAx35h4aq/E8tu/suOdSwcUCjbL9GoT5tueabtH+nGQtab6zRcT5my8TA1K63WPLCVWjo='} 2024-03-26T08:44:17.460 DEBUG:teuthology.nuke:shortname: smithi183 2024-03-26T08:44:17.461 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T08:44:17.470 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi052.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622859', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 08:01:41.335277', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPDmoGQ8x1m+DjjCuVi05h5AL0D32ZkYNW7vpMtjqsfAkKVxWnwHq4rVn+0LsMX/EWuhyE38tYEc0w8fR6weaFk='} 2024-03-26T08:44:17.482 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622859', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 08:01:41.332967', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHnsG7FtOLb0DVBiNZN4/949cs/40B5XkLLnMRNH0GgTSj6HsaOofln4VgLEUFw2+lTtox1LiudHiKLGWb2qHGQ='} 2024-03-26T08:44:17.498 INFO:teuthology.orchestra.console.smithi043:Power off 2024-03-26T08:44:17.499 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T08:44:17.521 DEBUG:teuthology.orchestra.console.smithi043:power off output: Chassis Power Control: Down/Off 2024-03-26T08:44:17.521 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T08:44:17.538 INFO:teuthology.orchestra.console.smithi052:Power off 2024-03-26T08:44:17.538 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T08:44:17.557 INFO:teuthology.orchestra.console.smithi183:Power off 2024-03-26T08:44:17.557 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T08:44:17.565 DEBUG:teuthology.orchestra.console.smithi043:check power output: Chassis Power is on 2024-03-26T08:44:17.565 DEBUG:teuthology.orchestra.console.smithi052:power off output: Chassis Power Control: Down/Off 2024-03-26T08:44:17.565 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T08:44:17.578 DEBUG:teuthology.orchestra.console.smithi183:power off output: Chassis Power Control: Down/Off 2024-03-26T08:44:17.578 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T08:44:17.598 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is on 2024-03-26T08:44:17.601 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2024-03-26T08:44:21.566 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T08:44:21.599 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T08:44:21.607 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T08:44:21.681 DEBUG:teuthology.orchestra.console.smithi043:check power output: Chassis Power is on 2024-03-26T08:44:21.709 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is on 2024-03-26T08:44:21.717 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2024-03-26T08:44:25.685 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T08:44:25.710 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T08:44:25.719 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T08:44:25.795 DEBUG:teuthology.orchestra.console.smithi043:check power output: Chassis Power is off 2024-03-26T08:44:25.820 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is off 2024-03-26T08:44:25.828 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2024-03-26T08:44:25.896 INFO:teuthology.orchestra.console.smithi043:Power off completed 2024-03-26T08:44:25.921 INFO:teuthology.orchestra.console.smithi052:Power off completed 2024-03-26T08:44:29.830 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T08:44:29.940 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off 2024-03-26T08:44:30.041 INFO:teuthology.orchestra.console.smithi183:Power off completed 2024-03-26T08:44:30.146 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/iozone}} duration: 2024.926735162735 failure_reason: '"2024-03-26T08:27:51.056308+0000 mon.a (mon.0) 631 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log' flavor: default mon.a-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 mon.b-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 mon.c-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 owner: mchangir success: false 2024-03-26T08:44:30.147 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T08:44:30.225 INFO:teuthology.run:FAIL